Mutual information

Mutual information measures how much learning one variable tells you about another. If a retrieved document carries exactly the facts an answer needs, the mutual information between them is high; if it is irrelevant, it is near zero. It is the uncertainty before, minus the uncertainty after.

This is the rigorous version of the idea that retrieval is a channel: a good document injects the specific bits the answer was missing. It is the same information theory the rest of our research runs on, applied to RAG.