--- title: RAG Retrieval Lab — Precision@k, Recall@k, MRR and nDCG description: Explore Precision@k, Recall@k, MRR and nDCG on a visible ranking; test reranking and keep retrieval quality separate from chunking and overlap cost. keywords: RAG retrieval, precision recall RAG, MRR, nDCG, reranking, RAG chunking, chunk overlap, retrieval evaluation hide: - toc - navigation - footer ---
Tools · RAG · 07

Measure the ranking you retrieve, not only the final answer.

Change k, the relevance threshold and reranking depth on a visible synthetic ranking. Separately, change chunk size and overlap to see how much text you duplicate in the index without pretending that this storage cost predicts semantic retrieval quality.

Precision@krelevant retrieved ÷ k
Recall@krelevant retrieved ÷ total relevant
nDCG@kgraded gain discounted by rank

Ranking evaluation

0 keeps the initial ranking; a larger value reranks only that prefix.
Recall needs a denominator outside the top-k. Use the known relevant count for your query.

Chunking footprint

Must stay below the chunk size.
Precision@k—purity of the top-k
Recall@k—coverage of known relevant items
MRR@k—rank of the first relevant item
nDCG@k—ordering + graded relevance
Relevant inside k—
Estimated chunks—
Indexed tokens—
Overlap duplication—
Stride—
Metric reading—
Chunking reading—

Deliberate separation: the chunking panel computes index size and textual duplication. It does not turn chunk size or overlap into a supposed relevance gain. Demonstrating a gain requires running the retriever against real qrels.

—results inside k are highlighted
Method

Each metric answers a different question.

Precision@k asks what proportion of the first k results clears your relevance threshold. Recall@k asks what fraction of all known relevant items that top-k managed to recover.

Precision@k = relevant_in_top_k / k · Recall@k = relevant_in_top_k / total_relevant

MRR@k depends only on the rank of the first relevant result. It is useful when finding one good piece of evidence early is enough, but it cannot distinguish an excellent ranking from a mediocre one after that first hit.

MRR@k = 1 / rank_of_first_relevant

nDCG@k supports graded relevance. This lab uses gain 2^rel − 1 with logarithmic rank discount, divided by the ideal DCG for the same judgments.

DCG@k = Σ (2^relᵢ − 1) / log₂(i + 1) · nDCG@k = DCG@k / IDCG@k

Reranking. The lab applies a second score only to the prefix you select. This makes a common pattern explicit: a cheap retriever generates candidates, then a more expensive reranker tries to improve their order without rescuing documents that never entered the candidate set.

Chunking. Stride is chunk_size − overlap. Overlapping windows increase indexed tokens; that redundancy can reduce boundary cuts, but it also increases storage and redundant candidates. The lab computes that footprint exactly for the configured corpus and leaves quality as a separate measurement.

Incomplete judgments. Recall requires knowing how many relevant results exist. If your qrels are partial, the denominator is partial too; do not read a high score as absolute coverage of the knowledge base.

Sources: Karpukhin et al. (DPR, EMNLP 2020), Lewis et al. (RAG, 2020), Thakur et al. (BEIR, 2021), and Järvelin & Kekäläinen (2002).

01Context budgetTurn retrieved chunks into an explicit token budget.Tool 02Model evaluationKeep the metric separate from the decision you are trying to make.Concept 03Engineering notesArchitecture and operation of production AI systems.Engineering