Retrieval & Context

Reranking

Reranking is a second retrieval stage that reorders an initial candidate list by scoring each query-document pair with a more accurate model, usually a cross-encoder, so the most relevant items rise to the top before they reach the generator.

What reranking is

Reranking is the act of taking an ordered list of candidate documents returned by a first-stage retriever and reordering it with a more precise relevance model. The retriever optimizes for recall and speed: it scans millions of items quickly and returns a shortlist, typically the top 100 to 1000, that probably contains the answer. The reranker optimizes for precision on that small shortlist, recomputing a relevance score for each candidate against the query so that the genuinely best items occupy the top positions.

The reason a second stage helps is that the two stages can use different scoring functions with different cost profiles. A first-stage retriever must score every item in the corpus, so it relies on a cheap function such as inner product over precomputed embeddings or term overlap. A reranker only has to score a handful of candidates, so it can afford a slower, more expressive model that reads the query and the document together. The shortlist makes that expense affordable.

In retrieval-augmented generation, reranking sits between retrieval and the language model. After candidates are fetched from a vector database or a keyword index, the reranker reorders them and the top few are placed into the prompt as context. Because the context window is limited and language models attend unevenly across long inputs, putting the most relevant passages first measurably improves answer quality.

Reranking is a precision-focused second pass over a retriever's candidate shortlist.
It assigns a fresh relevance score to each query-document pair and sorts by that score.
It is affordable because it runs only on the shortlist, not the full corpus.
In RAG it decides which passages, and in what order, enter the prompt.

Cross-encoders versus bi-encoders

The architectural distinction at the heart of reranking is the difference between bi-encoders and cross-encoders. A bi-encoder encodes the query and each document separately into fixed vectors, then compares them with a similarity function such as cosine or dot product. Because document vectors can be computed and indexed ahead of time, a bi-encoder supports fast approximate nearest neighbor search over large corpora, which is why it powers the first stage.

A cross-encoder instead feeds the query and a candidate document into the model together as a single concatenated input and outputs one relevance score. By letting every query token attend to every document token, a cross-encoder captures fine-grained interactions that a bi-encoder's independent embeddings cannot. The same joint encoding that makes a cross-encoder more accurate is exactly what makes it impossible to precompute: the score depends on the specific query-document pair, so it cannot be cached and demands a fresh forward pass for every pair at query time.

This tradeoff defines the standard division of labor. Bi-encoders are fast but less accurate and serve as retrievers. Cross-encoders are accurate but expensive and serve as rerankers. The 2019 paper by Rodrigo Nogueira and Kyunghyun Cho, Passage Re-ranking with BERT, popularized using a BERT cross-encoder for this step and topped the MS MARCO passage ranking leaderboard at the time.

python

from sentence_transformers import CrossEncoder

# A BERT-based cross-encoder trained on MS MARCO passage ranking
model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2")

query = "How does reranking improve retrieval?"
candidates = [
    "Reranking reorders retrieved passages with a more precise model.",
    "Berlin is the capital of Germany.",
    "A cross-encoder scores the query and passage jointly.",
]

# rank() returns candidates sorted by relevance, highest first
rankings = model.rank(query, candidates)
for r in rankings:
    print(f"{r['score']:.3f}\t{candidates[r['corpus_id']]}")

A minimal cross-encoder rerank with the sentence-transformers CrossEncoder API. rank() scores each query-candidate pair and returns them ordered by relevance.

Bi-encoders embed query and document independently, enabling precomputed, indexable vectors.
Cross-encoders encode the pair jointly, so every query token attends to every document token.
Joint attention makes cross-encoders more accurate but forbids precomputation.
Standard practice: the bi-encoder retrieves for recall, the cross-encoder reranks the shortlist for precision.

The retrieve-then-rerank pipeline

The two-stage retrieve-then-rerank pattern is the workhorse of modern search and RAG systems. Stage one casts a wide net: a bi-encoder over a vector database, a keyword index such as BM25, or a hybrid of both returns a candidate set sized for recall, often the top 100 to 200. The goal of this stage is simply to ensure the relevant items are somewhere in the shortlist, even if their order is rough.

Stage two refines that shortlist. The reranker scores each candidate against the query and reorders the list, after which only the top-k items, frequently 3 to 10 for a RAG prompt, are kept. The size of the retrieval shortlist and the final top-k are the two knobs that govern the recall-precision-cost balance: a larger shortlist raises the chance the answer was retrieved at all, while a smaller final top-k keeps the prompt focused and cheap.

Each stage corrects a weakness of the other. The retriever alone has good recall but noisy ordering, so the right passage may sit at rank 40. The reranker alone cannot scan a corpus because scoring every document jointly is intractable. Composing them yields high recall from the cheap first stage and high precision from the expensive second stage, which is why production search systems almost always use both.

Stage one (retriever) maximizes recall over the full corpus and returns a shortlist.
Stage two (reranker) maximizes precision on the shortlist and selects the final top-k.
Shortlist size and final top-k are the main recall-precision-cost dials.
The two stages compensate for each other's recall and ordering weaknesses.

How relevance is measured

Reranking quality is judged with ranking metrics that reward putting relevant items near the top. Mean Reciprocal Rank, MRR, looks only at the position of the first relevant result: for one query the reciprocal rank is 1 divided by that position, so a relevant hit at rank 1 scores 1.0, at rank 2 scores 0.5, and so on. MRR averaged over a query set, often truncated as MRR@10, is the primary metric on the MS MARCO passage ranking leaderboard.

Normalized Discounted Cumulative Gain, NDCG, is the other standard metric and supports graded relevance rather than a single binary hit. It discounts the gain of each relevant item by a logarithm of its rank, so a relevant document found lower in the list contributes less, then divides by the ideal ordering's score so the result falls on a 0 to 1 scale and is comparable across queries. NDCG@10 is widely reported for passage reranking, including on TREC Deep Learning evaluations of MS MARCO.

These metrics are how rerankers are benchmarked and compared. A reranker is useful only if it raises NDCG or MRR over the retriever's raw ordering by enough to justify its added latency, so evaluation always weighs the precision gain against the per-query cost of the extra scoring pass.

DCG@k = Σ (i=1..k) relᵢ / log₂(i + 1)    and    NDCG@k = DCG@k / IDCG@k

DCG sums each item's relevance discounted by the log of its rank; NDCG normalizes by the ideal DCG, the score of the best possible ordering, so NDCG@k lies between 0 and 1.

MRR scores the rank of the first relevant hit: 1.0 at rank 1, 0.5 at rank 2.
NDCG handles graded relevance, applies a logarithmic rank discount, and is normalized to 0 to 1.
MS MARCO passage ranking uses MRR@10 as its primary leaderboard metric.
A reranker is judged by its NDCG or MRR gain weighed against added latency.

Latency, precision, and the cost tradeoff

The central tension in reranking is that accuracy and latency pull in opposite directions. A cross-encoder must run one forward pass per candidate, so reranking 100 passages means 100 model evaluations per query, all on the critical path before the answer can be generated. Doubling the shortlist roughly doubles reranking cost, which sets a practical ceiling on how many candidates a low-latency system can afford to rescore.

Several levers manage this cost. Smaller distilled cross-encoders, such as MiniLM-based rerankers, trade a little accuracy for much higher throughput. Trimming the shortlist before reranking reduces the number of forward passes. Hosted reranking APIs, such as Cohere Rerank, move the computation to optimized infrastructure and expose a query-plus-documents call that returns scored, ordered results, which is convenient when self-hosting a cross-encoder is undesirable.

Choosing a configuration is an exercise in matching the precision gain to a latency budget. Interactive applications may rerank only the top 20 to 50 candidates with a small model, while batch or high-stakes pipelines can rerank hundreds with a larger one. The recurring question is whether the improvement in NDCG or MRR is worth the milliseconds it costs, and the answer depends on the corpus, the model, and how much the downstream task suffers from a misordered context.

Cross-encoder cost grows roughly linearly with the number of candidates reranked.
Distilled rerankers and smaller shortlists are the main ways to cut latency.
Hosted APIs such as Cohere Rerank offload scoring to optimized infrastructure.
The right setting balances NDCG or MRR gain against a fixed latency budget.

Key takeaways

Reranking is a precision-focused second stage that reorders a retriever's candidate shortlist before it reaches the generator.
Bi-encoders embed query and document separately for fast retrieval; cross-encoders score the pair jointly for higher accuracy.
The retrieve-then-rerank pipeline pairs a high-recall first stage with a high-precision second stage to get both.
Shortlist size and final top-k are the knobs controlling the recall, precision, and cost balance.
NDCG and MRR are the standard metrics, with MRR@10 primary on MS MARCO passage ranking.
Cross-encoder cost scales with the candidate count, so reranker choice is a precision-versus-latency tradeoff.

Frequently asked questions

Retrieval scans the entire corpus with a cheap, fast scorer to return a candidate shortlist optimized for recall. Reranking then rescores only that shortlist with a more accurate model to optimize precision, reordering the candidates so the best ones rise to the top. Retrieval finds candidates; reranking orders them well.

A cross-encoder reads the query and document together, letting every query token attend to every document token, which captures relevance signals that separately encoded bi-encoder vectors miss. This makes it more accurate but too slow to run over a whole corpus. So the bi-encoder retrieves and the cross-encoder reranks the small shortlist where its cost is affordable.

Reranking reorders the retrieved passages so the most relevant ones occupy the top positions that enter the prompt. Because context windows are limited and language models attend unevenly across long inputs, supplying better-ordered and more relevant passages raises answer accuracy. It also lets you keep a smaller, cleaner top-k in the prompt.

The two standard metrics are Mean Reciprocal Rank, which scores the position of the first relevant result, and Normalized Discounted Cumulative Gain, which handles graded relevance with a logarithmic rank discount and normalizes to a 0 to 1 scale. MS MARCO passage ranking uses MRR@10 as its primary leaderboard metric, while NDCG@10 is common on TREC Deep Learning evaluations.

Yes. A cross-encoder runs one forward pass per candidate, so reranking 100 passages means 100 evaluations per query on the critical path. Latency is managed by reranking fewer candidates, using a smaller distilled reranker, or calling a hosted reranking API. The gain in NDCG or MRR has to justify the added milliseconds.

A common pattern is to retrieve the top 100 to 200 candidates and rerank them down to a final top-k of roughly 3 to 10 for a RAG prompt. A larger shortlist raises the chance the relevant item was retrieved at all, while a smaller final top-k keeps the prompt focused and cheap. The exact numbers depend on your corpus, reranker speed, and latency budget.

Put the idea into practice

MemX is an AI memory agent built on these ideas: store anything, skip the folders, and find it again by asking in plain English.

Try MemX Free