Retrieval & Context

Hybrid Search

By Arpit Tripathi, Founder

Hybrid search is a retrieval strategy that runs both lexical (sparse, keyword-matching such as BM25) and semantic (dense, vector-similarity) search, then fuses the two ranked lists into one result set so each method covers the other's blind spots.

What Hybrid Search Is

Hybrid search combines two retrieval paradigms that have historically been studied separately. The first is sparse, or lexical, retrieval: it scores documents by exact and near-exact term overlap with the query. The canonical sparse method is BM25, a bag-of-words ranking function from the probabilistic relevance framework developed by Stephen Robertson, Karen Sparck Jones, and others. The second is dense retrieval: queries and documents are mapped to fixed-length vectors by a neural encoder, and relevance is measured by vector similarity such as cosine or dot product. Dense Passage Retrieval, introduced by Karpukhin et al. at EMNLP 2020, popularized the dual-encoder form of this approach.

Neither method dominates the other across all queries. Sparse retrieval excels at exact identifiers, rare tokens, product codes, names, and acronyms, where a single literal match is decisive. Dense retrieval excels at paraphrase and synonymy, matching a query to passages that share meaning but no surface words. Running both and merging the results yields a ranking that is more reliable than either alone, which is the practical motivation for hybrid search in retrieval-augmented systems.

The term refers to the whole pipeline: two independent retrievers produce two ranked candidate lists, and a fusion step combines them into a single ordering. The fusion step is what distinguishes hybrid search from simply running two searches side by side. The rest of this entry covers why each method misses results, how fusion works, and what production systems implement.

  • Sparse retrieval (BM25) ranks by term overlap and rewards exact, rare tokens.
  • Dense retrieval ranks by vector similarity and rewards semantic closeness.
  • Hybrid search runs both retrievers and fuses their ranked lists into one.
  • Fusion is the defining step: run BM25 and a vector index without merging their rankings and you have two searches that disagree, not one hybrid search.

Sparse Retrieval and BM25

BM25 scores a document by summing, over each query term, an inverse-document-frequency weight multiplied by a saturated term-frequency component that is normalized by document length. Term-frequency saturation means that the twentieth occurrence of a word adds far less than the second, controlled by the parameter k1. Length normalization, controlled by b, prevents long documents from scoring highly merely because they contain more words. Typical defaults are k1 between 1.2 and 2.0 and b around 0.75.

The strength of BM25 is precision on literal matches. A query containing an exact part number, a function name, or an unusual surname will surface documents that contain that exact string, even if the model has never seen the token before. Because it operates on an inverted index, BM25 is fast at scale and fully interpretable: the score decomposes into per-term contributions.

Its limitation is the vocabulary mismatch problem. BM25 has no notion that car and automobile mean the same thing, or that a query about lowering cloud costs should match a passage about reducing infrastructure spend. If the query and the relevant document share no surface terms, BM25 assigns a score near zero and the document is missed entirely. This is the gap dense retrieval is meant to fill.

score(D,Q) = Σ_i IDF(qᵢ) · ( f(qᵢ,D)·(k₁+1) ) / ( f(qᵢ,D) + k₁·(1 − b + b·|D|/avgdl) )
Okapi BM25. f(qᵢ,D) is the frequency of query term qᵢ in document D, |D| is the document length, avgdl is the average document length, and IDF(qᵢ) = ln( (N − n(qᵢ) + 0.5) / (n(qᵢ) + 0.5) + 1 ) with N documents and n(qᵢ) containing the term.
  • k1 controls term-frequency saturation; typical values are 1.2 to 2.0.
  • b controls document-length normalization; a common default is 0.75.
  • IDF down-weights common terms and up-weights rare, discriminating ones.
  • BM25 fails on vocabulary mismatch: synonyms and paraphrases score near zero.

Dense Retrieval and Its Blind Spots

Dense retrieval encodes the query and each document into vectors using a learned encoder, usually a transformer such as a BERT-based dual encoder. Relevance becomes a geometric quantity: documents whose vectors lie close to the query vector are retrieved, typically via approximate nearest neighbor search over a vector index. Because the encoder is trained on supervision signals, it learns that semantically related phrases map to nearby regions of the embedding space, which is precisely what BM25 cannot do.

Dense Passage Retrieval reported that its dense retriever outperformed a strong Lucene-BM25 baseline by 9 to 19 percent absolute on top-20 passage retrieval accuracy across several open-domain question answering benchmarks. That result helped establish dense retrieval as a default building block for modern retrieval-augmented generation.

Dense methods have their own failure modes. They can miss exact-match queries: a rare token, a specific error code, or an out-of-vocabulary identifier may not be well represented in the embedding space, so the literally correct document ranks below semantically similar but wrong ones. Encoders also generalize poorly to domains far from their training distribution, and their scores are opaque compared with BM25. These complementary weaknesses, sparse failing on synonymy and dense failing on exact tokens, are exactly why combining them helps.

  • Dense retrieval maps query and documents to vectors and ranks by similarity.
  • It captures synonymy and paraphrase that exact-match methods cannot.
  • DPR beat BM25 by 9 to 19 percent absolute on top-20 retrieval accuracy.
  • Dense methods can miss rare tokens, identifiers, and out-of-domain terms.

Fusion Methods: RRF and Convex Combination

Once two ranked lists exist, they must be merged. The two dominant approaches are score-based convex combination and rank-based Reciprocal Rank Fusion. A convex combination computes a weighted sum of the two methods' scores, for example alpha times the dense score plus (1 minus alpha) times the BM25 score. The difficulty is that BM25 scores and vector-similarity scores live on different, unbounded scales, so they must be normalized before they can be added, and the normalization choice strongly affects results.

Reciprocal Rank Fusion sidesteps the scaling problem by ignoring raw scores and using only rank positions. It was introduced by Cormack, Clarke, and Buettcher in the SIGIR 2009 paper Reciprocal rank fusion outperforms condorcet and individual rank learning methods. For each document, RRF sums 1 divided by (k plus its rank) across the lists it appears in, where k is a constant that dampens the influence of top ranks; a value of 60 is the common default and is the default rank_constant in Elasticsearch. A document near the top of either list scores well, and a document present in both lists receives an additive boost.

RRF is attractive because it needs no score normalization and no tuning of relative weights, which makes it a sturdy default. Convex combination can outperform RRF when the two score distributions are well calibrated and the weight is tuned for the corpus, because it preserves the magnitude of confidence rather than collapsing everything to rank. In practice many systems offer both and let the application choose.

RRF(d) = Σ_{r ∈ R} 1 / ( k + rankᵣ(d) ), k = 60 (typical)
Reciprocal Rank Fusion. For document d, sum over each ranked list r in the set R the reciprocal of k plus the rank of d in that list (best rank = 1). Documents appearing high in multiple lists accumulate the largest fused score.
  • Convex combination adds normalized scores: alpha·dense + (1 − alpha)·sparse.
  • RRF fuses by rank only, avoiding the cross-scale normalization problem.
  • RRF's constant k (often 60) dampens the dominance of the very top ranks.
  • Tuned convex combination can beat RRF when both score scales are calibrated.

Hybrid Search in Practice

Hybrid search is now a first-class feature in major search and vector engines. Elasticsearch provides a native rrf retriever and also supports a linear (weighted) combination of keyword and vector results. OpenSearch supports hybrid queries with score normalization and combination. Weaviate exposes a hybrid query with an alpha parameter that interpolates between keyword and vector search, where alpha equal to 1 is pure vector and alpha equal to 0 is pure keyword, and offers both ranked fusion and relative score fusion, with relative score fusion the default since version 1.24. PostgreSQL with the pgvector extension can implement hybrid search by combining its full-text search with vector similarity, with the fusion logic written in SQL.

A common production pattern places hybrid search as the candidate-generation stage of a two-stage pipeline. Hybrid retrieval produces a broad, high-recall candidate set, and a cross-encoder reranker then reorders the top candidates for precision. Fusion handles the recall side cheaply, and reranking handles the final ordering where accuracy matters most.

Choosing parameters is corpus-dependent. The weight between sparse and dense, the RRF constant, the number of candidates pulled from each retriever, and whether to rerank all interact. The code below shows the minimal logic of RRF fusion over two ranked lists, which is the core operation regardless of which engine performs it.

python
def reciprocal_rank_fusion(ranked_lists, k=60):
    """Fuse multiple ranked lists of doc ids into one RRF ordering.
    ranked_lists: list of lists, each ordered best-first."""
    scores = {}
    for ranked in ranked_lists:
        for rank, doc_id in enumerate(ranked, start=1):
            scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + rank)
    return sorted(scores, key=scores.get, reverse=True)

# Example: BM25 list and a dense (vector) list
bm25_results  = ["doc3", "doc1", "doc7", "doc2"]
vector_results = ["doc1", "doc4", "doc3", "doc9"]

fused = reciprocal_rank_fusion([bm25_results, vector_results], k=60)
print(fused)  # doc1 and doc3 rise because they appear in both lists
Minimal Reciprocal Rank Fusion of two ranked lists. Each document accumulates 1/(k+rank) from every list it appears in; documents ranked high in both the sparse and dense results win. Here doc1 wins with 1/(60+2) + 1/(60+1) = 0.0325, edging out doc3 at 0.0323, because both appear in each list while singletons like doc4 score only 1/62 = 0.0161. This is the same computation Elasticsearch's rrf retriever performs server-side.
  • Elasticsearch has a native rrf retriever plus linear weighted combination.
  • Weaviate uses an alpha parameter and defaults to relative score fusion.
  • pgvector enables hybrid search by combining Postgres full-text and vectors.
  • Hybrid search commonly feeds a reranker in a two-stage retrieval pipeline.

When Hybrid Search Helps and When It Does Not

Hybrid search pays off most when a corpus mixes natural-language passages with literal identifiers: technical documentation with code symbols, e-commerce catalogs with SKUs, or legal text with statute numbers. In these settings dense retrieval handles the conceptual queries and BM25 catches the exact-match queries that embeddings blur. Empirically, hybrid configurations frequently recover relevant documents that either method alone ranks below the cutoff.

The cost is added complexity. Two indexes must be built and kept in sync, two retrievers run per query, and fusion adds a tuning surface. If a corpus is purely conversational with no important exact-match tokens, a well-tuned dense retriever may match a hybrid setup at lower operational cost. Conversely, for short keyword-style lookups over structured fields, BM25 alone can be sufficient.

Hybrid search is therefore best understood as a recall-oriented foundation rather than a universal upgrade. It widens the candidate set so that downstream stages, including reranking and the generator in a RAG pipeline, have the right documents to work with. The decision to adopt it should rest on whether the query and document distributions actually exhibit both lexical and semantic relevance signals.

  • Hybrid helps most when text mixes prose with identifiers, codes, or names.
  • It improves recall by catching documents one retriever alone would drop.
  • The cost is two synced indexes, two queries, and extra fusion tuning.
  • Pure conversational or pure keyword corpora may not need both retrievers.

Key takeaways

  • Hybrid search runs sparse (BM25) and dense (vector) retrieval, then fuses the two ranked lists into one.
  • BM25 excels at exact and rare-token matches but fails on synonyms and paraphrase (vocabulary mismatch).
  • Dense retrieval captures meaning but can miss literal identifiers and out-of-domain terms.
  • Reciprocal Rank Fusion fuses by rank using a sum of 1/(k+rank), with k commonly 60, avoiding score-scale issues.
  • Convex combination adds normalized scores and can beat RRF when both score distributions are well calibrated.
  • Elasticsearch, OpenSearch, Weaviate, and pgvector all support hybrid search, often feeding a downstream reranker.

Frequently asked questions

Semantic search uses dense vector similarity alone to match meaning, while hybrid search runs semantic search together with lexical keyword search such as BM25 and fuses both result lists. Hybrid search adds the exact-match strength of keyword retrieval on top of semantic matching. It is a superset of semantic search rather than a replacement for it.
Reciprocal Rank Fusion (RRF) merges multiple ranked lists by giving each document a score equal to the sum of 1 divided by (k plus its rank) across the lists in which it appears. The constant k dampens how much the very top ranks dominate; a value of 60 is the common default and is the default rank_constant in Elasticsearch. RRF was introduced by Cormack, Clarke, and Buettcher at SIGIR 2009.
BM25 scores documents by overlap of query terms, so if a relevant document expresses the same idea with different words, it shares no surface terms with the query and scores near zero. This is the vocabulary mismatch problem. Dense retrieval addresses it by matching on learned meaning rather than literal tokens.
RRF needs no score normalization or weight tuning, which makes it a sturdy default that works well out of the box. A tuned convex (weighted) combination can outperform RRF when the sparse and dense score distributions are well calibrated, because it preserves the magnitude of each method's confidence. Many engines offer both so the application can choose.
Elasticsearch offers a native rrf retriever and linear combination, OpenSearch supports hybrid queries with score normalization, and Weaviate exposes a hybrid query with an alpha parameter and both ranked and relative-score fusion. PostgreSQL with pgvector can implement hybrid search by combining full-text search with vector similarity in SQL. Most pair naturally with a reranking stage.
Hybrid search and reranking solve different problems: fusion widens recall cheaply, while a cross-encoder reranker improves the precision of the final top results. A common pattern uses hybrid search for candidate generation and a reranker to reorder the top candidates. Whether you need both depends on how much final-ordering accuracy your application requires.

Put the idea into practice

MemX is an AI memory agent built on these ideas: store anything, skip the folders, and find it again by asking in plain English.

Try MemX Free