The Alignment

Works and notes by Joseph Gabito

Say hello

What is a good RRF score? Hybrid search and the noise floor

Author
Joseph Gabito
Date published
Last updated
Estimated reading time
8 minutes

We’ve all been there. You build a RAG (Retrieval-Augmented Generation) pipeline, dump your documents into a vector database, and feel like a wizard. You search for ‘guidelines for remote work’ and it finds documents about ‘working from home policy.’ It feels like magic.

Then, you put it into production.

A user searches for a specific error code like ERR-502 or a specific product SKU like XJ-900.

The result? Hallucinations, or worse, totally irrelevant documents that just happen to share a similar ‘vibe’ in the embedding space.

The hard truth is that Vector Search is not a silver bullet. It understands meaning, but it is terrible at precision.

The Two Extremes of Search

To understand why your AI agent is failing, you have to look at the two distinct ways computers search for information.

1. The ‘Vibe’ Search (Vector Embeddings)

Vector search turns text into numbers (embeddings). It finds concepts that are semantically close.

Superpower: Understanding synonyms and intent (e.g., ‘monitor’ = ‘display’).

Kryptonite: Exact matches. To a vector model, ‘Version 1.2’ and ‘Version 2.1’ might look nearly identical because they are semantically similar concepts, even though they are completely different technical facts.

2. The ‘Ctrl+F’ Search (BM25 / Keywords)

This is the old-school search logic (like Elasticsearch or Lucene), ranked by BM25. It looks for the exact tokens you typed.

Superpower: Precision. If you search for ERR-502, it finds that exact string.

Kryptonite: Vocabulary mismatch. If you search ‘laptop screen,’ it might miss a document that only uses the word ‘display.’

The Fix: Hybrid Search

If you want a production-grade system, you cannot choose one. You need both. This is called Hybrid Search.

In a Hybrid architecture, every user query triggers two parallel searches:

Vector Search scans for semantic meaning.

BM25 Search scans for exact keywords.

You get two lists of results. One captures the intent, the other captures the specifics.

The ‘Merging’ Problem

Now you have a new engineering problem.

Vector search returns a Cosine Similarity score (usually 0.0 to 1.0).

BM25 returns a Relevance Score (which can be 5.0, 15.0, or 42.0 depending on document length and frequency).

You cannot compare these numbers. A 0.8 in vectors is not ‘better’ or ‘worse’ than a 12.0 in BM25. They are apples and oranges.

Enter Reciprocal Rank Fusion (RRF)

The solution is to ignore the scores and look at the rank.

Reciprocal Rank Fusion (RRF) doesn’t care that Document A got a score of 0.89. It cares that Document A was the #1 result. It takes the rankings from both lists and fuses them into a new, unified score.

If a document appears at the top of both lists, it skyrockets to the top of the final list. If it appears in only one, it gets pushed down.

But… What is a ‘Good’ RRF Score?

Congratulations, you have implemented Hybrid Search! But you have introduced a final, tricky variable.

RRF outputs abstract scores that look like 0.0163 or 0.032.

Unlike Cosine Similarity, where we know 0.85 is usually ‘good,’ RRF scores are unintuitive.

Is 0.016 a strong match or noise?

At what threshold should you cut off the results to prevent hallucinations?

Most developers guess. They pick a random number like 0.02 and hope for the best. Do not do that.

There is a mathematical way to calculate exactly where the ‘Noise Floor’ ends and ‘High Confidence’ begins.

Finding the Noise Floor

Start with the formula. For every document, RRF adds up one small fraction per list it appears in:

RRF(d) = Σ 1 / (k + rank_i(d))

Here rank_i(d) is the document’s position in list i, and k is a constant that softens the gap between the top ranks. The original RRF paper fixed k = 60 during a pilot study and found the exact value ‘not critical.’ Most systems still use it, including Elasticsearch and Azure AI Search.

Notice what is missing: the original scores. RRF only looks at ranks, so its range is fixed. You can calculate every possible score before you run a single query.

Look at the two extremes with k = 60:

  • #1 in only one list: 1 / 61 = 0.0164
  • #1 in both lists: 2 / 61 = 0.0328

Those are the 0.0163 and 0.032 from earlier. They are not random. They are the ceiling of each case.

And that first number is the line you are looking for. A document that appears in only one list can never score above 1 / (k + 1), no matter how high it ranks there. So any score above 0.0164 proves that both searches found the document. The meaning matched and the keywords matched.

One condition keeps the rule exact: each search should return no more than k + 1 results (61 with the default). A document found by both lists at rank 61 or better always scores above the floor. Most pipelines retrieve 20 to 50 per list, so they are already safe.

Make it readable: normalize

Raw RRF scores are hard to talk about. Divide every score by the maximum, 2 / (k + 1), and you get a value from 0 to 1. The noise floor lands at exactly 0.5.

Now the rule fits in one sentence: above 0.5, both searches agree; at or below 0.5, only one of them found it.

In code

Here is the whole idea in a function you can drop into a Python pipeline. It takes the two ranked lists of document ids, best first, and labels every result:

def rrf_with_zones(vector_ids, keyword_ids, k=60):
    """Fuse two ranked lists of document ids (best first) and label each result."""
    if max(len(vector_ids), len(keyword_ids)) > k + 1:
        raise ValueError("Keep each list to k + 1 results so the noise floor stays exact.")
    scores = {}
    for ranked in (vector_ids, keyword_ids):
        for rank, doc_id in enumerate(ranked, start=1):
            scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + rank)
    floor = 1.0 / (k + 1)  # the most a single list can give
    results = []
    for doc_id, score in sorted(scores.items(), key=lambda item: item[1], reverse=True):
        results.append({
            "id": doc_id,
            "score": round(score, 4),
            "normalized": round(score / (2 * floor), 2),
            "zone": "confirmed" if score > floor else "single-source",
        })
    return results


vector_hits = ["remote-work-policy", "wfh-guide", "office-hours"]
keyword_hits = ["err-502-runbook", "remote-work-policy"]
for row in rrf_with_zones(vector_hits, keyword_hits):
    print(row)

Running it prints:

{'id': 'remote-work-policy', 'score': 0.0325, 'normalized': 0.99, 'zone': 'confirmed'}
{'id': 'err-502-runbook', 'score': 0.0164, 'normalized': 0.5, 'zone': 'single-source'}
{'id': 'wfh-guide', 'score': 0.0161, 'normalized': 0.49, 'zone': 'single-source'}
{'id': 'office-hours', 'score': 0.0159, 'normalized': 0.48, 'zone': 'single-source'}

The remote work policy was found by both searches, so it clears the floor. The ERR-502 runbook was found only by keyword search, and lands exactly on 0.5: the best score a single list can give.

Three Zones Instead of One Threshold

With the floor in place, a single magic cut-off turns into three zones you can reason about:

Confirmed (above 0.5): Both retrievers agree. Hand these to the model first.

Single-source (0.5 and below): One retriever found it, the other did not. This is where exact lookups like ERR-502 usually land, because only BM25 understands the literal string. Keep them, but check that the exact term is really present before you trust them.

Nothing confirmed and nothing exact: Do not let the model guess. Tell the user you could not find it. An honest ‘I don’t know’ is the cheapest hallucination fix there is.

Stop Guessing

Hybrid search gives your agent both meaning and precision. RRF merges them without comparing apples to oranges. And the noise floor tells you exactly which results both searches stand behind.

You do not need to pick 0.02 and hope. The math already drew the line at 1 / (k + 1).

Sources

This article was checked against the sources below on October 28, 2026. The noise floor and the scores in the code example were calculated, not quoted.