The failure class this article exists for
Let us start with the strongest measurement. A peer-reviewed study built questions from 24 Wikidata relations — roughly 24 thousand questions about facts whose subject is a proper name.
| Question template | Vector search | Keyword search | Difference |
|---|---|---|---|
| "Who is [X]'s child?" | 19.2% | 82.9% | 63.7 points |
| "Where was [X] born?" | 25.4% | 75.2% | 49.8 points |
| "Where did [X] study?" | 26.4% | 73.0% | 46.6 points |
| On the full dataset | 49.7% | 71.2% | 21.5 points |
And now the contrast that makes this genuinely interesting. The same two methods on another, more general set of questions:
| Method | Accuracy |
|---|---|
| Vector search | 80.1% |
| Keyword search | 64.5% |
The same two techniques, two question types, and the ranking flips completely. In one place the vector is 15.6 points better, in the other the keyword is ahead by 21.5.
The authors explain it in two points:
- Popularity bias. Vector search does well on frequent entities and degrades quickly on rarer ones. Keyword search is indifferent to how often something has occurred.
- Pattern dependence. The vector model has to have seen the question pattern during training. When it was specifically tuned to the "where was X born" pattern, it caught up — to 73.9% against the keyword method's 75.2%.
So vector search understands meaning, but does not recognise the rare name. Your company name, your part number, your error code and your customer's name are all exactly that. How meaning-based search works at all is explained by the embeddings and vector search piece, from the ground up.
What does hybrid search give you?
Hybrid search merges the two result lists. The numbers come from a large provider's own measurement, on three datasets:
| Step | Customer datasets | Public benchmark | Multilingual dataset |
|---|---|---|---|
| keyword → vector | +3.2 | +4.4 | +8.7 |
| vector → hybrid | +4.6 | +3.4 | +0.5 |
| hybrid → re-ranking | +11.7 | +1.6 | +13.2 |
Here is the uncomfortable truth of this article, and it has to be said: hybrid on its own gives a modest improvement. Three to five points, and on the multilingual dataset practically nothing.
The big gain comes from re-ranking — on the same data, +11.7 and +13.2 points.
Another vendor's step-by-step measurement confirms it, on the retrieval failure rate:
| Step | Failure rate | Share of the remainder it removes |
|---|---|---|
| Baseline | 5.7% | — |
| + context added to the chunks | 3.7% | — |
| + keyword branch (hybrid) | 2.9% | 21.6% of what was left |
| + re-ranking | 1.9% | 34.5% of what was left |
Two independent measurements, the same ordering.
So what is hybrid for? Because the two steps solve different problems. Hybrid makes sure the good result gets into the candidate set. Re-ranking makes sure it rises to the top. What never entered the candidate set cannot be brought back by re-ranking — and that is exactly the situation with the proper-name questions above.
Re-ranking: why it works, and why it is expensive
The difference in one sentence: in retrieval the question and the document each get their own separate vector, and the two are compared afterwards. In re-ranking, the question and the document go into the model together, and it returns a single score.
That is far more accurate — and far slower. The example in the official documentation shows it perfectly: comparing ten thousand sentences means scoring roughly 50 million pairs.
| Method | Time |
|---|---|
| Joint scoring | ~65 hours |
| Separate vectors | ~5 seconds |
Roughly a forty-seven-thousandfold difference. That is why every serious system is two-stage: the fast method brings in the top 100–150 candidates, and only those are rescored by the slow, accurate method.
And there is a failure class that only this solves. In the study measuring negation, separate vectors scored 11.1% and joint scoring 50.6% — four and a half times the difference. If your question is "which contract has no non-compete clause", plain vector search is essentially answering blind.
Where hybrid does NOT help
This chapter matters most, because three documented cases show that hybrid search does not always help — and can make things worse.
1. When your users type keywords. From the same provider measurement, the "keyword query" row:
| Keyword | Vector | Hybrid | Hybrid + re-ranking |
|---|---|---|---|
| 79.2 | 11.7 | 61.0 | 66.9 |
Hybrid is 18.2 points worse than plain keyword search — and even with re-ranking it does not catch up. The reason is understandable: the vector branch scores 11.7 here, so it adds noise, and the fusion lets that noise into a good lexical ranking.
In the "exact passage" row of the same table: keyword 51.1, hybrid 51.0. It added nothing.
2. When the fusion weighting is wrong. A peer-reviewed study on a difficult dataset:
| Setting | Result |
|---|---|
| Keyword only | 0.682 |
| Default fusion | 0.675 |
| Fusion tuned on another dataset | 0.621 |
| Well-weighted fusion | 0.699 |
Default fusion is worse than plain keyword search. And with parameters tuned elsewhere it is six points worse. As the authors put it: tuned fusion generalises badly to other datasets.
3. On multilingual data. In the table above, hybrid gave +0.5 points on the multilingual dataset. The whole improvement came from re-ranking.
(This would be especially interesting from a Hungarian point of view, but I found no measurement covering Hungarian — neither for the Hungarian part of the multilingual dataset nor for Hungarian stemming. Conceptually all you can say is that the keyword branch is sensitive to stemming and Hungarian is an agglutinative language — but that is an expectation, not a measurement.)
The right order
A decision sequence follows directly from this, and it differs from what most guides tell you:
1. LOOK AT WHAT THEY ACTUALLY TYPE
This is the one step you cannot skip.
Read a hundred real queries.
2. IF THEY ARE KEYWORDS (part no., error code, name)
-> KEYWORD search, possibly with re-ranking.
Hybrid measurably HURTS here.
3. IF THEY ARE CONCEPTUAL ("how do I resign")
-> VECTOR search + re-ranking.
4. IF THEY ARE MIXED
-> HYBRID + re-ranking. And MEASURE IT, because
on your data it may not improve anything.
5. IF YOU CAN ONLY AFFORD ONE
-> RE-RANKING. On the measurements that is what
gives the bigger gain.
EXCEPT when your failure class is the exact ID:
re-ranking cannot bring that back if it never
made the candidate list at all.
The exception to the fifth point is the most important subtlety in all of this. Re-ranking can only lift what is already on the list. If the right document fell past position one hundred on a part-number search, the best ranker in the world will not help.
What is the fusion itself worth?
A peer-reviewed study compared fusion methods across eight datasets. Two things can be read out of it:
The first: keyword search is not obsolete. On two datasets it beat the vector method — 0.682 against 0.520 on one, 0.689 against 0.558 on the other.
The second: fusion always improves things a little, but not much. On the first dataset the best single method is 0.682 and the best fusion 0.699 — plus 1.7 points. Meanwhile the theoretical ceiling of perfect fusion would be 0.767. So there is still something to win in the choice of fusion function — if you have the capacity to measure.
Common misconceptions
- "Hybrid search is the biggest improvement to a RAG system." The measurements do not support that: in three comparisons out of four, re-ranking gave more. Hybrid solves one specific failure class — but it solves that one dramatically.
- "Keyword search is obsolete." It beat the vector method in two peer-reviewed measurements, by 21.5 points on proper-name questions — and it is decades older.
- "I switch hybrid on and things get better." On keyword queries it made things 18 points worse in the measurement. And bad weighting is worse than having no fusion at all.
- "I will take the fusion parameters that worked elsewhere." Demonstrably does not generalise: the tuned setting fell six points below plain keyword search on another dataset.
- "Re-ranking solves everything." It can only lift what made the candidate set. What was left out is lost.
- "Re-ranking is cheap." It is roughly forty-seven thousand times slower than a vector comparison. That is why it is run only on the top 100–150 candidates.
What to take away
Vector search understands meaning, but not the rare name and not the exact identifier. This is not a fault you can tune away, it is the nature of the method — which is why hybrid search exists at all.
But hybrid is not the biggest win. Two independent vendor measurements produced the same order: hybrid adds a few points, re-ranking more than ten in the vendor data — while the peer-reviewed studies confirm the modest benefit of fusion. If you build one thing, build that one.
And the most useful thing you can do today is not technical: read a hundred real queries from your system. If most of them are part numbers and error codes, switching hybrid search on will make things worse — and you find that out in a morning, not in a quarter. The chunking side is covered by the chunking article, and measurement by the RAG evaluation post, step by step.