Hybrid Retrieval
The search behind your AI combines two ways of looking — by exact words and by meaning — then gives the best candidates a careful second read, so answers that exist in your documents actually get found.
Think of looking for a song. Sometimes you know the exact title, and a word-for-word search wins. Sometimes you can only hum the melody, and you need someone who matches on feel. A good record-store clerk uses both — and then listens closely to the top few before handing you one.
The problem, in plain words
You ask your AI assistant about error code E-4402, and it confidently returns something about a vaguely similar error — not that one. The right page was in your documents the whole time. This happens because search by meaning alone (vector search) is great at 'close in spirit' but surprisingly bad at exact codes, product names, and part numbers. Search by exact words alone has the opposite flaw: it nails the precise term but misses a question phrased differently from how the document was written. Either way alone, real questions fall through the crack in the middle.
What we set up
Two searches run side by side on every question. One is the classic exact-word search, a method called BM25 (running on OpenSearch or Elasticsearch). The other is vector search (on pgvector, Qdrant, Weaviate, or Pinecone), which matches on what a passage means rather than the words it happens to use. The two result lists are merged with a fusion method (such as reciprocal rank fusion — a fair way to combine two rankings into one). Then a reranker — a more careful model (Cohere Rerank, BGE, or Voyage) — re-reads the top candidates, typically around fifty, against the actual question and reorders them. The mixing weights are not guessed: they are tuned against your eval set, your own library of test questions with known good answers.
How it works, step by step
- A question comes in
From a person, or from an agent that needs facts before it answers.
- Two searches run at once
Exact-word matching and meaning matching, in parallel. Each catches what the other misses.
- The two lists become one
A fusion method merges both rankings, so a passage that scores well on either path stays in the running.
- A careful model re-reads the finalists
The reranker reads the top candidates against the actual question and reorders them — a slow, careful read of a short list.
- The settings are tested, not guessed
Weights and rerank depth are tuned against your own test questions, and re-tuned when your documents change.
What changes for you
Before: the assistant regularly missed answers that hinged on an exact code, a name, or an unusual phrasing — and users learned not to trust it with precise questions. After: both kinds of questions land, and recall on the hard ones goes up, which is a fancy way of saying the answer that exists gets found. What it won't do: it cannot find what isn't there. If the answer is missing from your documents, better search just returns better-organized nothing — that is a documentation gap to fix, and the search results will now make it visible.