Reranking
A second-stage retrieval step where a cross-encoder model re-scores the top candidates from the first stage — the configuration that consistently improves answer quality on real corpora.
Production AI is a system: context, tools, permissions, traces, evals, and feedback loops around the model.
What it is
Reranking is the practice of running two retrieval stages. Stage one: a fast but coarse retriever (BM25, vector search, hybrid) finds a candidate set of, say, the top 50 chunks. Stage two: a slower but more accurate cross-encoder model re-scores those 50 against the query and returns the top 5 or 10. The result is consistently better than what stage one alone produces.
Why it matters
First-stage retrievers are tuned for recall — bring back anything plausible. They're not great at picking the best 3 out of 50. A cross-encoder reranker is — but it would be too slow to run on the whole corpus, which is why two stages exist.
How it works
Common rerankers: Cohere Rerank, BGE Rerank, Voyage Rerank. The reranker reads the query and each candidate chunk together (not separately), so it can judge relevance with full context. The latency cost is real and the quality lift is also real — workflows balance the two against their budget.
Related resources
How an AI agent finds the right document, chunk, or row to ground its answer in — and why the part that matters is the pipeline around the database, not the database itself.
The search behind your AI combines two ways of looking — by exact words and by meaning — then gives the best candidates a careful second read, so answers that exist in your documents actually get found.
A deliberate, tested decision about how many search results get an expensive careful second read, and with which model — tuned per workflow instead of copied from a tutorial.
The pattern where an AI agent retrieves relevant context from your data before generating an answer — instead of relying only on what the model learned during training.