Reranking Policy
A deliberate, tested decision about how many search results get an expensive careful second read, and with which model — tuned per workflow instead of copied from a tutorial.
Think of hiring. You don't run a full interview with every applicant — you shortlist. How many make the shortlist, and how thorough the interview is, depends on the job. Reranking policy is that same decision for AI search: how many candidates get the careful, costly read, and when it's worth it.
The problem, in plain words
Adding a reranker — a careful model that re-reads the top search results and reorders them — usually improves answers. But it costs time and money on every single query. So you copy the defaults from a tutorial: rerank the top fifty, always, with the biggest model. Now your instant chat feels sluggish because every quick question pays for a deep read it didn't need, while your overnight research job skimps in the one place it could afford to be thorough. Nobody actually chose those numbers. They were just there.
What we set up
Each workflow gets its own written reranking policy: how many candidates to re-read (the depth), which reranker model to use, and an early-exit rule (when the first results are clearly strong enough, skip the rest and save the cost). None of it is guessed. Your eval set — the library of test questions with known good answers — runs the same queries under several configurations and measures quality, speed, and cost side by side. You keep the setting that buys the most quality your speed and budget can actually afford: the best point on the trade-off curve (what engineers call the Pareto point).
How it works, step by step
- Each workflow states its budget
How long can a user wait here, and what is one query worth? A live chat and an overnight report give very different answers.
- Several settings are tried for real
The eval set runs the same questions at different depths and with different reranker models.
- Results are measured, not felt
Quality, speed, and cost come out as numbers per configuration — a table you can argue about, instead of vibes.
- Each workflow keeps its best fit
The chat gets a shallow, fast policy; the deep-research path gets a thorough one. Neither pays for the other's needs.
- Obvious cases exit early
When the first results are clearly right, the reranker steps aside — no careful re-read of a question that was never in doubt.
- Policies get re-tested
New documents or a new model version triggers a re-run, so every setting stays earned instead of remembered.
What changes for you
Before: reranking was a single on-or-off switch for everything, set by folklore. After: each workflow runs the configuration the evidence picked for it — fast paths stay fast, careful paths stay careful, and when someone asks why the depth is fifty and not a hundred, there is a measured answer. What it won't do: reranking reorders what search already found. It cannot rescue a search that missed the right document entirely — that is what hybrid retrieval is for.