Model Routing Policy
Written rules that decide which AI model handles each step of a job — weighing privacy, speed, cost, and quality — so switching models is a settings change, not a rebuild.
Think of a shipping desk choosing a courier for each package: the fragile one goes premium, the routine one goes standard, and the confidential one goes with the bonded carrier. A routing policy is that desk for AI calls — each step of a workflow states what it needs, and the system picks the model that fits.
The problem, in plain words
Your workflows all call one AI model, because that's how they were built. Then the provider raises prices, or retires that model, or a new one comes out that's better for half your tasks — and you discover the model's name is hardcoded in thirty places. Meanwhile, a simple yes/no sorting step is paying premium-model prices, a time-sensitive step is waiting on a slow model, and a step touching sensitive data is quietly sending it to the same outside service as everything else. Nobody chose any of this. It just accumulated.
What we set up
We move the choice of model out of the code and into a policy at the gateway (the single doorway all your AI calls pass through). Each workflow step declares what it needs: how fast the answer must come (a latency budget), how sensitive the data is (a privacy class), how good the output must be (a quality floor), and how much it may cost (a cost ceiling). The gateway matches those needs against a catalog of available models — with current pricing and limits — and picks the route. If a provider goes down or degrades, a declared fallback chain takes over. Every call is traced, so you can see exactly which model served which step, at what cost.
How it works, step by step
- Each step states its needs
Speed, privacy, quality, cost — declared per step, in plain terms, where anyone can read them.
- The gateway keeps the catalog
Which models exist, what they cost, what they're good at, where they run — maintained in one place instead of thirty.
- Calls are matched to models
At run time, the gateway resolves each step's needs against the catalog and routes the call. The workflow code never names a model.
- Trouble triggers the fallback
An outage or a quality dip shifts calls down the declared fallback chain automatically — a routing event, not an incident.
- Every route is on record
Traces show which model handled which call, so cost and quality can be attributed and tuned step by step.
What changes for you
Before: a model retirement or price change means finding and editing every place the model is named, then hoping. After: it's an edit to the routing policy — one change, visible, reversible. Cost tuning becomes an experiment (route this step to a cheaper model, watch the quality numbers), and a regulator requiring in-region processing becomes a routing rule instead of a crisis. What it won't do: it won't decide your trade-offs for you. Whether a step values speed over cost, or privacy over quality, is still your call — the policy just makes that call explicit, enforced, and easy to change.