LiteLLM
An open-source library and proxy that exposes one uniform API across 100+ LLM providers — and maintains the model-price registry that many cost-attribution stacks rely on.
Production AI is a system: context, tools, permissions, traces, evals, and feedback loops around the model.
What it is
LiteLLM is a Python SDK and proxy server that translates one common interface — completions, embeddings, batch — into each provider's native API. It also ships a continuously maintained price map per model, which is widely reused on its own for cost accounting.
Why it matters
Multi-provider support without N bespoke integrations is the obvious value. The quieter one is cost: accurate spend attribution needs the price at time of call for every model, and maintaining that table by hand is a losing game.
How we use it
Embeddings and batch submissions in our production stack run through LiteLLM, and its price registry drives the per-call USD cost recorded in the usage log. Agent runs themselves go through Pydantic AI with the same usage accounting on top.
Related resources
A single integration point in front of every AI model your applications use — for routing, key rotation, rate limits, fallback, cost attribution, and observability.
The component of an AI platform that routes every model call — choosing the provider, applying rate limits and fallback, attributing cost, and emitting traces — so applications never call providers directly.
The total spend on language-model APIs across an organization — input tokens, output tokens, embeddings, fine-tuning — and the practice of attributing, optimizing, and budgeting it.
How an AI system decides which model to call for each step — based on privacy, cost, latency, quality, and what happens when a provider goes down.