Tokens
The unit models read, write, and bill in — roughly three-quarters of a word in English — the currency of every context window and every AI invoice.
Production AI is a system: context, tools, permissions, traces, evals, and feedback loops around the model.
What it is
Models don't process words; they process tokens — common fragments of text. 'Understanding' might be two tokens; 'the' is one. A useful rule of thumb: 1,000 tokens is about 750 English words. French runs slightly more tokens for the same text.
Why it matters
Tokens are how everything is measured: the context window is a token limit, latency grows with tokens generated, and the invoice is priced per million tokens — with input, output, and cache-read tokens each at different rates. Cost work in AI is mostly token work.
How it works
Every call's usage log records tokens in and out, by class: fresh input, cached input, output, and reasoning tokens where the model exposes them. Those counts, multiplied by price-at-time-of-call, become the per-workflow cost attribution the operation is managed on.
Related resources
The maximum amount of text a model can consider in one call — instructions, documents, conversation, and tool results all share it, measured in tokens.
The total spend on language-model APIs across an organization — input tokens, output tokens, embeddings, fine-tuning — and the practice of attributing, optimizing, and budgeting it.
Reusing the already-processed prefix of a prompt across calls — system instructions, tool definitions, long documents — so repeated context costs a fraction of full price and returns faster.
The total number of tokens an AI agent has available for instructions, memory, retrieved context, conversation history, and tool results — and how that budget is allocated across them.