Voice Agents
AI agents you talk to — streaming speech-to-text, an agent turn, and streaming text-to-speech composed into a full-duplex loop where interruptions and latency are measured in milliseconds.
Production AI is a system: context, tools, permissions, traces, evals, and feedback loops around the model.
What it is
A voice agent is a realtime pipeline: microphone audio streams to speech-to-text, the agent processes the transcribed turn, and text-to-speech streams audio back — with barge-in (the caller interrupts playback and the agent stops) and end-of-turn detection making it feel like conversation. Two architectures compete: unified speech-to-speech models such as OpenAI Realtime, and composed pipelines that chain dedicated STT, an LLM agent, and dedicated TTS.
Why it matters
Voice remains the highest-volume channel in many operations, and its latency budget is unforgiving: the caller expects first audio in under a second. That budget forces architectural decisions — streaming everywhere, sentence-level TTS chunking, interrupt handling — that text chat never faces.
How we build it
A composed pipeline over WebSockets: streaming speech-to-text (Deepgram-class realtime STT) with tuned end-of-turn thresholds, the same governed agent runtime as text chat — same tools, same permissions, same traces — and sentence-chunked streaming text-to-speech with barge-in that cancels playback the moment the caller speaks. Keeping voice on the same runtime means the eval and governance story does not fork per channel.
Related resources
An interaction mode where a client maintains a persistent WebSocket connection to the AI platform — for low-latency streaming, real-time voice, multi-turn collaboration, and live tool feedback.
The execution engine that turns an AI agent from a chat-window demo into a long-running, event-driven, restartable process you can trust with real operations.
The safety checks and policy enforcements that sit around an AI agent's inputs and outputs — content filters, scope enforcers, PII redactors, refusal patterns, and tool-call validators.
The defined states an AI agent task moves through — received, in-progress, awaiting-approval, completed, failed, expired — and the events that drive each transition.