Research notes

Signal Extraction

Turning raw conversation transcripts into structured fields — intent, subject, sentiment, CSAT, tool performance, product mentions — that downstream systems can query, dashboard, and act on.

Operating principle

Production AI is not a prompt. It is a system of context, tools, permissions, traces, evals, and feedback loops.

What we extract

Intent (what is the speaker trying to accomplish), subject (what product area or feature is in scope), sentiment and CSAT proxy, satisfaction with the resolution if any, named product or competitor mentions, and tool-performance signals (the agent retried three times before giving up, the API returned 503, the answer cited a stale document).

  • Intent and subject classification
  • Sentiment and CSAT proxy scoring
  • Named entity extraction (products, accounts, tools)
  • Tool-performance and failure signals

How extraction is run

Task-appropriate models routed through the platform gateway. Classification and structured extraction use smaller, cheaper models with a confirmation pass when confidence is low; reasoning-heavy extraction (resolution quality, intent disambiguation) routes to larger models on a sample. Outputs are versioned so re-running an old listener against an improved extractor produces a deterministic diff.

What downstream uses it for

Failure clustering, eval-case capture, knowledge updates, product signal feeds, and the live operational dashboards that show where the system is leaking. The same extracted fields feed both review tooling (humans triaging clusters) and automated routing (escalate to engineering, route to product, attach to a ticket).

Related resources