How it works

Visceral sits beside your agent’s LLM traffic. It does three things: observe, prove, and optimize — strictly in that order.

​

Observe

The SDK records each LLM call as an event — hashes, token counts (including cached and reasoning tokens), costs, model names, tool names, and timing. Events are metadata only: no prompt or response text, no PII. Content is stripped inside your process, before export — see Privacy & data. From this traffic Visceral reconstructs how your agent actually runs: which calls repeat the same work, which prompt prefixes almost cache but miss, and where the money actually goes. Agents don’t need registering — they appear the moment they report, keyed by the agent_id you pass to wrap().

​

Prove (the gate)

Before Visceral changes anything, a proposed optimization has to pass the gate: Visceral must be able to demonstrate that the cheaper call produces an output-equivalent result for the task at hand. An optimization that can’t clear the gate is never applied. This is what makes Visceral a performance engine rather than a router — it doesn’t gamble on “probably fine.” The first optimization family to clear the gate is cache layout: rewrites that change how a provider caches and bills a request without changing a single token the model reads. Those are output-neutral by construction, so the proof obligation is about profitability and stability — and Visceral mints a rule only where your traffic shows the same structure recurring reliably. A second family, tool-call reuse, applies the same proof discipline to read-only tool calls: where the same call recurs and its result is safe to reuse, Visceral can serve a stored result instead of re-running the tool. You opt in by annotating tools with @tool.

​

Optimize

Proven optimizations ship as rules the SDK fetches and applies locally — see Optimization. Rules are versioned, centrally revocable, and annotated onto every trace they touch, and savings are reported per workspace as estimated_cache_savings_usd.

​

Findings: the waste map

Waste that Visceral can measure but cannot prove safe to auto-fix becomes a finding instead: an X-ray of the opportunity — what’s wasted, where, and why it can’t be fixed automatically (a volatile block breaking an otherwise stable prefix, a provider without a safe rewrite, history too thin to prove against). Findings are surfaced in the dashboard and the findings API; acting on them is always your call.

​

Fail open

The fail-open invariant is enforced at every layer, concretely:

• No key / no backend — wrap() returns your client uninstrumented or the exporter drops telemetry; your calls never wait on Visceral.
• Applying rules — any error mid-rewrite relays the original request unchanged; a failed rule fetch keeps the last snapshot.
• Ingestion — a span that fails to decode or persist is dropped and counted, never taking the batch (or your agent) down with it.