> ## Documentation Index
> Fetch the complete documentation index at: https://docs.visceralai.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Optimization

> Opt in with apply=True and Visceral rewrites your prompts' cache layout — never their content.

Observation is the default. Passing `apply=True` opts a wrapped client in to
Visceral's **cache-layout optimizations** — request rewrites the backend has
already proven safe and profitable for your traffic:

```python theme={null}
client = wrap(Anthropic(), agent_id="support-bot", apply=True)
```

Providers with apply support today: **Anthropic** and **OpenAI**. On other
providers the flag is ignored.

## What gets applied

Visceral analyzes your traffic server-side and mints **rules** only where the
same prompt structure recurs enough to prove the rewrite pays for itself. The
SDK fetches your workspace's active rules and applies the ones that match each
outbound request:

<CardGroup cols={2}>
  <Card title="Anthropic — cache breakpoints" icon="anchor">
    Inserts `cache_control` markers at proven-stable prefix boundaries so
    Anthropic's prompt cache actually hits. Markers are billing metadata — the
    tokens the model sees are identical.
  </Card>

  <Card title="OpenAI — cache routing" icon="route">
    Sets `prompt_cache_key` on requests whose prefix matches a proven-stable
    bucket, so repeated prefixes land on the same cache shard instead of
    missing.
  </Card>
</CardGroup>

Both rewrites are **output-neutral by construction**: they change how the
provider caches and bills the request, never the tokens the model reads or the
response your agent receives.

## How rules reach the SDK

* On `wrap(..., apply=True)` the SDK fetches your workspace's rules once, then
  refreshes them in a background thread every 60 seconds
  (`VISCERAL_APPLY_RULES_TTL_SECONDS`).
* The hot path reads an in-memory snapshot — applying rules adds **no network
  call and no lock** to your LLM requests.
* Rules carry a schema version; an SDK never applies a rule version it doesn't
  understand.
* Every applied rewrite is annotated on the trace (`visceral.apply.decision_id`),
  so you can see exactly which optimization touched which call.

## Guardrails

The rewriter is deliberately conservative:

* Your request objects are never mutated — rewrites happen on a copy, and only
  the provider sees them.
* A caller-supplied `prompt_cache_key` or existing `cache_control` markers are
  respected, never overridden. Anthropic's four-breakpoint budget is honored,
  counting any markers you already set.
* `prompt_cache_key` is only sent to `api.openai.com` itself — OpenAI-compatible
  providers that would reject the parameter are left alone. If OpenAI ever
  rejects it anyway, the SDK retries the call without it.
* Rules can be withdrawn centrally at any time; a fetch failure keeps the last
  known-good snapshot and never touches your call.
* Any error while rewriting relays the **original request unchanged**. Apply is
  fail-open end to end.

## Measuring the win

Realized savings show up in the dashboard and in
[`GET /v1/workspaces/{id}/stats/summary`](/api-reference/introduction) as
`estimated_cache_savings_usd`, alongside cached-token counts per model.

Where Visceral finds waste it *cannot* prove safe to auto-fix — a prefix that
almost caches but is broken by a volatile block, say — it files a
[finding](/concepts/how-it-works#findings-the-waste-map) instead of touching
your traffic.