Prompt Caching

Prompt caching is the provider-side discount that makes serious agent work affordable — and a hidden architectural constraint that quietly punishes teams who don't know the rule. The deal, on Anthropic's API: when the leading portion of your request — the prefix — is byte-identical to one recently processed, that portion bills at ~10% of normal input price; writing a new cache entry costs a one-time ~25% premium. Since agent sessions re-send the entire conversation every turn (lesson 2), and input dominates agent cost (lesson 3), a well-shaped session often runs 5–10x cheaper than a naive one. Claude Code arranges its requests cache-friendly automatically; if that's all you use, this lesson is mostly why your bills look the way they do. The moment you assemble API requests yourself — SDK agents, pipelines, anything from lesson 18 onward — the rule below is load-bearing.

The rule: a prefix, not a set

The cache does not match content you've sent before. It matches an unbroken run of identical bytes starting at position zero. Every consequence follows from that one sentence:

• Position matters. Identical content in a different position is a miss. Reorder two blocks and everything from the first differing byte onward bills at full price — even though every byte was "cached" yesterday.
• One volatile byte poisons everything after it. A timestamp near the top of your prompt, a request id, a randomized example — the cacheable prefix ends there, permanently, invisibly. Nothing looks wrong. The bill is just 5x.
• The design goal is a maximal stable prefix: stable content first, in a frozen order; volatile content last. System prompt → standing instructions → tool definitions → reference docs → conversation history → this turn's input. Mark the cache breakpoint (the API's cache_control) at the stable/volatile boundary.

Treat segment order as an API contract and test it like one, because this is the classic silent regression: someone "harmlessly" reorders request assembly, nothing fails, output is identical — and every request afterward costs 5x. A unit test that asserts your assembled prefix is byte-stable across two consecutive requests is ten lines and worth more than most dashboards.

The collision with lazy loading

Here's where caching stops being a billing detail and becomes architecture: it conflicts with load-on-demand context (lesson 20). Progressive disclosure says "inject material only when needed" — but inject it above stable content, and the prefix breaks at the injection point: everything below re-bills at full price, on every subsequent request, whether or not the injected material mattered. A 50-token late-loaded snippet can cost thousands of tokens per turn in lost discounts — wildly counterintuitive until you picture the prefix snapping.

The resolution is placement, not abstinence: late-loaded material goes at the tail, below everything stable, where it costs only itself. Both disciplines survive; the request layout is where they're reconciled. If you build agents, this paragraph is the one to remember from this lesson.

Three layers of cost truth

Honest cache accounting separates what you can know, and confusing the layers produces confident nonsense in cost projections:

Measurable by you: what fraction of your request is a stable prefix across turns (a well-shaped agent session: 90%+), and where the first volatile byte sits. Pure arithmetic over your own requests — measure it.

Modeled from published prices: what the discount is worth if reads hit. At 1.25x write and 0.1x read, a prefix reused twice has already paid for its write premium; steady reuse approaches the full 90% discount on the stable region. A model, not a bill — label it as one (lesson 3's pedigree habit).

Not knowable in advance: the hit rate. Cache entries have a lifetime (minutes by default on Anthropic's API; a longer-TTL tier exists at extra cost), minimum sizes, and provider-side eviction. Whether your second request lands inside the window is a fact about infrastructure and timing. Design for hits; never assume them in a promise; verify with the API's actual cache-read/write counters in responses when the difference matters.

The practical checklist

For Claude Code users:

• Don't fight the tool — it's already shaped for caching. Your leverage is behavioral: steady, focused sessions cache beautifully.
• /clear costs one cache-write cycle on the next turn; a cleaner desk usually repays it (lesson 2's advice survives the arithmetic).

For anyone assembling requests:

1. Order stable → volatile; freeze the order; test the freeze.
2. Put breakpoints at the stable boundary; keep timestamps, ids, and per-request noise strictly below it. Audit for accidental volatility — the "static" prompt that interpolates today's date is the classic self-inflicted miss.
3. Put late-loads at the tail (see the collision above).
4. In batch pipelines (lesson 19), group runs sharing a prefix close in time — they reuse one warm entry instead of each paying a cold write. Batch ordering is a cache lever.
5. Watch the counters for a day after any assembly change. A cache regression is silent by construction; the counters are the only place it shows.

One closing observation worth internalizing: caching pays you for discipline you should have anyway. Stable standing instructions (lesson 8), frozen request assembly, boring long-lived conventions — the provider is literally discounting architectural stability at 90%. Take the money.