Ultrathink

Claude Code exposes a family of magic words — "think", "think hard", "think harder", "ultrathink" — that allocate the model an escalating budget of thinking tokens: private reasoning space it spends working through a problem before answering. Ultrathink is the top of the range. That's the entire feature — a knob, not an architecture. But it's a knob on the quality/cost/latency triangle, and it comes with a measurement trap subtle enough that most teams' conclusions about it are fiction. This lesson: what the knob actually buys, when to turn it, and the resolution arithmetic that decides whether you can trust anything you observe about it.

What extended thinking buys

More budget lets the model explore before committing: enumerate approaches, chase implications a few steps, notice its own first-draft errors and repair them privately. Work that benefits is high branching factor work — problems where the first plausible answer is often wrong and the good answer requires comparing paths:

• Architectural decisions with genuine alternatives
• Subtle concurrency and state bugs
• Debugging where the obvious hypothesis has already failed
• Sequencing problems (migration order, dependency untangling)
• Anything where you'd tell a human "sleep on it"

Work that doesn't benefit: anything that's essentially retrieval or transcription — boilerplate, renames, well-specified edits, summaries, format conversions. On flat problems, extra thinking produces the same answer, later, at higher cost: the model rehearses conclusions it would have reached anyway. (This is lesson 10's capability-flat/steep distinction, applied to a single model's effort dial instead of the tier ladder.)

So the operating habit is escalation, not default: run without a modifier; reach for the budget when you know branching is real ("this deadlock has survived two fixes — ultrathink it") or when a cheap first attempt came back shallow. Escalating on retry is the efficient pattern — you pay the premium only on problems that have demonstrated they need it. And pair the dial with the planning move from lesson 10: an ultrathink plan followed by cheap execution often beats uniform heavy thinking, because the plan is where the branching lived.

The measurement trap

Here's what makes this a lesson instead of a tooltip. You will be tempted to conclude things like "ultrathink improved our success rate about 2%," or — more dangerous — "we removed it and nothing got worse." Both conclusions are probably beyond your instruments.

Recall lesson 21's granularity point and make it concrete: a suite with 20 scored checks moves in 5-point steps. A real 2-point effect is invisible on it — literally cannot register except as occasional flicker. A measured 5-point swing might be one flaky case. To detect a small effect, the claimed difference must exceed the suite's resolution — roughly, one over the number of independent scored checks — so a 1–2% effect needs on the order of a hundred checks before yes and no even look different, and several hundred before you'd bet on the answer.

The regression direction is where this bites hardest, because the incentives point that way: dropping a thinking budget saves visible money, and the quality cost is equally invisible to a small suite. "We removed ultrathink and the eval didn't move" is only evidence if the eval could have moved — otherwise it's the sound of an instrument that can't hear, presented as silence. Before trusting any conclusion about a thinking-budget change, in either direction, ask the resolution question first: what's the smallest difference this suite can see? If the effect you're claiming is smaller, the honest verdict is "can't tell from here" — not "no difference."

The practical protocol

For a workflow you run constantly (the volume cases from lessons 17, 18, 22): try the tiers against your eval; adopt the cheapest budget whose score holds; and treat a tie at your suite's resolution as a statement about the suite as much as about the budgets. If the decision matters — high-volume pipeline, meaningful money — grow the suite until it can actually resolve the question, then decide. That sequence (instrument first, then dial) is cheaper than it sounds and infinitely cheaper than shipping a quality regression you couldn't see.

For everything else — interactive, occasional — skip the ceremony entirely: escalate when stuck, enjoy the better answer, and resist writing the anecdote into a slide. One good ultrathink save on a nasty bug feels like data. It's one sample from a distribution you haven't measured — which, after lesson 21, is a sentence you can now hear as the caution it is.