Why agents fail at step twelve

An agent demo goes perfectly. You wire the same thing into production, and two out of five runs derail somewhere in the middle, on a step that worked fine yesterday. Nobody changed the model. The task just got long enough for the math to catch up.

There is a second bill hiding behind the first. In a long agent session, every new step reprocesses everything that came before, so token cost climbs the longer the loop runs. Utkarsh Kanwat, who builds these systems, describes hitting the point where each response late in a session cost multiple dollars, more than the answer was worth. Both problems, the derailing and the cost, come straight out of what an agent loop actually is.

The loop is the whole story

An agent is not a new category of software. It is a model calling tools in a loop: the model picks a tool, the tool runs, the result gets appended to the context, and the model looks at everything so far and picks the next tool, until some stop condition. That is the entire mechanism. I walked through a three-tool version of it in give the agent a room, not a building. Everything about reliability follows from that loop, so it is worth holding the shape in your head before we do any arithmetic.

Success multiplies, so it collapses

The step that ruins your intuition is that per-step success rates multiply. A chain of independent steps succeeds only if every single step succeeds, so the whole-chain probability is the per-step rate raised to the number of steps. Kanwat lays out the arithmetic that names this module: at 95% per step, which is already optimistic for current models, you get 77% over 5 steps, 59% over 10, and about 36% over 20.

So a 10-step agent built on genuinely reliable 95% steps is a coin-flip in a trench coat, right around 60%. And you cannot prompt your way out of it, because it is not a prompt problem, it is multiplication. Even a heroic 99% per step, which nobody actually has, only gets you to 82% over 20 steps, still nowhere near the 99.9% a production system wants. Turn the dials yourself:

Notice the shape of the curve. Reliability does not decline gently, it falls off a cliff once the step count climbs, because you are compounding a number less than one. The lesson is not "give up on agents." It is that step count is the enemy, and every step you can remove from the critical path is worth more than a small bump in per-step reliability.

Passing once is not passing every time

Here is a distinction that trips up every agent demo. Passing a task once, what benchmarks call pass@1, is a different thing from passing it every time you run it. An agent can nail a task in the demo and fail the same task on a retry, with the same prompt, because the model's choices are probabilistic and the loop gives probability many chances to visit a wrong action.

Sierra built the tau-bench benchmark partly to measure this, using a metric called pass^k, the chance a task succeeds on all of k independent attempts. The gap is brutal: on retail tasks, GPT-4o scored around 61% on a single attempt but dropped to roughly 25% when it had to succeed eight times in a row. Reliability and capability are different axes. The question to ask a vendor demo is never "did it work," it is "does it work every time," and the honest answer is usually no.

Context fills with garbage

Module 3 called it context rot, and the agent loop is where it bites hardest. Every iteration appends something: a tool result, an error message, a retry of a call that timed out. None of it leaves unless you make it leave. So the window fills with the debris of the run, and just as the model needs to recall a detail from step two, that detail is buried under fourteen steps of noise and the model's recall of it has degraded. This is also the source of that quadratic bill, because each step reprocesses the whole growing pile. Context is a budget, and a naive loop blows it on its own transcript.

The design responses

None of this says agents are a dead end. It says the reliability has to come from structure around the loop, not from hoping the model is good enough. The clearest playbook is Anthropic's Building effective agents, and it comes down to a few moves.

Start simpler than an agent. Anthropic's own guidance is to find the simplest solution possible and add complexity only when it measurably helps, and they note that for many applications one good model call with retrieval is enough. When the steps are predictable, use a workflow, code that orchestrates model calls along a fixed path, not an agent that decides its own control flow. A workflow is cheaper, it fails legibly, and it dodges the compounding math because you, not the model, chose the steps. Reserve real agents for the open-ended tasks where you genuinely cannot enumerate the steps in advance.

When you do run a loop, cut its length and put a checker at the exit. Fewer steps means less to multiply. A verification gate, tests, a validator, a second model, or a human on the irreversible actions, resets the odds instead of letting them compound. And bound the blast radius: sandboxes, permission tiers, and human checkpoints on anything you cannot undo, so the worst run is affordable rather than catastrophic. Autonomy is a slider you widen as evidence accumulates, not a switch you flip on day one.

In production

PermitCheck's growth agent runs once a day by cron, and that cap is not incidental, it is the reliability design. An agent that runs once a day can only be so wrong before I read the log. Its writes go through the tiered mutation layer from give the agent a room: reads are free, anything that mutates returns a dry-run preview first, and applying a change takes a second confirmed call with an idempotency key, so a retried tool call becomes a no-op instead of a double-send. Where the task is actually a fixed sequence, like Enacted's detect-fetch-diff-summarize-gate, it is a pipeline, not an agent, because I can write the control flow in advance and every step I hardcode is a step that cannot compound. The rule I keep coming back to is suggest, don't act on anything expensive to undo.