The multiplication that explains everything
The formula is simple: if one step succeeds with probability p, and the task needs n steps, end-to-end success is p⁽ⁿ⁾.
| Per-step accuracy | 5 steps | 10 steps | 20 steps | 50 steps | 100 steps |
|---|---|---|---|---|---|
| 99.9% | 99.5% | 99.0% | 98.0% | 95.1% | 90.5% |
| 99.5% | 97.5% | 95.1% | 90.5% | 77.8% | 60.6% |
| 99.0% | 95.1% | 90.4% | 81.8% | 60.5% | 36.6% |
| 98.0% | 90.4% | 81.7% | 66.8% | 36.4% | 13.3% |
| 95.0% | 77.4% | 59.9% | 35.8% | 7.7% | 0.6% |
| 90.0% | 59.0% | 34.9% | 12.2% | 0.5% | 0.0% |
(Own calculation. The model assumes that the steps are independent and that a single bad step sinks the task. In reality neither is exactly true — more on that below.)
Read the 98% row. That is a very good agent: nineteen out of every twenty steps come out flawless. And it gets a third of the tasks wrong.
The reversal that is genuinely startling
Turn the question around: how much per-step accuracy does a given end-to-end target require?
| Number of steps | 90% end-to-end target | 95% target | 99% target |
|---|---|---|---|
| 5 | 97.9% | 99.0% | 99.8% |
| 10 | 99.0% | 99.5% | 99.90% |
| 20 | 99.475% | 99.74% | 99.95% |
| 50 | 99.79% | 99.90% | 99.98% |
| 100 | 99.895% | 99.95% | 99.99% |
(Own calculation: the per-step accuracy you need is the nth root of the target.)
At twenty steps a "ninety per cent agent" does not mean 90% per-step accuracy, it means 99.5%. That difference is precisely what users experience as "almost right, but it always gets something wrong somewhere".
Real measurement: one call or several turns?
None of this is only theory. There is a benchmark that compares single tool-call accuracy with multi-turn task accuracy on the same models.
| Single call | Multi-turn | Relative drop | |
|---|---|---|---|
| The best models | 86–91% | 53–68% | −22% … −38% |
| The weakest result in the field | 85.4% | 33.9% | −60% |
(Own calculation from public leaderboard data, as of April 2026.)
So models call a single function with close to 90% accuracy — but the moment they have to do it several times in a row, they fall to somewhere between 53 and 68%.
And there is a separate, separable effect on top. The same benchmark also measures the long-context variant:
| Multi-turn, base | Multi-turn, long context | Drop | |
|---|---|---|---|
| The top of the field | 63–81% | 56–70% | −0.5 to −10.5 points, model by model |
So two different things are dragging the result down. Step count on its own costs 20–33 percentage points, and long context adds another 5–10 to that — between 0.5 and 10.5 points depending on the model, with a median of 8 points across the seven models measured.
Not the difficulty, the execution
There is a 2025 experiment that separates planning from execution elegantly: it gives the model an explicit plan and every piece of knowledge it needs, and measures only whether it can carry the plan out.
The result:
- Almost every model performs the first step with 100% accuracy.
- The best model falls below 50% accuracy within 15 steps.
This is the empirical refutation of the idea that long tasks fail because they are "hard". The knowledge is there, the plan is there — it is the execution that degrades.
And there is a second, more uncomfortable finding from the same source, which they named self-conditioning: if the model's own earlier mistakes are sitting in the context, the probability of further mistakes goes up. The more errors they injected into the context, the worse the accuracy was at the hundredth step.
This is not a problem of handling long context, and it does not disappear because the model got bigger. What does help with it: thinking mode.
The 80% horizon that has not moved
There is a metric that measures how long a piece of human work a model can complete at a given probability. Two values are usually published: the 50% and the 80% horizon.
Out of seven years of data one ratio stands out: the 80% horizon has been 4–6 times shorter than the 50% one the whole way through.
In the wording of the people who ran the measurement: even models that are sometimes successful on hard and varied tasks cannot reliably complete medium-length tasks.
This is the single most important number in the subject. What counts is not how long a task the model finishes sometimes — it is how long a task it finishes reliably. And the distance between the two has been the same for seven years.
Where does it break? A failure taxonomy from measurement
There is a study that worked through 1,642 real, annotated agent runs from seven different frameworks, with 0.88 agreement between human annotators — methodologically strong.
They identified fourteen failure modes in three groups:
| Group | Share | The most important failure mode in it |
|---|---|---|
| System design failure | 44.2% | step repetition, looping — 15.7% (the single most common failure mode) |
| Inter-agent misunderstanding | 32.3% | reasoning–action mismatch — 13.2% |
| Missing verification | 23.5% | incorrect verification — 9.1%, missing verification — 8.2% |
A few further items that will look familiar from practice:
- fails to recognise the termination condition: 12.4% — in other words it does not know when it is done;
- task specification violation: 11.8% — it does something other than what you asked for;
- premature termination: 6.2% — it believes it is finished when it is not;
- fails to ask for clarification: 6.8%.
Two things here are worth pulling out. One: the loop is the single most common failure mode — the agent does not get the task wrong, it cannot get out of it. Two: "does not know when it is done" and "stops too early" come to 18.6% together — nearly one failure in five is about stopping, not about the work.
(One limitation: this taxonomy has no separate category for picking the wrong tool.)
What helps, and what does not?
Self-correction does not. According to a study published in conference proceedings, models cannot fix their own reasoning without external feedback, and performance occasionally gets worse after self-correction.
External checking does, and measurably. That same study measured two interventions end to end, with the same base model, changing only how the system was built:
| Intervention | Effect |
|---|---|
| Reworking the workflow | +9.4 percentage points |
| Strengthening the verification step | +15.6 percentage points |
This is the most important practical message in the article: the difference is not in the model, it is in whether the system has an external, deterministic verification point. Self-checking improves nothing. External verification does.
And retrying is not free. If the process has side effects — it writes to a database, sends an email, places an order — then the second attempt is not "one more chance", it is one more operation.
The counterweight: it really did improve
This section is compulsory, otherwise the article paints a false picture.
Measurable capability is growing fast. The doubling time of the 50% horizon is 188 days over the full time series, but 129 days since 2023 — and shorter still in the most recent segment. Concretely: a February 2025 model had a 50% horizon of 60 minutes, an April 2026 preview 1,045 minutes. That is a seventeen-fold increase in fourteen months.
But four caveats come with it, and all four have to be said out loud:
- The uncertainty bands are enormous. For one model the band around the measured 719 minutes runs from 317 to 3,634 minutes — the upper bound is eleven times the lower one.
- For the longest tasks the human reference is incomplete. By the measurers' own account, of the tasks longer than eight hours 5 out of 31 have an actually measured human baseline.
- The benchmark gets contaminated. At a June 2026 evaluation the measurers wrote that the model under test attempted to cheat at a higher rate than any before it — it exploited flaws in the scoring and extracted hidden source code. Depending on how the cheating was handled, the measured horizon came out at 11 hours, 71 hours or above 270 hours, and the measurers themselves noted that they regard none of those numbers as a reliable measurement.
- The 80% horizon is still 4–6 times shorter. Capability grew, the ratio did not — the gap between "sometimes does it" and "always does it" is as wide as it was seven years ago.
Common misconceptions
- "This is a 95% agent." For a single step, yes. At twenty steps end-to-end success is 35.8%.
- "If it gets it wrong, just run it again." If the process has side effects, the second run is not a repair, it is another operation.
- "Let's ask it to check itself." The measurement says self-correction without external feedback does not help, and sometimes hurts.
- "A long task is hard because it is complicated." In one experiment the model was handed a complete plan and all the knowledge, and accuracy still fell below 50% within fifteen steps.
- "A bigger model will sort it out." Self-conditioning — the model's own earlier mistakes degrading it further — does not go away with model size, according to the measurement.
- "The agent gets the task wrong." The single most common failure mode is the loop: it does not get it wrong, it cannot get out of it.
What to take away
Three sentences worth keeping in mind if you are rolling out an agent.
First: step count is the most important design decision. Not the model. Turning a twenty-step task into a five-step one is worth more than any model switch.
Second: you need an external verification point, not self-checking. In the measurement the first is worth 15 points, the second zero or less.
Third: "sometimes does it" is not a business number. What has to be decided is eight successes out of eight — and that is exactly where public data is thinnest.
Where the human approval point belongs in that chain is measured out in the human in the loop post, with real numbers; breaking the task up and keeping the context clean is what the subagents and parallel work post goes into next.