The Consistency Gap: Why Your Agent's Success Rate Is Lying to You

Enterprise AI procurement has a measurement problem. Teams evaluating agentic systems are routinely shown benchmark scores that reflect average performance across runs, and those averages can look compelling enough to justify deployment. The trouble is that averages obscure the metric that actually determines whether a workflow is viable in production: whether the agent succeeds every time it runs the same task, not just most of the time.

Companion piece to our broader work on agent reliability. See Why AI Coding Agents Produce Different Results for a closer look at the structural factors that drive inconsistent agent outputs across repeated runs.

What the Headline Number Is Actually Measuring

The standard evaluation protocol for production agents reports Mean@k: run a benchmark suite k times, average the pass rate, and publish the result. A score of 77% sounds like a system that works reliably. It is not.

IBM Research ran a ReAct agent using GPT-4.1 on the AppWorld benchmark across five repeated runs and found that while the agent succeeded on 77.4% of runs on average, it succeeded on all five runs for only 53.0% of tasks. That is a 24.4-point gap between what the headline metric reports and what a production operator actually needs (Duesterwald et al., arXiv 2026).

Source: IBM Research, "Your Agent Aced the Task. Will It Do It Again?", 2026. Evaluation conducted on AppWorld test_normal using a ReAct agent with GPT-4.1 across five repeated runs per task. [research.ibm.com]

On harder tasks, that gap widens to 30 points. For any workflow where failure has a cost, the number that matters is Pass@5, not Mean@5.

Why Agents Fail Inconsistently on the Same Task

The mechanism behind this gap is not model error in the conventional sense. It is decision-point variance: at certain steps in a multi-step task, the model is sampling from a probability distribution over next tokens, and the distribution is not sharply peaked. A slightly different sample produces a different action, which propagates into a different trajectory, which may or may not reach the correct outcome.

This is structurally different from a model that simply does not know how to do a task. The agent has the capability; it just does not apply it reliably. That distinction matters for remediation, because the failure is not in the model's knowledge but in how confidently it commits to a decision path at ambiguous steps.

The IBM Research team built a tool called the Consistency Analyzer to locate these flip-prone decision points. Rather than re-running tasks end-to-end, it resamples each decision point within a single recorded trajectory by requesting multiple completions at that step. This means diagnosis requires one trace and no ground truth labels, which makes it practical to run on real production logs.

What Consistency Guidelines Do

Once flip-prone steps are identified, the IBM Research system converts them into consistency guidelines: targeted instructions that are injected at inference time to steer the agent away from the decision-point variance that caused previous failures. These guidelines are generated automatically from the agent's own trajectory data, without requiring human annotation.

The results are specific. Applying consistency guidelines reduced the 24.4-point gap to 12.0 points. Same-task Pass@5 improved by 16.0 percentage points, and similar-task Pass@5 improved by 13.0 percentage points, with no reduction in average accuracy (Duesterwald et al., arXiv 2026). That last point is worth holding: the improvement in consistency did not come at the cost of average performance, which is the typical trade-off when you constrain model behaviour.

This matters commercially because it means the two metrics are not in tension. You can improve the reliability metric that production requires without degrading the benchmark metric that procurement uses.

What This Means for Procurement and Deployment Decisions

Enterprise buyers evaluating agentic systems for regulated or operationally sensitive workflows should treat Pass@k as a required disclosure, not an optional supplement to Mean@k. For financial reconciliation, contract review, or compliance checking, a workflow that fails 47% of the time on repeated identical tasks is not production-ready regardless of its average score.

The practical implication for RFPs and vendor evaluations is straightforward. Require vendors to report both Mean@k and Pass@k across a minimum of five runs per task. Require that the benchmark tasks reflect the actual distribution of inputs the system will encounter in your environment, not a curated subset. And treat a large gap between the two numbers as a signal that the system's decision-making is brittle at specific steps, not that it is unreliable in general, because those two diagnoses have different remediation paths.

For internal deployments, the same logic applies to evaluation before rollout. Running a benchmark once and observing a pass is not evidence of production readiness. It is evidence that the agent can succeed, which is a necessary but insufficient condition.

Building Consistency Into Your Evaluation Framework

Operationalising consistency measurement does not require a complete overhaul of existing evaluation infrastructure. The Consistency Analyzer approach demonstrates that useful diagnostic information can be extracted from existing trajectory logs without additional end-to-end runs, which keeps the cost of measurement low.

The more significant organisational change is shifting the primary production-readiness criterion from average accuracy to consistent task success. This affects how evaluation results are reported internally, what thresholds trigger deployment approval, and how post-deployment monitoring is structured. An agent that begins showing a widening gap between average success and consistent success in production logs is exhibiting early-stage reliability degradation, even if its average score remains stable.

For teams operating in regulated environments, this framing also has a compliance dimension. A system that succeeds on average but fails unpredictably on specific runs is difficult to audit and difficult to defend. Consistency is not just a reliability property; it is a precondition for the kind of deterministic accountability that regulated workflows require.