Why Your Company-Wide Agent Needs an Agent Harness Before It Needs Another LLM Integration

When Stripe built Kai, its internal company-wide agent, the engineering team did not start by evaluating which LLM had the best benchmark scores. They started by asking what infrastructure would need to exist before any LLM could be trusted at company scale. That ordering matters. Most organisations get it backwards, integrating models first and discovering the orchestration problem only after several teams have built incompatible layers on top of each other and production incidents have made the fragmentation visible.

This article is about the infrastructure decision that precedes model selection: the agent harness. It covers what a harness is, why it becomes load-bearing when you move from one team's experiment to a company-wide deployment, and what the evaluation and reliability requirements look like in practice.

The Fragmentation Pattern That Looks Fine Until It Doesn't

When a single team builds an agent workflow, they make sensible local decisions. They pick an orchestration library, write a retry wrapper, define their tool schemas, and ship something that works. The problem is that the next team makes equally sensible but different decisions. Within six months, a mid-sized engineering organisation can have four separate orchestration layers, each with its own logging format, its own error-handling conventions, and its own implicit assumptions about how tools are called and retried.

This is not a people problem. It is a coordination problem that emerges naturally when there is no shared infrastructure to constrain local choices. The fragmentation is invisible in demos because each team's workflow runs in isolation. It becomes visible under production load, when you need to audit tool calls across teams, when a shared tool changes its schema, or when you want to run evaluation across the whole agent estate.

Stripe's path to a shared harness was driven precisely by this recognition. Giving every employee access to an agent at once is an organisational forcing function. It exposes every assumption that was safe to leave implicit when only one team was using the system.

What an Agent Harness Actually Is

An agent harness is the shared infrastructure layer that sits between your LLMs and your tools. It handles orchestration, tool invocation, retry logic, context management, and evaluation instrumentation in a single place. Teams build their agent behaviours on top of it rather than alongside it.

Orchestration Reliability

The orchestration layer is where most production failures originate. An agent that works correctly in testing can fail in production because a tool returns a response 200ms slower than expected, because a retry loop amplifies an upstream error, or because the agent's context window fills in a way that was never tested. A shared harness defines the retry policy, the timeout thresholds, and the context management strategy once, and every team inherits those decisions.

Research on agent behaviour in dynamic environments reinforces why this matters. When tool schemas drift or tool behaviour becomes unpredictable, agents without structured recovery mechanisms fall back to exhaustive search rather than deductive adaptation (Toh et al., arXiv 2026). A harness that enforces consistent tool schema versioning and provides structured error signals is a precondition for agents that degrade gracefully rather than catastrophically.

Tool Registry and Schema Governance

A harness gives you a centralised tool registry. When a tool changes, the change propagates through the registry rather than requiring every team to update their own integration. This is operationally significant at scale: a company-wide agent that touches dozens of internal systems needs a single source of truth for what each tool does, what parameters it accepts, and what errors it can return.

The Evaluation Gap Between Demos and Production

Evaluation is where the absence of a harness becomes most costly. Without shared infrastructure, each team runs its own evaluation in isolation, using different metrics, different test sets, and different definitions of what a correct agent response looks like. You cannot compare performance across use cases, you cannot detect regressions introduced by a model update, and you cannot demonstrate to the business that the agent estate is improving over time.

A harness provides the instrumentation layer that makes evaluation possible at scale. Every tool call, every LLM invocation, and every agent decision is logged in a consistent format. Evaluation pipelines can then operate across the entire estate rather than within individual team silos. This is not a convenience. It is the mechanism by which engineering leadership maintains visibility into a system that is now touching every employee's workflow.

The Organisational Forcing Function of Company-Wide Access

Deploying an agent to every employee simultaneously changes the risk profile of every engineering decision. A bug that would affect one team's workflow now affects the whole company. A poorly specified tool schema that one team would catch in code review can now produce incorrect outputs at scale before anyone notices.

This is the argument for building the harness before the rollout, not after. The harness is what makes the rollout auditable. It is what allows you to gate access by capability, to roll back tool integrations without disrupting every team, and to run A/B evaluations on agent behaviour changes without manual coordination across the engineering organisation.

The forcing function also applies to model selection. When orchestration is centralised, model routing decisions become explicit infrastructure choices rather than implicit per-team preferences. We have written previously about how model routing inside agent workflows carries governance and cost implications that most orchestration designs underweight. A harness makes those decisions visible and controllable in a way that distributed orchestration cannot.

Companion piece to our broader work on agent orchestration economics. See Model Routing in Agent Workflows: Cost & Governance for how routing decisions interact with governance and audit requirements at the infrastructure level.

Building the Harness: What the Implementation Sequence Looks Like

The practical sequence for building a shared harness follows a clear dependency order. You define the tool registry and schema governance model first, because every other component depends on it. You then build the orchestration layer, including retry logic, timeout handling, and context management. Evaluation instrumentation comes next, because it needs to be designed into the logging architecture rather than retrofitted. Access control and capability gating come last, because they operate on top of the orchestration and evaluation layers.

What Teams Give Up and What They Gain

Teams that adopt a shared harness give up the ability to make local orchestration decisions. That is the point. The constraint is what produces the consistency that makes company-wide deployment viable.

What they gain is significant. They inherit a tested, instrumented orchestration layer rather than building one. They get evaluation pipelines that work out of the box. They get a tool registry that handles schema changes centrally. And they get the ability to demonstrate to the business that their agent workflows are measurably improving, because the evaluation infrastructure exists to show it.

The engineering leaders who build the harness first and the integrations second are the ones who find that company-wide agent deployment is an infrastructure problem they have already solved, rather than one they discover mid-rollout.