Agentic AI , AI Strategy , Software development Sep 10, 2026

Why Agent Harness Design Is the Variable Your Model Fine-Tuning Budget Is Ignoring

VECTOR Labs Team

Last updated on: Sep 10, 2026

Most enterprise ML teams treat the agent harness as configuration, not engineering. The system prompt gets drafted once, the tool set gets assembled around available APIs, and the real budget conversation happens around model selection and fine-tuning. Recent research from Salesforce AI suggests this sequencing is not just suboptimal but actively counterproductive: harness evolution alone can close meaningful performance gaps between weaker and frontier models, while naively fine-tuning on a stronger model's trajectories causes regression across every benchmark tested (Yu et al., arXiv 2026).

Companion piece to our broader work on agent harness infrastructure. See The Future of AI Infrastructure: Self-Optimizing Agent Harnesses for how execution-trace-driven optimization works in practice and what it means for managing deployments at scale.

The Harness Is Not a Wrapper, It Is Half the System

An agent harness encompasses the system prompt, the available tool set, execution hooks, and context-management scaffolding. Together, these components shape how a model plans, when it calls tools, and how it recovers from errors. Teams that treat these as static configuration are effectively fixing half the system while trying to optimise the other half in isolation.

The practical consequence is that fine-tuning spend gets evaluated against a baseline that was never designed for the model being trained. Performance gains look modest, so the conclusion is that the model needs more data or a larger LoRA rank, when the actual constraint is that the harness was not built around the model's planning behaviour.

What Harness Evolution Actually Achieves

Automated harness evolution, where the system prompt and scaffolding are iteratively refined using the weaker model's own execution traces, consistently produces performance gains on domain-specific enterprise tasks. Yu et al. (arXiv 2026) demonstrate this across seven enterprise agentic benchmarks, finding that evolving the harness around the weaker model's native planning style is the correct first step before any weight-level intervention.

The mechanism matters here. A harness evolved around a specific model encodes implicit assumptions about how that model structures its reasoning steps and when it requests tool calls. It is not a generic improvement; it is a fitted scaffold. This is why the gains do not transfer automatically when you swap in a different model, and why the sequencing of harness work before fine-tuning is not a stylistic preference but a technical dependency.

Why Imitation Learning from a Stronger Model Backfires

The intuitive next step after harness evolution is to collect trajectories from a stronger model operating under the evolved harness and fine-tune the weaker model on those demonstrations. The logic is straightforward: the expert makes better decisions, so teaching the weaker model to imitate those decisions should close the gap.

Yu et al. (arXiv 2026) tested exactly this and found the opposite result. Across two model families, Qwen3-Coder and Gemma 4, imitation of expert trajectories caused the weaker model to regress on all seven benchmarks, with performance drops ranging from four to thirty points. The weaker model acquires the expert's planning strategy but lacks the capacity to execute it reliably. Critically, it also loses its fit to the harness, which was designed around its native planning style, not the expert's.

The failure mode is not a data quality problem. The same imitation procedure works when applied under the unevolved baseline harness. The evolved harness is what makes imitation destructive, because the scaffold assumes a specific planning style and the weaker model abandons that style when it imitates the expert.

On-Policy Correction as the Practical Alternative

The approach that resolves this tension is on-policy expert correction. Rather than replacing the weaker model's trajectories with the expert's, this method runs the weaker model's own rollouts, identifies the specific turn where the trajectory fails, and has the expert rewrite only that turn. The rest of the trajectory, and therefore the weaker model's planning style, is preserved.

This is automated end-to-end by a meta-level MLE agent in the Salesforce implementation (Yu et al., arXiv 2026). The result is a teaching signal that improves execution quality without disrupting harness fit. The weaker model learns from the expert's judgment at the point of failure rather than adopting the expert's entire reasoning pattern.

For engineering teams, this reframes the fine-tuning problem. The question is not which model's outputs to imitate but which specific failure modes to correct, and at which turn in the trajectory. That is a different data collection problem, a different labelling problem, and a different evaluation problem than standard supervised fine-tuning.

How to Sequence Harness and Model Investment

The sequencing that emerges from this research is specific and ordered. Evolve the harness around the weaker model first, using the model's own execution traces to refine the system prompt and scaffolding. Measure the performance ceiling of the evolved harness before committing to weight-level adaptation. Only then introduce a teaching signal, and use on-policy correction rather than imitation to preserve harness fit.

This has direct budget implications. Harness engineering is cheaper than fine-tuning at scale, and the Salesforce results suggest it is also the prerequisite that determines whether fine-tuning spend produces positive returns at all. Teams that skip harness evolution and go straight to imitation-based fine-tuning are not just leaving performance on the table; they are actively degrading the model's ability to use the infrastructure built around it.

The co-evolution loop described by Yu et al. (arXiv 2026) treats harness and weights as jointly optimised variables rather than independent levers. That framing is more operationally demanding, but it reflects how these systems actually behave in production. Treating them as independent is a modelling assumption that the empirical results do not support.

Where Vector Labs Fits

We build agent harness infrastructure designed to co-evolve with the models deployed inside it, including the scaffolding, evaluation, and correction pipelines that make iterative improvement tractable. In our agent harness infrastructure guide, we set out the architecture decisions that determine whether harness investment compounds or stalls as model capabilities change. If you are currently allocating budget between harness engineering and fine-tuning programmes and want a structured assessment of your sequencing, contact us at vector-labs.ai/contacts.

FAQs

If harness evolution is so effective, when does fine-tuning actually add value?

Fine-tuning adds value after harness evolution has been completed and its ceiling measured. At that point, on-policy correction can close the remaining gap between the weaker model's performance and the expert's, without breaking the harness fit that evolution established. Fine-tuning applied before harness work, or via imitation of a stronger model under an evolved harness, produces regression rather than improvement based on the Salesforce AI findings (Yu et al., arXiv 2026).

Does this research apply to proprietary frontier models, or only to smaller open-weight models?

The Salesforce research was conducted on smaller mixture-of-experts models, specifically Qwen3-Coder at 30B total parameters and Gemma 4 at 26B. The harness evolution and on-policy correction findings are most directly applicable to teams using fine-tuneable open-weight models. For teams using proprietary frontier models via API, harness engineering remains the primary lever available, since weight-level adaptation is not possible, which makes the case for harness investment even stronger in that context.

How do we measure whether our current harness has been evolved or is just a static configuration?

The diagnostic is whether your harness was designed using the target model's own execution traces or assembled independently of model behaviour. A static harness is typically authored once based on task requirements and developer intuition. An evolved harness has been iteratively refined by observing where the specific model fails, which turns produce errors, and how the scaffolding can be adjusted to support the model's actual planning patterns rather than an assumed ideal.

What does on-policy expert correction require in terms of infrastructure compared to standard fine-tuning?

On-policy correction requires the ability to run the weaker model's rollouts, identify the failing turn programmatically or via a meta-level agent, and invoke the stronger model to rewrite that turn. This is more operationally complex than collecting static demonstration data, because it requires a live evaluation loop rather than a fixed dataset. The Salesforce implementation automates this end-to-end with an MLE agent (Yu et al., arXiv 2026), but teams should expect to invest in rollout infrastructure and turn-level failure localisation before this approach is viable at scale.

How should we present this sequencing to stakeholders who are already committed to a fine-tuning programme?

The most direct framing is that harness evolution is a prerequisite for fine-tuning to produce positive returns, not a competing investment. If an existing fine-tuning programme is underperforming, the question to ask is whether the harness was evolved before training data was collected. If not, the training data may encode an expert's behaviour under a harness that does not fit the weaker model, which is precisely the condition that produces regression. Redirecting a portion of the fine-tuning budget toward harness engineering first is a risk reduction measure, not a change of direction.

A team that understands you

With 20+ years of experience in the world's leading consultancy companies, implementing AI and ML projects in industry-specific contexts, we are ready to hear your challenges.

Talk with an AI expert