# HarnessEvolve: Learning from Reference Trajectories for Reliable Agent Self-Evolution

> HarnessEvolve, a self-evolving agent framework with reference-guided error diagnosis and dual gating, outperforms all baselines across five benchmarks, achieving up to 21.6% accuracy gains.

- **Source:** [arXiv](https://arxiv.org/abs/2609.00829)
- **Published:** 2026-09-12
- **Permalink:** https://picx.dev/p/jBfnlh
- **Whiteboard:** https://picx.dev/p/jBfnlh/image

## Summary

## Summary (Overview)

- **HarnessEvolve** is a self-evolving agent framework that optimizes the entire agent harness (prompts, skills, tools, and execution logic) by learning from reference trajectories—execution paths produced when the agent is given ground-truth answers.
- The framework addresses three core challenges in self-evolution: **credit assignment failure** (identifying which step caused an error), **shortcut learning** (agents memorizing task-specific patterns), and **catastrophic forgetting** (degrading previously acquired capabilities).
- HarnessEvolve uses a decoupled multi-agent architecture with four independent modules: execution, evaluation, optimization, and gating, ensuring that optimization decisions are auditable and cannot be gamed by the execution agent.
- It consistently outperforms state-of-the-art baselines (GEPA, ACE, SkillOpt) across all five benchmarks, including a 21.6 percentage point improvement over the strongest baseline on the CloudCoreNetwork-QA dataset.
- Ablation studies confirm that reference-guided error diagnosis is the most critical component, with the quality gate and error clustering also contributing significantly to performance.

## Introduction and Theoretical Foundation

### Background and Motivation

Large language models (LLMs) have enabled autonomous agents that tackle complex tasks across domains. At the core of such agents is their **harness**: the prompts, skills, tools, and execution logic that shape their capabilities. The design of harnesses has evolved through several waves:

1. **Handcrafted workflows**: Human experts explicitly design execution logic, search topologies, and collaboration structures (e.g., ReAct, CoT). These offer high controllability but lack flexibility and remain bounded within human-designed frameworks.
2. **Skill-augmented agents**: Humans provide skills in natural language while models operate within autonomous perception-planning-action-observation loops (e.g., Claude Code, OpenClaw). These enable dynamic trial-and-error but remain bounded by the quality of human-provided skills.
3. **Self-evolving agents**: Systems that autonomously synthesize, refine, and iteratively improve their own harness from environmental feedback.

### Three Fundamental Challenges

The paper identifies three fundamental challenges hindering reliable self-evolution:

- **Credit Assignment Failure**: In long-horizon tasks, agents receive only binary success/failure signals. When failure occurs, the agent cannot identify which specific action caused it because later errors are downstream consequences of earlier mistakes.
- **Shortcut Learning**: Without safeguards, agents tend to hardcode answers or inject excessive task-specific in-context examples into the harness, causing data leakage and prompt bloat that inflate training accuracy without genuine capability gain.
- **Catastrophic Forgetting**: Since each harness update is optimized on a subset of trajectory data, it may conflict with previously acquired competence, gradually degrading existing capabilities.

### Theoretical Foundation

The paper argues that these challenges cannot be resolved by designing yet another iterative prompt optimization algorithm; they necessitate a **system-level architectural shift**. HarnessEvolve decouples the execution agent from the evolutionary pipeline, assigning execution, evaluation, optimization, and gating to independent modules.

## Methodology

### Problem Formulation

Given a task corpus $\mathcal{T} = \{ (q_i, a_i^*) \}_{i=1}^M$ of M query-answer pairs split into training $\mathcal{T}_{train}$, validation $\mathcal{T}_{val}$, and test $\mathcal{T}_{test}$ sets, and a base execution agent $A_{exec}$ powered by a frozen LLM, the goal is to find the optimal execution agent $A^*$ by iteratively optimizing its harness on $\mathcal{T}_{train}$ without human intervention, such that accuracy on $\mathcal{T}_{val}$ is maximized.

### Architecture Overview

HarnessEvolve uses four independent agent modules:

- **Execution agent ($A_{exec}$)**: The target of optimization, runs tasks from $\mathcal{T}_{train}$ using its harness. Also produces reference trajectories when given ground-truth answers.
- **Evaluation agent ($A_{eval}$)**: (1) Evaluates task accuracy and identifies failed trajectories; (2) verifies that reference trajectories are genuine execution paths rather than shortcuts.
- **Optimization agent ($A_{opt}$)**: Analyzes failed trajectories, compares them against reference trajectories to identify root causes, clusters errors into systematic patterns, and generates candidate harness modifications.
- **Gate agent ($A_{gate}$)**: Enforces quality checks (data leakage and prompt bloat inspection) and performance checks (accuracy improvement without degradation).

### Key Mechanisms

#### 1. Reference-Guided Error Diagnosis

Prior to the optimization loop, $A_{exec}$ is run on all tasks with ground-truth answers to produce **reference trajectories** $\tau_i^+$. The evaluation agent verifies each trajectory for genuineness (checking that it follows a legitimate reasoning chain rather than trivially restating the answer). During optimization, failed trajectories $\tau_i^-$ are compared against reference trajectories to identify the **first point of divergence** $t_i^*$—the earliest step where the action deviates—enabling fine-grained root-cause localization.

#### 2. Error Clustering

Individual error signals $\mathcal{F}_i = (s_i, m_i, h_i)$ (severity, error cause, fix hint) are clustered by error cause $m_i$ into groups $\mathcal{P} = \{C_1, \ldots, C_K\}$, following three principles:
- **Cause-Based Grouping**: Groups formed by error cause rather than coarse severity labels
- **Root-Cause Priority**: Prioritize the first action divergence point as the root error cause
- **Long-Tail Protection**: Preserve single-member clusters to ensure rare but critical failure patterns are not absorbed

#### 3. Quality Gate

The gate agent checks candidate updates for:
- **Data leakage**: Whether edits directly embed failed queries and ground-truth answers into harness files (LLM-as-judge with score threshold $\eta_{leak} = 0.8$)
- **Prompt bloat**: Whether newly injected in-context examples exceed threshold $\eta_{blo} = 5$

A candidate update passes if and only if, for every file in the update, the data leakage score does not exceed $\eta_{leak}$ and the number of injected in-context examples does not exceed $\eta_{blo}$.

#### 4. Performance Gate

A candidate agent $A_{candidate}$ is accepted if and only if it improves accuracy on the current batch by at least margin δ while not degrading on any recent batch beyond tolerance ε:

$$\mathcal{A}_{\text{candidate}} \text{ is accepted} \iff \left\{ \begin{array}{l} \operatorname{Acc}(\mathcal{A}_{\text{candidate}}, B_j) - \operatorname{Acc}(\mathcal{A}_{\text{current}}, B_j) \geq \delta, \\ \max_{l \in [\max(1, j-R), j-1]} \left[ \operatorname{Acc}(\mathcal{A}_{\text{current}}, B_l) - \operatorname{Acc}(\mathcal{A}_{\text{candidate}}, B_l) \right] \leq \epsilon. \end{array} \right.\tag{1}$$

#### 5. Epoch-End Selection

At epoch completion, all snapshots in the pool V are evaluated on the validation set, and the best-performing one is selected:

$$\mathcal{A}_{\text{current}} \leftarrow \arg \max_{\mathcal{A} \in \mathbb{V}} \operatorname{Acc}(\mathcal{A}, \mathcal{T}_{\text{val}}), \quad \mathcal{A}^* \leftarrow \mathcal{A}_{\text{current}}.\tag{2}$$

### Implementation Details

- Models: Qwen3.6-27B (post-trained with domain-specific fine-tuning) and DeepSeek-V4-Flash (no fine-tuning)
- Frameworks: LAMAgent (in-house, for enterprise datasets) and OpenClaw (for open-source datasets)
- Hyperparameters: $T_{att} = 5$ (max reference attempts), $P_{batch} = 10$ (batch patience), $P_{ep} = 5$ (epoch patience), $E = 20$ (epoch limit), $b = 40$ (batch size), $R = 2$ (replay buffer size), $\delta = 0.0$ (step margin), $\epsilon = 0.025$ (degradation tolerance), $\eta_{leak} = 0.8$, $\eta_{blo} = 5$, $T_{rev} = 3$ (revision limit)

## Empirical Validation / Results

### Main Results on In-House Datasets

**Table 1: Main results on in-house datasets. Accuracy (%) is reported. Best results are in bold.**

| Method | CloudCoreNetwork-QA (Qwen) | CloudCoreNetwork-QA (DeepSeek) | Wireless-QA (Qwen) | Wireless-QA (DeepSeek) |
|---|---|---|---|---|
| Base | 43.4 | 47.5 | 79.0 | 85.9 |
| GEPA | 65.3 | 57.6 | 82.4 | 86.5 |
| ACE | 59.3 | 64.6 | 84.3 | 90.1 |
| SkillOpt | 61.9 | 65.3 | 89.0 | 89.3 |
| **HarnessEvolve** | **86.9** | **85.9** | **89.7** | **92.8** |

On CloudCoreNetwork-QA with Qwen3.6-27B, HarnessEvolve improves accuracy from 43.4% (Base) to 86.9%, outperforming the strongest baseline GEPA at 65.3% by **21.6 percentage points**.

### Main Results on Open-Source Datasets

**Table 2: Main results on open-source datasets. Accuracy (%) is reported. Best results are in bold.**

| Method | SearchQA | OfficeQA | SpreadsheetBench |
|---|---|---|---|
| Base | 86.5 | 62.8 | 44.3 |
| GEPA | 88.6 | 64.0 | 69.6 |
| ACE | 90.0 | 68.9 | 52.2 |
| SkillOpt | 89.4 | 66.9 | 74.6 |
| **HarnessEvolve** | **92.9** | **70.9** | **76.4** |

### Cross-Framework Generalization

**Table 3: Cross-framework transfer: skills optimized on OpenClaw, evaluated on four frameworks. Accuracy (%) is reported. Best results are in bold.**

| Framework | SearchQA (Base) | SearchQA (HarnessEvolve) | OfficeQA (Base) | OfficeQA (HarnessEvolve) | SpreadsheetBench (Base) | SpreadsheetBench (HarnessEvolve) |
|---|---|---|---|---|---|---|
| Hermes | 95.0 | 95.0 | 77.9 | **80.8** | 72.1 | **73.2** |
| OpenCode | 90.0 | **92.7** | 75.6 | **80.3** | 57.5 | **87.9** |
| LAMAgent | 92.9 | **94.3** | 74.4 | **75.0** | 45.0 | **80.0** |
| DeepSeek Harness | 93.6 | **95.0** | 79.1 | **79.7** | 56.4 | **86.8** |

### Ablation Study

**Table 4: Ablation study results on CloudCoreNetwork-QA (Qwen3.6-27B). Accuracy (%) is reported. Best results are in bold.**

| Variant | CloudCoreNetwork-QA |
|---|---|
| **HarnessEvolve (full)** | **86.9** |
| M1: w/o reference trajectory | 57.8 |
| M2: w/o error clustering | 68.6 |
| M3: w/o quality gate | 80.1 |

Key findings:
- **M1 (w/o reference trajectory)**: Largest degradation (86.9% → 57.8%), confirming reference-guided diagnosis is the most critical component
- **M2 (w/o error clustering)**: Significant degradation (86.9% → 68.6%), confirming error aggregation is essential for coherent modifications
- **M3 (w/o quality gate)**: Moderate degradation (86.9% → 80.1%), indicating the quality gate helps prevent shortcut learning

### Self-Evolution Curve

The evolution curves show that a substantial fraction of candidate updates are rejected by the two-tier gating mechanism, indicating effective filtering of low-quality modifications. The peak accuracy in the snapshot pool increases gradually throughout evolution, reaching its highest point at the final harness.

## Theoretical and Practical Implications

### Theoretical Implications

1. **Credit Assignment Resolution**: The reference-guided error diagnosis approach provides a principled method for isolating root causes in long-horizon tasks, addressing a fundamental limitation of sparse-reward environments.

2. **Guarded Evolution**: The two-tier gating mechanism (quality + performance) offers a theoretical framework for stable self-improvement, balancing exploration of new capabilities with preservation of existing ones.

3. **Comprehensive Harness Optimization**: Unlike prior methods optimizing only single components (e.g., SkillOpt optimizes only skill.md), HarnessEvolve demonstrates that optimizing the entire harness—including Python scripts, prompt instructions, tool-argument specifications, and execution logic—yields substantially better results.

### Practical Implications

1. **Enterprise Deployment**: The significant gains on enterprise datasets (CloudCoreNetwork-QA: 21.6 percentage points over strongest baseline) demonstrate practical viability for complex multi-skill enterprise tasks.

2. **Framework Transferability**: Skills optimized by HarnessEvolve transfer across four different agent frameworks without re-optimization, indicating generalizable improvements rooted in error patterns rather than framework-specific shortcuts.

3. **Reliability**: The decoupled architecture ensures optimization decisions are auditable, addressing deployment concerns about uncontrolled self-modification.

## Conclusion

HarnessEvolve presents a comprehensive solution for reliable agent self-evolution by:
1. **Overcoming credit assignment failure** through reference-guided error diagnosis
2. **Preventing shortcut learning** via the quality gate (data leakage and prompt bloat inspection)
3. **Mitigating catastrophic forgetting** through the performance gate with epoch-end validation

The framework consistently outperforms state-of-the-art baselines across all five benchmarks spanning open-domain and enterprise scenarios, with substantial gains on complex multi-skill tasks. Future directions may include extending the framework to more diverse task domains, exploring additional gating mechanisms, and investigating the interaction between harness optimization and model fine-tuning.

---

_Markdown view of https://picx.dev/p/jBfnlh, served by PicX — AI-generated visual whiteboard summaries of research papers._