Recursive Harness Self-Improvement

Authors: Hyunin Lee, Jinglue Xu, Jeffrey Seely, Donghyun Lee, Matei Zaharia, and Yujin Tang (Sakana AI, UC Berkeley)

Summary (Overview)

Core Contribution: The paper introduces Recursive Harness Self-Improvement (RHI), a computationally lightweight method that iteratively refines prompt-level harnesses (agent roles, instructions, communication contracts, and workflow hops) to improve the performance of fixed coding agents on open-ended tasks.
Key Finding: A few RHI iterations (1–2) can raise the performance ceiling of low-reasoning-effort agents beyond their corresponding maximum-reasoning-effort test-time scaling baselines (e.g., opus-4.8-high+H[2] outperforms opus-4.8-ultracode), while reducing inference cost by up to 60%.
Mechanism: Performance gains arise primarily from improved task-specific context management (more effective inter-agent information flow via contracts and hops), not from longer reasoning traces—output-token usage remains nearly constant across iterations.
Theoretical Formalization: The authors propose an information-theoretic hypothesis for RHI's implicit optimization objective: increasing mutual information between externally emphasized harness components (contracts, hops) and the task, while penalizing redundancy (total correlation) across all components.
Evaluation: Validated on 30 synthetic open-ended ML research tasks spanning quantitative finance, robotics, and pharmacy, using pairwise LLM-as-a-judge evaluation.

Introduction and Theoretical Foundation

Background and Motivation

The paper frames AI progress within the paradigm of model–harness co-evolution: deployed AI systems improve not only through larger or better-trained foundation models, but through the co-evolution of models and their surrounding harnesses (agent workflows, prompts, tools, etc.). A key bottleneck in this loop is the quality of execution traces produced by harnesses—these traces can serve as training data for future foundation models.

The authors focus on the first half of this recursive loop: for a fixed foundation model, how to improve the harness to generate higher-quality execution traces. They argue that:

Provider-built harnesses must generalize across diverse users and tasks, making continual updates prohibitively costly.
User-constructed harnesses can be optimized for individual tasks, making them a practical target for improving execution-trace quality.
For practical optimization, each update must be computationally lightweight and converge within only a few update iterations.

Problem Definition

Formally, let $V$ denote the vocabulary space, $V^*$ the set of token sequences, $\mathcal{A}$ a coding agent with a fixed language model $\mathcal{L}$ , $x \in X$ a task prompt, $H \in \mathcal{H}$ a harness, and $y \in Y$ a code repository output. The agent produces output $y \sim \mathcal{A}(H, x)$ .

The ideal population objective is:

H_x^* \in \arg\max_{H \in \mathcal{H}} f_x(H), \qquad f_x(H) = \mathbb{E}_{H' \sim \mu(\cdot | H' \neq H), y \sim \mathcal{A}(H,x), y' \sim \mathcal{A}(H',x)} \left[ \mathbf{1}\{\mathcal{L}_{\text{eval}}(y, y'; x_{\text{eval}}) = y \succ y' \} \right] \tag{1}

where $\mu$ is a reference distribution over competing harnesses and $\mathcal{L}_{\text{eval}}$ is an LLM-based pairwise evaluator.

Limitations of Existing Methods

Existing population-based methods (e.g., Meta-Harness, AutoHarness, ADAS, GPTSwarm, AFlow, AlphaEvolve) approximate Equation (1) by replacing $\mu$ with a finite sampled population $\mathcal{S}_i = \{H_{i,1}, \ldots, H_{i,m}\}$ , estimating:

\widehat{f}_x(H; \mathcal{S}_i) = \frac{1}{|\mathcal{S}_i| - 1} \sum_{H' \in \mathcal{S}_i \setminus \{H\}} \mathbb{E}_{y \sim \mathcal{A}(H,x), y' \sim \mathcal{A}(H',x)} \left[ \mathbf{1}\{\mathcal{L}_{\text{eval}}(y, y'; x_{\text{eval}}) = y \succ y' \} \right] \tag{2}

These methods are ill-suited for user-constructed harness optimization because every additional candidate requires a full black-box agent execution and evaluation, making search cost prohibitive. The authors cite Wang et al. (2026) showing that, once search cost is accounted for, automatic harness evolution fails to consistently outperform simple test-time scaling baselines.

Methodology

RHI Objective Function

RHI adopts a trajectory-local relaxation of Equation (1). At iteration $i$ , it replaces the broad competitor distribution $\mu$ with a local competitor distribution concentrated on the previous harness, $\nu_i = \delta_{H_x^{(i-1)}}$ :

\widetilde{f}_x^{(i)}(H) = \mathbb{E}_{y \sim \mathcal{A}(H,x), y^- \sim \mathcal{A}(H_x^{(i-1)},x)} \left[ \mathbf{1}\{\mathcal{L}_{\text{eval}}(y, y^-; x_{\text{eval}}) = y \succ y^- \} \right] \tag{3}

Computational Lightness: RHI requires exactly one new agent execution and one pairwise evaluation per iteration, compared to $\Theta(M^2)$ for the ideal objective and $\Theta(m^2)$ for finite-population search:

Objective	$N_{\text{trace}}$	$N_{\text{pair}}$	Total Cost
Ideal objective (Eq. 1)	M	$\binom{M}{2}$	$\Theta(M^2)$
Finite-population search (Eq. 2)	m	$\binom{m}{2}$	$\Theta(m^2)$
Trajectory-local RHI (Eq. 3)	1	1	$\Theta(1)$

Noisy Local Ascent: Under a pairwise-preference model with task utility $u_x: H \to \mathbb{R}$ and strictly increasing link function $\sigma$ with $\sigma(0) = \frac{1}{2}$ satisfying $\Pr(H \succ H') = \sigma(u_x(H) - u_x(H'))$ , both objectives are monotone functions of the same latent utility:

f_x(H) = \mathbb{E}_{H' \sim \mu}[\sigma(u_x(H) - u_x(H'))], \qquad \widetilde{f}_x^{(i)}(H) = \sigma\left(u_x(H) - u_x(H_x^{(i-1)})\right)

Self-History: RHI accumulates preference feedback across iterations in the history:

\mathcal{D}_x^{(i)} = \left\{\mathcal{L}_{\text{eval}}\left(y_x^{(k)}, y_x^{(k-1)}; x_{\text{eval}}\right)\right\}_{k=1}^{i} \tag{4}

which serves as a "momentum-semantic" signal guiding future harness revisions.

RHI Algorithm

Algorithm 1: Recursive Harness Self-Improvement

Input: task $\{x_j\}_{j=1}^n \in \mathcal{X}$ , agent $\mathcal{A}$ , evaluator $\mathcal{L}_{\text{eval}}$ , harness optimizer $\mathcal{L}_{\text{harness}}$ , evaluation prompt $x_{\text{eval}}$ , stopping threshold $\epsilon$

Initialize: harness $H_x^{(0)} \in \mathcal{V}^*$ , harness history $\mathcal{D}_x \leftarrow \emptyset$ for $\forall x \in \mathcal{X}$

Agent $\mathcal{A}$ solves task $x$ using initial harness $H_x^{(0)}$
for $i = 1, 2, \ldots$ do
for $j = 1, 2, \ldots, n$ do
Agent $\mathcal{A}$ solves task $x_j$ using harness $H_j^{(i)}$ , obtains outputs $y_j^i$
Evaluator compares $y_j^i$ and $y_j^{i-1}$ , updates history $\mathcal{D}_j$
end
Compute improvement rate $s_i = \frac{1}{n} \sum_{j=1}^n \mathbf{1}[y_j^i \succ y_j^{i-1}]$
if $s_i < \epsilon$ then BREAK
Update harness: $H_x^{(i+1)} \leftarrow \mathcal{L}_{\text{harness}}(H_x^{(i)}, \mathcal{D}_x)$
end

The harness update is:

H_x^{(i+1)} = \mathcal{L}_{\text{harness}}(H_x^{(i)}, \mathcal{D}_x^{(i)}) \tag{5}

Importantly, $x_{\text{eval}}$ is used only by the evaluator, not directly provided to $\mathcal{L}_{\text{harness}}$ —RHI optimizes an implicit preference objective through accumulated pairwise comparisons.

Harness Representation

The harness is defined as the agent loop itself, decomposed into:

Agent Design: Roles and instructions of candidate agents
Agent Workflow: Further decomposed into:
- Contracts: What information is passed between agents through the communication interface
- Hops: The interaction structure (orchestrator-subagent workflow steps)

RHI prioritizes workflow updates over agent design, motivated by the hypothesis that task-specific contracts and hops improve both performance and inference efficiency through more effective context management (analogous to imposing a task-dependent sparsity pattern on inter-agent information flow).

Empirical Validation / Results

Experimental Setup

Models: Claude Sonnet 4.6, Claude Opus 4.7, Claude Opus 4.8 (all at "high" reasoning effort as base)
Benchmark: 30 synthetic open-ended ML research tasks (10 each in quantitative finance, robotics, pharmacy), generated from industry job postings
Evaluation: Pairwise LLM-as-a-judge comparisons using two evaluator configurations (gpt-5.5-max and opus-4.7/4.8-xhigh), three random seeds
Baselines: Higher test-time reasoning settings: xhigh, max, ultracode

Claim 1: Few-shot RHI Lifts the Performance Ceiling of Test-Time Scaling

Across all three base models, RHI-improved harnesses achieve higher pairwise win counts than all stronger test-time scaling baselines:

sonnet-4.6-high+H[2] wins 20/30 comparisons against sonnet-4.6-max
opus-4.7-high+H[1] outperforms both opus-4.7-xhigh and opus-4.7-max
opus-4.8-high+H[2] outperforms opus-4.8-xhigh, opus-4.8-ultracode, and opus-4.8-max

Claim 1.1: opus-4.8-high+H[2] outperforming opus-4.8-ultracode (which uses a built-in dynamic multi-agent workflow) demonstrates that a task-specific, user-constructed prompt-level harness can outperform a provider-built system-level harness.

Claim 2: Gains Not Explained by Increased Output-Token Usage

sonnet-4.6: Output-token usage stays nearly constant (1.71 → 1.86) across iterations while performance improves
opus-4.8: Output-token usage stays flat (1.42 → 1.81) while performance improves
opus-4.7: Evidence is inconclusive (only 2 iterations, tokens increase with performance)

Claim 3: Improved Cost Efficiency Through Reduced Cache Read/Write Usage

Model	Cost Reduction vs Max	Cache R/W Reduction vs Max	Cost Reduction vs Ultracode
sonnet-4.6-high+H[2]	7% (2.56 → 2.38)	33% (4.91 → 3.31)	—
opus-4.7-high+H[1]	18% (2.60 → 2.11)	37% (3.37 → 2.11)	—
opus-4.8-high+H[2]	23% (2.19 → 1.69)	32% (2.51 → 1.69)	60% (4.15 → 1.69)

Ablation: Train-Time Scaling Complementarity

RHI on sonnet-4.6-high does not consistently close the gap to opus-4.7-high/xhigh baselines—RHI complements rather than replaces train-time scaling.

Harness Component Evolution

Embedding analysis (t-SNE/UMAP projections, cosine similarity) reveals:

Contracts exhibit the clearest task-dependent clustering across all domains
Hops and instructions show intermediate task-dependent clustering
Roles are less separable (many tasks share common high-level roles)
First RHI update produces the largest semantic change (similarity 0.82); subsequent updates are incremental (0.97–0.99)
Within-domain across-task similarity increases after the first iteration

Information-Theoretic Implicit Objective

The authors hypothesize RHI implicitly optimizes:

J(g_i) = \underbrace{\sum_{\mathrm{hc} \in \mathcal{C}_{\mathrm{ext}}} \frac{1}{K_{Xi}^{\mathrm{hc}}} \sum_{k=1}^{K_{Xi}^{\mathrm{hc}}} I\left(z_{Xk}^{\mathrm{hc},(i)}; X\right)}_{f_{\mathrm{ext}}} - \beta \underbrace{\mathrm{TC}\left(\left\{z_{Xk}^{\mathrm{hc},(i)} : \mathrm{hc} \in \mathcal{C}, k \in [K_{Xi}^{\mathrm{hc}}]\right\} | X\right)}_{f_{\mathrm{int}}}, \quad \beta > 0 \tag{6}

Evidence for increasing $f_{\text{ext}}$ : Mutual information $I(\text{hc}; \text{task})$ increases monotonically for contracts (1.14 → 1.42) and hops (2.10 → 2.66) but decreases for roles (0.63 → 0.42) and stays flat for instructions (1.14 → 1.09).

Evidence for decreasing $f_{\text{int}}$ : Task-conditional total correlation decreases monotonically across all configurations (e.g., debiased estimate 4.84 → 3.63 nats with text-embedding-3-large), indicating reduced redundancy among harness components.

Theoretical and Practical Implications

Theoretical Implications

Harness-as-prompt formulation: RHI demonstrates that harness design can be treated as an explicit, task-specific optimization variable (a prompt-level specification) rather than fixed backend workflow code, enabling lightweight adaptation.
Information-theoretic account: The paper provides a testable hypothesis for what harness optimization implicitly does—maximizing task-relevant information in coordination components (contracts, hops) while minimizing redundancy across all components (functional specialization guidance).
Context management as sparsity: Optimizing contracts is conceptually analogous to imposing a task-dependent sparsity pattern on inter-agent information flow (similar to sparse attention in transformers), suggesting a principled connection between harness optimization and efficient context management.

Practical Implications

Cost efficiency: RHI can reduce inference cost by up to 60% while improving performance, making it practical for users specializing agents to many open-ended tasks under limited budgets.
Complementarity with scaling: RHI works alongside both train-time and test-time scaling—it raises the performance ceiling achievable through same-family test-time scaling without replacing the benefits of stronger base models.
User-accessible optimization: RHI requires only a few iterations (1–2) and is computationally lightweight ( $\Theta(1)$ per iteration), making it feasible for everyday users to continually specialize coding agents.

Conclusion

The paper argues that future progress in harness-model co-evolution should focus on improving the data flywheel—specifically, the quality of agent execution traces that can serve as post-training data for future foundation models. RHI is presented as a practical framework for the first half of this loop: optimizing user-constructed harnesses for task-specific execution-trace generation.

Key takeaways:

RHI is a computationally lightweight, few-shot method that substantially raises the performance ceiling of test-time scaling while reducing inference cost.
Gains arise from improved task-specific context management (contracts and hops), not longer reasoning traces.
RHI's learning dynamics are consistent with an information-theoretic implicit objective: increasing task information in coordination components while penalizing redundancy.

Future work: The authors plan to complete the second half of the loop by investigating how the resulting high-quality execution traces can be effectively internalized into future foundation models (e.g., via post-training).

Note: The paper includes substantial supplementary material (Appendices A–F) with RHI harness examples, optimizer prompts, task examples, evaluator prompts, and cost distribution analyses. The full harness evolution trajectories for specific tasks are provided in Appendix B, and the harness optimizer's system/user prompts in Appendix C.

Summary