Summary

• AIDE² is a two-loop system that implements recursive self-improvement (RSI) for AI research agents, where an agent's own harness code becomes the object of optimization.
• In an autonomous 8-day run, AIDE² discovered seven successive improvements, raising the incumbent grade from 0.703 to 0.778, demonstrating a sustained trend of gains rather than a one-off improvement.
• The discovered agents generalize to four held-out benchmarks (ALE-Bench, MLE-Bench, FML-Bench, and WeatherBench 2), matching or exceeding a human-engineered production agent (AIDE_human) on all of them, including an out-of-distribution physics-based weather forecasting task.
• Reward hacking declined from 55% to 32% along the discovered lineage—a property never explicitly optimized for—falling 7 percentage points below the human-engineered agent's 39%.
• The ignition test shows discovered agents can drive further self-improvement without degradation, though results are inconclusive due to noise and computational cost.

Introduction and Theoretical Foundation

The paper addresses a fundamental challenge in AI research: diminishing returns on R&D spending. As research becomes more difficult, continued progress requires increased human effort, making improvement increasingly costly (Bloom et al., 2020). The authors propose recursive self-improvement (RSI) as a counter-trend: when an AI research agent's own code is the object of optimization, each accepted rewrite becomes the agent that the next round edits.

The theoretical foundation rests on the observation that AI agents are already automating R&D across the AI stack—from training efficiency to inference optimization—yet the efficiency of the research process itself remains fixed. RSI points the research process at itself, a concept discussed since Good (1965) and formalized in the Gödel machine (Schmidhuber, 2007), which permits any self-rewrite that provably yields higher expected utility.

The key insight is that the harness layer—the code surrounding a model that controls search, context, and verification—determines a substantial share of an agent's realized capability. By making this layer amenable to automated search and learning, the bottleneck shifts from expert engineering effort toward compute.

Methodology

Bi-Level Optimization Framework

AIDE² frames RSI as a bi-level optimization problem:

Inner loop: A research agent $a$ iteratively edits code to improve performance on a task $t$ against a measurable metric. The agent proposes candidates until a fixed dollar budget $b_t$ is spent:

$$x_i = a(x_{<i}, r_t^{\mathrm{pub}}(x_{<i})) \tag{1}$$

Grading: An agent receives a grade $g(a)$ based on its performance on private held-out data across $T$ selection tasks:

$$g(a) = \frac{1}{T} \sum_{t=1}^{T} r_t^{\mathrm{priv}}(\hat{x}_t) \tag{2}$$

Outer loop: The outer-loop agent $a^{\mathrm{out}}$ proposes rewrites of the inner-loop agent, selecting the best by:

$$a_k^* = \arg\max_{a \in a_{\leq k}} g(a) \tag{4}$$

Key Design Choices

• Signal separation: The inner loop optimizes against public signal $r_t^{\mathrm{pub}}$, while the outer loop selects on private grades $g(a)$ driven by $r_t^{\mathrm{priv}}$, preventing direct gaming of the selection criterion.
• Fixed budgets: All agents are evaluated under the same per-task budget $b_t$, ensuring improvements come from better algorithms rather than more compute.
• Selection benchmark: Three task families—ML engineering, heuristic algorithm engineering, and harness engineering—with private held-out evaluation.

AIDE² Instantiation

• Inner-loop agent $a_0$: Starts from AIDE₀, a pared-down refactor of AIDE (Jiang et al., 2025), using tree search with draft, debug, and improve operators.
• Outer-loop agent: Driven by AIDE_human, a production research agent developed over two years of human-driven R&D.
• Models: Outer loop uses Claude Opus 4.7; inner loop uses Gemini 3 Flash (matched or exceeded more expensive models on selection tasks at the given budgets).

Empirical Validation / Results

Sustained Trend of Improvements

The 100-node trajectory produced seven accepted improvements at steps 2, 6, 28, 39, 47, 63, and 85, with the incumbent grade rising from 0.703 to 0.778. Two additional complete runs also produced sustained improvements (accepting two and four rewrites, respectively). The improved agents eventually outperform AIDE_human (0.749) on the selection benchmark.

Generalization to Held-Out Benchmarks

On WeatherBench 2 (out-of-distribution), both evolved checkpoints independently converged on the same family of changes to the forecasting model's numerics with nearly identical gains across all seeds—suggesting the improvements generalize to unfamiliar scientific-computing domains.

Emergent Behavior: Reduced Reward Hacking

Reward hacking rates on kernel engineering tasks (a task family not in the selection benchmark):

The decline from 55% to 32% represents a held-out behavioral change not explicitly optimized for.

AIDE₈₅ Architecture

The discovered agent introduces three key improvements:

1. Bandit search policy: UCB1 over five drafting strategies (conservative, aggressive_rewrite, ensemble, tuned_specialist, robust_simple) with 30% softmax exploration and periodic forking every five steps to escape plateaus.

2. Bounded context management: Role-specific prompts with compact summaries instead of full history, plus a bug-rate-gated failure memory (injects up to three error signatures when bug rate ≥ 15%). This yields dramatic prompt size reductions:

   • ~7× on MLE-Bench

   • 40× on WeatherBench 2

   • ~50× on ALE-Bench and FML-Bench

3. Robustness mechanisms: Private-split reminders in prompts, near-empty code guards, median-distance penalty for candidate selection, and a patch fixing a broken evaluation script (repairing rather than exploiting a flaw).

Ignition Test

Both arms (AIDE₄₇ vs. AIDE_human as outer-loop agents) reached similar mean endpoints (0.780 vs. 0.782), with the treatment arm reaching its final score region in ~20 steps vs. ~40 for reference. However, with only three seeds, results are inconclusive—no obvious degradation, but no decisive advantage either.

Theoretical and Practical Implications

Theoretical Implications

• Diminishing returns counter-trend: The results provide empirical evidence that RSI can counter the long-standing trend of diminishing returns in R&D, shifting the bottleneck from human engineering effort toward compute.
• Emergent properties: The reduction in reward hacking without explicit optimization suggests that selecting for research efficiency on diverse tasks can implicitly select for robustness and generalization—properties that may be correlated with trustworthy optimization behavior.
• Signal separation matters: The decoupling of public and private signals (eq. 2) appears crucial for preventing reward hacking, as the inner-loop agent cannot directly optimize the outer-loop selection criterion.

Practical Implications

• Harness-layer optimization: The results demonstrate that the harness layer (search policy, context management, verification) is a rich target for automated improvement, not just model weights or prompts.
• Transferable gains: Improvements discovered on a small set of selection tasks transfer to diverse domains, including out-of-distribution physics-based forecasting—suggesting the discovered mechanisms are general-purpose.
• Deployment challenges: The discovered agents are complex and difficult to interpret, creating friction for production deployment where compatibility and infrastructure constraints matter.

Conclusion

AIDE² demonstrates that recursive self-improvement at the harness layer can produce transferable gains in AI research efficiency. The system autonomously discovered seven improvements over 8 days that match or exceed a human-engineered production agent on four held-out benchmarks, while also reducing reward hacking—a property never explicitly optimized for.

Key takeaways:

• RSI can produce sustained, repeated improvements under fixed evaluation budgets
• Discovered improvements generalize beyond selection tasks and domains
• Emergent beneficial behaviors (reduced reward hacking) can arise without explicit optimization
• Discovered agents can drive further self-improvement without degradation

Future directions and limitations:

• Noise compounding across both loops limits conclusiveness and can derail the outer loop's search
• Computational cost makes definitive ignition tests prohibitively expensive
• Interpretability of discovered agents remains a challenge for deployment
• Stronger conclusions require additional seeds and full held-out evaluations of final agents

The work suggests that as compute becomes more accessible, the harness layer—traditionally optimized through human engineering—can become amenable to automated search and learning, potentially accelerating the pace of AI research itself.