Metaⁿ: Recursive Self-Improvement through Emergent Depth

Summary (Overview)

• Core contribution: Metaⁿ introduces a novel recursive self-improvement architecture where a single fixed meta-operation Ω is applied repeatedly to its own outputs, rather than modifying the improver itself. This dissolves the stability-vs-depth dilemma inherent in prior approaches.
• Key mechanism: Each application of Ω inspects the traces of the solver stack below and the code that produced them, enabling higher-order reasoning (e.g., rolling back harmful directives) that flat self-refinement cannot perform.
• Empirical results: Across 8 benchmark families and 2 backbones (Gemma 4 31B-IT, GPT-5.2), Metaⁿ outperforms prior self-improving agents (Gödel Agent, OpenEvolve) on every benchmark family, with the largest margins on the hardest tasks.
• Notable finding: On ARC-AGI-2 (designed to resist skill memorization), Metaⁿ is the only system to achieve above-floor scores (0.331 vs. 0.003–0.054 for baselines).
• Ablation insight: ~72% of recursion's gain comes from the conditioning channel (a simple context string passed between layers), ~15% from callable code libraries, and ~13% from recursion machinery itself.

Introduction and Theoretical Foundation

The Problem with Existing Meta-Improvement

The paper frames three paradigms of meta-improvement:

1. Hand-crafted meta-systems (FunSearch, AlphaEvolve, ADAS): A fixed external process rewrites the agent, but the meta-mechanism never changes → realized meta-depth = 1.

2. Self-referential agents (Gödel Agent, DGM): The agent edits its own source, but a frozen driver layer caps realized meta-depth at ~2.5. The editable surface is a strict subset of the improvement machinery.

3. Recursive n-layer meta (Metaⁿ, ours): One meta-operation Ω is applied at every depth and accumulates, so version v^d carries d nested meta-processes; depth is set by convergence.

The Core Insight

The key reframing: recursing the improver buys depth only at the price of stability. Metaⁿ dissolves this dilemma by:

• Freezing the improver (Ω) by design
• Applying recursion to its input instead
• Each application operates on strictly higher-order inputs (traces + code stack)
• The operation never mutates, so it cannot destabilize the system

Methodology

Problem Formulation

A benchmark supplies N tasks and an evaluator scoring candidate scripts on [0,1]. Each solver run leaves an execution trace τ (script, stdout/stderr, exit code, score, feedback). The goal is a stack of solvers maximizing mean score, with depth determined by convergence rather than prescribed.

The Meta-Layer at Depth d

Build-step (writing the layer): Ω is a fixed LLM-prompted procedure that takes:

• Previous depth's traces {τᵢ^(d-1)}
• Stack of prior code [C₂, ..., C_{d-1}]
• Task descriptions T
• Current depth d

and outputs new code C_d = (f_pre^(d), L^(d)), where:

• f_pre^(d): a pre-process Python function injecting strategic context
• L^(d): reusable helper functions the solver may call

Run-step (executing the layer): The wrapper M_d slots C_d around the previous solver in five stages:

1. Outermost pre-process runs, producing a context string
2. Context threads inward through lower layers
3. Base solver sees merged contexts, returns a script
4. Union of code libraries prepended to script
5. Sandbox-executed to produce depth-d trace

The depth-n solver is nested composition: S_d = M_d ∘ M_{d-1} ∘ ... ∘ M₂ ∘ S₁

Orchestration

Linear meta-recursion: Greedily deepens the stack; stops when Ω returns empty code, P consecutive layers fail to improve, or max depth D reached.

Evolutionary archive: Maintains a monotonically growing archive of candidate chains; samples parents with weight w(c) ∝ S̄(c) + α/(1+children(c)); produces K children per parent with temperature cycling; tracks per-task best scores.

Consolidation guard: Each candidate targets one focus task while inheriting frozen best traces for others, guaranteeing monotone per-task-best trajectory.

Empirical Results

Main Results (Gemma 4 31B-IT)

Key Findings

ARC-AGI-2 (categorical case): Object-level iteration stays near floor (0.123 best single chain), but full meta-level stack reaches 0.331. Baselines: OE 0.003, GA 0.054.

CO-Bench: Archive recovers tasks a single LLM call cannot solve (constrained guillotine cutting: 0.000→0.996; maximal independent set: 0.000→0.908). Archive-best leads OE by +0.168 on GPT-5.2 with disjoint per-seed ranges.

Emergent roles: Despite no prompt prescribing roles, a progression emerges:

• Depth 2: generic tactical primitives (local_search, simulated_annealing)
• Depth 3: specialization + layer interference (41% of task pairs regress)
• Depth 4-6: corrections and refinements (rollback intent near 50% of emissions)

Baseline parity: Metaⁿ outperforms OE at compute parity (S2D 0.732 vs. 0.718; LawBench 0.784 vs. 0.745) and uses ~13× fewer candidate evaluations on CO-Bench.

Ablation Results

Theoretical and Practical Implications

Theoretical Significance

1. Dissolves the stability-depth dilemma: By freezing the improver and recursing on input, Metaⁿ achieves unbounded meta-depth without destabilization.
2. Multiplicative coverage: Conditioning between layers admits up to Π k_d joint configurations vs. Σ k_d for flat architectures (27 vs. 9 at n=4, k_d=3).
3. Strict information monotonicity: Ω's information at depth d is a strict superset of its information at depth d-1 (traces + code stack).

Practical Implications

• Where Metaⁿ pays off: Benchmarks with diverse failure modes (CO-Bench's 36 NP-hard shapes, SR-matsci's 25 laws, TB2's 13 categories)
• Where it doesn't: When seed is already strong (SWE-Bench) or when pre-optimized contracts leave little headroom (AlgoTune, where over-constraint hurts)
• Compute efficiency: More sample-efficient than baselines (29 vs. 378 evaluations on CO-Bench)

Limitations and Future Work

1. Same model at all layers: Correct control but leaves untested the practical case of a stronger model at Ω over a weaker base
2. Conditioning channel dominance: Could enrich context with structured/typed representations
3. Depth limits: Runs stop at depth 3-6 because Ω stops finding improvements, not because of model capacity limits

Conclusion

Metaⁿ demonstrates that recursive self-improvement can achieve genuine meta-depth beyond prior systems' ~2.5 cap by holding the improver fixed and recursing on its input. The architecture produces emergent role differentiation across depths, achieves state-of-the-art results on diverse benchmarks, and provides the first empirical demonstration that meta-depth beyond two yields structurally distinct levels rather than redundant ones.