Recursive Self-ImprovementIssue 14Nov 7 – 14, 2026

Self-improvement starts being tracked: costs, grounding, and verification

Highlights

The main theme of this issue is that "self-improvement starts being tracked": cost, grounding, and verification three lines simultaneously push the closed loop from "reporting the best once" to "reporting the complete account." On the cost side, The RSI Tax for the first time makes the token cost of the self-improvement process an explicit measurement object, decomposing it into three categories—verification/exploration/writing—and tracking the cost share associated with subsequent usage states: verification accounts for 74.8% of Meta-Harness, 78.1% of EvoSkill, and 93.6% of RRSI's RSI Tax, and more than half of the verification cost corresponds to states never used by subsequent tasks; its simple baseline SRN saves 65.6–71.0% of total tokens while matching or surpassing the accuracy of complex RSI methods, serving as a strong negative result on "whether complex self-improvement is worth it." On the grounding side, When Does Exploration Prevent Collapse provides a three-region closed-form analysis of self-consistency training collapse, proving that when the exploration term is propagated throughout the reward path, learning rate and optimizer scaling do not move the stability boundary, while bypassing one step (e.g., adding a bonus after GRPO normalization) shifts the threshold by 2–3 times; Wrong the Same Way turns cross-prompt parameter sharing itself into a bidirectional channel for reward hacking amplification/suppression, using per-prompt parameter copies as a control to show that sharing suppresses leakage accepted by few prompts and amplifies leakage accepted by many prompts, with the direction determined by the kernel-weighted quality distribution of the base model on the leakage rather than the verifier error rate. Agent Self-Improvement with Frozen Weights uses pre-registered matched no-evidence search to separate search and evidence gains, finding that no-evidence search alone contributes about 60% of the gain, and the point estimate of trace gain is below the pre-registered 5-point threshold.

On the verification side, LongTails explicitly defines "improvement" as achieving a lower oracle-complexity bound under a fixed formal Setup: the Lean 4 framework uses a Comparator to independently audit theorems and complexity bounds, and agents cannot pass checks by adding assumptions, omitting oracle calls, or leaving proof gaps; it has achieved 100+ new results. SwarmEvolve increases the throughput of evolutionary program search by 5–13 times: each round, 450 latent agents reason in embedding space without decoding text, use KV cache passing for crossover, and use cosine distance of hidden states for novelty filtering; the 8B model surpasses AlphaEvolve's frontier model results on three public problems.

Component decomposition continues to advance. Organon turns "the revision mechanism itself becoming the object of revision" into a recursive closed loop: a single persistent agent autonomously revises the evolutionary loop, parent selection, mutation operators, population management, and worker coordination during the search process, with revisions being diagnosable, correctable, or reversible (state-preserving reversal); revisions occurred in all 21 tasks, and in 14/21 tasks, more than 70% of gains appeared after the first revision. SkillDuet makes "skill utility collapsing with training" a measurable object, using a Strength score (the magnitude of log-prob shift in the current model's responses caused by a skill) to filter Local skills and periodically merge them into Global; fixed skill probes show that the same skill's usefulness to the model drops from +26.98 to -2.00 percentage points.

Community and dynamics

The Evaluator Dilemma systematically audits multiple RSI systems and points out that the "turning signal" and the "gain claim signal" are the same signal, and that this shared signal fails under gaming/grading/overfitting—this is the most direct cross-system negative result on the grounding of evaluators, corroborating this issue's Wrong the Same Way's "overlap of verifier error patterns."

FRI's evidence review on RSI treats the self-improvement rate as a measurable object, examining laboratory thresholds and conditions for self-sustaining acceleration, concluding that feedback loops are not yet strong enough and that measurement deficiencies weaken laboratories' ability to detect RSI onset—sharing the third-party perspective of "using laboratory disclosures as data sources" with last issue's METR audit.

The loop that found its own bug records a concrete failure in invalid run detection: a silent continue statement caused API call failures with exhausted quotas to be mistakenly recorded as model behavior, and held-out problems were silently discarded—metrics computation itself can hide invalid runs. Why AI scientists fail, in six patterns uses the Lossfunk case (3/4 failure rate) to give six failure patterns and points out the evaluation pitfall of relying on agent self-summaries rather than raw logs. AI agents don't work reproducibly if you don't tell them to quantifies agents' failure rates on version control, fixing dependencies, and saving source code, showing that explicit instructions are necessary—this provides practical evidence for "invalid run detection."

Open questions

  1. RSI Tax shows that verification accounts for as much as 74.8–93.6% of the RSI Tax in Meta-Harness/EvoSkill/RRSI, and more than half of the verification cost corresponds to states never used by subsequent tasks—should "verification cost" become a mandatory report item in any closed-loop evaluation? When the simple baseline SRN matches accuracy while saving 65.6–71.0% of tokens, what should be the criterion for judging "whether complex self-improvement programs are worth it"?

  2. Wrong the Same Way proves that shared parameters amplify leakage accepted by most prompts and suppress leakage accepted by few prompts, with the direction determined by the kernel-weighted quality distribution of the base model on the leakage—can this recommendation of "verifier repair should target model errors repeated across prompts" be converted into a precomputable leakage universality threshold? When two equal-size patches differ by 38 points in true accuracy, how much credibility does static auditing still have?

  3. LongTails defines "improvement" as lowering the oracle-complexity bound under a fixed Setup, using a Comparator to ensure proof correctness—can this formal verification generalize beyond optimization? When the Setup itself needs to be revised (as in Organon's recursive revision), how can formal guarantees and reversible revisions coexist?

  4. The debate from last issue on "whether experience should be stored in context or weights" receives new evidence this issue from SkillDuet: the same skill's usefulness to the training model drops from +26.98 to -2.00 percentage points—when skill utility collapses with training, should the criterion for "skill retention" shift from content quality to the response shift of the current model? Can the Strength score complement the external memory evolution of last issue's TTME in the same closed loop?

Papers in this issue

  1. LongTails: Automated Research in Optimization

    LongTails is the first framework to scale automated discovery of formally verified optimization complexity bounds, yielding over 100 new theoretical results.

    Editor's note

    Should read. It explicitly defines "improvement" as achieving a lower oracle-complexity bound under a fixed formal Setup, using Lean 4's Comparator to independently audit theorems and complexity bounds—agents cannot pass checks by adding assumptions, omitting oracle calls, or leaving proof gaps; it has achieved 100+ new results. Compared to existing formal verification work (CvxLean, AutoOPT), the increment lies in a unified Setup–Complexity abstraction that separates problem classes from reachable oracle budgets, scales across deterministic/random/composite/higher-order/zeroth-order settings, and reframes research as budget minimization under formal correctness constraints. Limitation: it doesn't disclose invalid run detection and failure rates, but as a reproducible, verifiable theoretical optimization discovery closed loop, it directly hits the 'self-improvement rate' measurement and component decomposition lines.

  2. The RSI Tax: When Recursive Self-Improvement Is Not Worth the Cost

    A simple bounded-note baseline (SRN) matches or beats costly recursive self-improvement methods' accuracy while cutting total token costs by 65.6–71.0%, revealing that verification-driven RSI taxes often exceed task execution costs.

    Editor's note

    Should read. It explicitly makes the token cost of the self-improvement process a measurement object, decomposing it into verification/exploration/writing categories and tracking the cost share associated with subsequent usage states; verification accounts for 74.8% of Meta-Harness, 78.1% of EvoSkill, and 93.6% of RRSI's RSI Tax, and more than half of the verification cost corresponds to states never used by subsequent tasks. Compared to the unified budget protocol in Issue 7's Rethinking the Evaluation of Harness Evolution, the increment lies in making "self-improvement cost" itself an independent measurement object and providing a cost-performance trade-off perspective; its simple baseline SRN saves 65.6–71.0% of total tokens while matching or surpassing complex RSI methods' accuracy, serving as a strong negative result on "whether complex self-improvement is worth it." Limitation: only two models, single run, and cost attribution is descriptive rather than causal intervention.

  3. When Does Exploration Prevent Collapse? Three Regimes and the Path of the Reward in Self-Consistency Training

    Self-consistency training exhibits three regimes of exploration strength, with a rare-answer bonus preventing collapse only when placed before GRPO normalization.

    Editor's note

    Should read. It provides a three-region closed-form analysis of self-consistency training (TTRL) collapse, proving that when the exploration term is propagated throughout the reward path, learning rate and optimizer scaling do not move the stability boundary, while bypassing one step (e.g., adding a bonus after GRPO normalization) shifts the threshold by 2–3 times, and derives precise thresholds for majority rewards and rare-answer bonuses. Compared to previous empirical studies on collapse, the increment lies in making "the position of the exploration term in the reward path" a provable source of stability boundaries, directly addressing the direction's concerns on reward grounding and self-play collapse. It reports honest failure data such as F95e collapse rate and pass@1 trade-offs, and distinguishes what exploration can do (it cannot correct erroneous majority answers). Limitation: theory targets single-problem/single-state, but the framework can be directly transferred to any closed-loop evaluation that relies on self-consistency rewards.

  4. Wrong the Same Way: Cross-Prompt Sharing Both Suppresses and Amplifies Reward Hacking in RLVR

    In RLVR, parameter sharing suppresses reward hacking for leaks accepted by few prompts but amplifies it for leaks accepted by many, with universality controlling the direction.

    Editor's note

    Should read. It turns cross-prompt parameter sharing itself into a bidirectional channel for reward hacking amplification/suppression: in a small fully enumerable environment, using per-prompt parameter copies as a control, it proves that sharing suppresses leakage accepted by few prompts and amplifies leakage accepted by many prompts, with the direction determined by the kernel-weighted quality distribution of the base model on the leakage rather than the verifier error rate. Compared to Issue 9's CATCH dynamic evaluation across training, the increment lies in elevating "sharing" from an implicit background to an intervenable independent variable, and providing clean per-prompt covariance decomposition and per-prompt copy controls; its "two equal-size patches differ by 38 points in true accuracy" is a direct warning to any closed-loop governance layer that relies on static auditing. Limitation: the environment is very small, but the mechanism and measurement protocol are transferable.

  5. SwarmEvolve: Latent Agent Swarms for Evolutionary Discovery and Optimization

    SwarmEvolve uses hundreds of low-cost LLM agents reasoning in latent space and communicating via KV caches to achieve 5–13x higher throughput and top scores, matching frontier-model results with an 8B model.

    Editor's note

    Should read. It scales a training-free mechanism of latent-space reasoning + KV cache communication to evolutionary program search: each round, 450 latent agents reason in embedding space without decoding text, use KV cache passing for crossover, use cosine distance of hidden states for novelty filtering, achieving 5–13x throughput improvement, and the 8B model surpasses AlphaEvolve's frontier model results on three public problems. Compared to Issue 7's Hill Sampling radical simplification, the increment lies in making "latent-space communication" itself the engine of evolutionary search, and reporting complete throughput of 734 programs/hour with three-seed means. Limitation: it doesn't report invalid run detection criteria, and the interpretability of latent-space reasoning is weaker than text reasoning.

  6. Agent Self-Improvement with Frozen Weights: The Added Value and Limits of Failure Traces

    Trace-based lessons improved decision accuracy by 4.0 points over matched search, but search alone captured 60% of the gain, with no improvement on conversations or held-out tasks.

    Editor's note

    Should read. It provides a controlled measurement of the causal contribution of failure trajectories to frozen-weight harness self-improvement: through pre-registered matched no-evidence search (same writer, loop, selection, only hiding episode content) separating search and evidence gains, it finds that no-evidence search alone contributes about 60% of the gain, and the point estimate of trace gain is below the pre-registered 5-point threshold. Compared to Issue 12's Self-Evolving Agents Can Repair Failures but Struggle to Generalize from Them failure-generalization gap, the increment lies in an explicit matched control protocol, harm constraint gating, and blind evaluation audit, directly quantifying the marginal value of "failure information"; its blind evaluation finding that 60.8% of BFCL instances are defective is a warning to any closed-loop evaluation relying on that benchmark.

  7. Organon: From Evolutionary Search to Recursive Self-Improving Agents for Scientific Discovery

    Organon, a recursively self-improving evolutionary agent that revises its own search strategies and tools during execution, outperforms all baselines across 21 tasks, including a 20.5% mean gain on ALE-Bench.

    Editor's note

    Should read. It turns "the revision mechanism itself becoming the object of revision" into a recursive closed loop: a single persistent agent autonomously revises the evolutionary loop, parent selection, mutation operators, population management, and worker coordination during the search process, with revisions being diagnosable, correctable, or reversible (state-preserving reversal); revisions occurred in all 21 tasks, in 14/21 tasks more than 70% of gains appeared after the first revision, and 12 explicit reversals occurred. Compared to Issue 12's AgenticEvolve persistent session experimental design, the increment lies in the recursive closed loop of "the revision mechanism itself becoming the object of revision" and the maintenance of historical evidence continuity through reversible revisions. Limitation: single run without multiple seeds and no ablation isolating individual revision contributions, but as empirical evidence that recursive self-improvement mechanisms can be revised, it has direct value for designing closed-loop governance layers.

  8. SkillDuet: Co-Evolving Models and Skills through Response Feedback

    SkillDuet co-evolves model parameters and reusable textual skills via response-based Strength scoring, achieving state-of-the-art 98.57% on ALFWorld and 92.97% on WebShop.

    Editor's note

    Should read. It makes "skill utility collapsing with training" a directly measurable benchmark, using a Strength score (the magnitude of log-prob shift in the current model's responses caused by a skill) to filter Local skills in on-policy trajectories and periodically merge them into Global; fixed skill probes show that the same skill's usefulness to the model drops from +26.98 to -2.00 percentage points. Compared to Issue 13's Beneath the Surface mechanism-level audit of harness evolution methods, the increment lies in first using "the magnitude of response shift of the skill on the current model" as an empirical selection criterion for long-term skill retention, and providing Global removal/neutral text controls and paired bootstrap intervals (Δ=20.95pp, CI [14.07,27.96]). Limitation: single model and task domain, but the separation evidence between skill content and model compatibility (the same Global harms SEED-API) has direct value for component-decomposed self-improvement.