Self-Improving Measurement Itself Begins to Be Measured
Highlights of This Issue
The main theme of this issue is "self-improving measurement itself is being measured"—the field is beginning to audit its own evaluation practices through large-scale empirical studies, advancing key risks from "detection during training" to "prediction before training."
Auditing the Product, Not the Recursion conducts a full-text coding census of 114 RSI studies within a frozen arXiv frame, separating "product claims" from "process claims": 90% of studies report product evaluations beyond the development task, but only 22% audit two or more complete loop reruns, and only 4 of 26 studies claiming "better improvement procedures" compare before-and-after improvers using identical starting points and matched budgets. It offers three propositions explaining why product evidence cannot substitute for process evidence, and proposes a claim-indexed reporting standard. Compared to the community column in Issue 10's structured audit of 13 systems, the increment here is the full-text coding at full-domain scale and the reproducible claim-indexed standard.
SimHack makes the onset prediction of reward hacking a pre-training measurement target for the first time: it learns prompt-level behavioral dynamics from a small number of reference runs, and counterfactually simulates hacking trajectories and onset distributions for unseen recipes without executing the target RL run. Median onset relative error is 11.3%, cross-recipe rank correlation is 0.85, and curriculum filtering application delays onset by 96.6%. Compared to CATCH's full-training dynamic evaluation in Issue 9, the increment lies in advancing prediction from "detection during training" to "prediction before training."
The decomposition of harness gains is also advancing. Elicitation or Expansion? uses three diagnostic axes—pass@k curves, stronger reference systems, and task difficulty stratification—to prove that harness optimization's pass@1 advantage reverses under larger attempt budgets, and that aggregate curves mask coverage losses on difficult subsets—a critical warning for any closed-loop relying on "successful experience" for recursive self-improvement.
On the ML-for-science side, Agent4Sci turns "scientific hill-climbing" into a standardized testbed across 11 domains and 20 benchmarks: all benchmarks are surpassed by at least one agent, but the gains come almost entirely from ML engineering tricks rather than research insights; lightweight plug-in skills (research ideation, experiment management, timed checklists) raise Opus from 10.4% to 20.1%. EvalResearchBench makes "evaluator design" itself a measured object: agents select/synthesize tasks under fixed budgets, implement and refine graders, and freeze executable evaluators, with the best evaluator ranking 75% of candidate pairs, below the 91% upper bound set by inter-target disagreement.
On the closed-loop constraint side, Multi-Reward Hacking formalizes for the first time "multi-reward aggregation itself" as an independent source of reward hacking: it proves that when auxiliary rewards are more easily obtainable, any symmetric aggregation rule causes GRPO to converge to auxiliary-preference behavior, and proposes structural gating defense via Trust-Anchored Gating.
Community and Dynamics
The more you automate, the less you see is the first peer-reviewed critique of failure modes in closed-loop AI-scientist systems: benchmark order sensitivity, undocumented synthetic data, and automated p-hacking through post-hoc selection. The key empirical finding is that paper-only audit detection accuracy is near random (53.7%), while access to full execution records (timestamped decision logs, intermediate outputs, code) raises it to 81.5%—a direct argument that "invalid run determination" requires workflow-level evidence.
Designing Reliable Harnesses for Autonomous Machine Learning Experiments systematically catalogs failure modes and run validity criteria for the first time at the scale of 15,000+ autonomous experiment runs, treating the harness itself as the research object rather than a single benchmark result—directly responding to the field's preference for invalid run determination.
Is Recursive Self-Improvement Really Here? argues that RSI cannot be verified on any benchmark, requiring write-protected holdout sets and differential auditing—a position paper advancing disclosure from a wish list to a concrete governance proposal.
hacker-fixer loop reports a quantifiable closed-loop hardening process: an automated loop detects reward-hacking trajectories and fixes the environment, reducing the unsolicited attack success rate on Terminal-Bench from 69.9% to 12.1% (1,860 tasks, 3,632 hacking trajectories), while maintaining a benign solution rate of 91.2%. It makes "the system improving its own evaluation environment" rather than weights the research object, but as a secondhand report lacking run-level details, it is best treated as a pointer to the original work.
Open Questions
- Auditing the Product shows only 4 of 26 studies claiming "better improvement procedures" compare before-and-after improvers with identical starting points and matched budgets—why are improver comparisons so rare? Is it cost constraints, or does the field's research claim structure itself fail to support such designs?
- SimHack's onset prediction relies on reference runs from the same source as the target and is validated only on LeetCode coding RL—can onset prediction generalize across task domains? Is the 96.6% delay from curriculum filtering stably reproducible in real training?
- Elicitation vs Expansion shows harness optimization's pass@1 advantage reverses under large budgets and coverage losses on difficult subsets are masked by aggregation—does this mean "coverage expansion" claims in harness evolution are systematically overestimated? Can selection methods be designed that reward verified hard-task success and retain complementary candidates?
- Agent4Sci's gains come almost entirely from ML engineering rather than research insights, and plug-in skills double gains but on a single benchmark without universality—can "research insights" become measurable, trainable, and verifiable components rather than context that can only be injected manually?
Papers in this issue
Only 22% of recursive self-improvement studies audit repeated runs, and just 15% of improver-gain claims use identifying same-start comparisons, revealing a systematic product-versus-process evaluation gap.
Editor's noteShould read. It advances the measurement of "self-improvement rate" from single papers to domain-wide census: full-text coding of 114 RSI studies within a frozen arXiv frame, separating "product claims" from "process claims," quantifying that 90% report product evaluations but only 22% audit two or more reruns, and only 4 of 26 improver claims use matched budgets and identical starting points. Compared to Issue 10's community column structured audit of 13 systems, the increment lies in full-text coding at full-domain scale, three propositions explaining selection effects/shared evaluator errors/state changes, and a claim-indexed reporting standard binding each claim to required evaluation design. It is an anchor point for designing any self-improvement closed-loop evaluation.
SimHack predicts reward hacking onset in LLM RL training from reference runs alone, achieving 11.3% median onset error and enabling curriculum selection that delays hacking by 96.6%.
Editor's noteShould read. It makes reward hacking onset prediction a measurable pre-training target: a lightweight behavioral simulator learns prompt-level dynamics from a few reference runs, counterfactually simulating hacking trajectories and onset distributions for unseen recipes without executing the target RL run. Compared to Issue 9's CATCH full-training dynamic evaluation and Issue 7's Reward Hacking Challenges external behavioral evidence, the increment lies in advancing prediction from "detection during training" to "prediction before training," with generalization evidence across model scales, curricula, rollout counts, and learning rates (median onset relative error 11.3%, cross-recipe rank correlation 0.85). Its curriculum filtering application delays onset by 96.6%, making it a strong pre-training risk screening tool.
Frontier coding agents improve every scientific ML benchmark they attempt, but gains stem from engineering tricks, not scientific insight.
Editor's noteShould read. It turns "scientific hill-climbing" into a standardized testbed across 11 domains and 20 benchmarks: starting from published SOTA, agents improve under fixed budgets, all benchmarks are surpassed by at least one agent, but gains come almost entirely from ML engineering tricks rather than research insights. Compared to Issue 10's Into the Unknown scientist standard formalization, the increment lies in a reproducible 25-task/20-benchmark testbed, full standardization pipeline (497→10 funnel, 98% rejection rate), and a three-stage evaluation protocol making evaluation gaming an explicit audit object, with plug-in skills raising Opus from 10.4% to 20.1%. Limitations: no invalid run determination criteria reported, no human researcher baseline.
Harness optimization mostly elicits existing capabilities rather than expanding solvable task range, with pass@1 gains reversing at higher attempt budgets.
Editor's noteShould read. It systematically distinguishes for the first time whether harness optimization gains are "elicitation" (making already-solvable tasks more reliable) or "expansion" (broadening solvable coverage), using three diagnostic axes—pass@k curves, stronger reference systems, and task difficulty stratification—revealing: pass@1 advantages reverse under larger attempt budgets, aggregate curves mask coverage losses on difficult subsets, and pass@64 falls below baseline under OOD transfer. Compared to Issue 9's Harness Evolution Hits a Ceiling failure composition diagnosis, the increment lies in making "elicitation vs expansion" an explicit measurement object, and its finding that "hard-focused optimization only improves pass@1, not pass@128" provides new grounds for choosing between harness evolution and weight training levers.
AI agents can design evaluators achieving only ~75% agreement with target benchmarks, far below the 91% human-benchmark ceiling, revealing a key bottleneck for recursive self-improvement.
Editor's noteShould read. It makes "evaluator design" itself a measured object: under fixed time and API budgets, researcher agents select/synthesize tasks, implement and refine graders, and freeze executable evaluators, with evaluation criteria being scoring agreement (CCC and pairwise agreement) with the evaluated model across 14 target benchmarks. Compared to Issue 10's DuoDiscover yardstick co-evolution and Issue 9's Who Verifies the Verifier verifier evolution, the increment lies in elevating "evaluator construction" from co-evolution object to independent measurement target, explicitly separating development/sealed targets, reporting construction and execution costs, and a 91% ranking upper bound. Limitation: single episode without multiple seeds, but its protocol and cost decomposition have direct value for designing closed-loop governance layers.
Symmetric reward aggregation in LLM post-training provably causes multi-reward hacking, favoring auxiliary rewards over correctness, and Trust-Anchored Gating reverses this collapse.
Editor's noteShould read. It formalizes for the first time "multi-reward aggregation itself" as an independent source of reward hacking: proving that when auxiliary rewards are more easily obtainable, any symmetric aggregation rule (additive, normalized, constrained) causes GRPO to converge from arbitrary initialization to auxiliary-preference behavior, and proposing Trust-Anchored Gating—electing an anchor channel via isolated policy drift, then eliminating auxiliary-only credit with structural gating. Compared to Issue 7's Reward Hacking Challenges external behavioral evidence and Issue 9's CATCH full-training dynamic evaluation, the increment lies in making "aggregation rules" a provable failure source with convergence guarantees, reporting complete failure rates of pass@4 collapse to 0.02–0.22 and an offline probe protocol for anchor election.
Explorer-Zero jointly generates compositional tasks and verifiable subtask rewards via self-play, outperforming human-data RL on AppWorld without annotations and improving subsequent RL.
Editor's noteShould read. It makes "reward collapse" in self-play a measurement object: decomposable, verifiable subtask rewards mitigate gradient vanishing in long-horizon tasks from binary rewards, and the learned proposer generates tasks through environment interaction while recording intermediate states as verification targets. Compared to Issue 7's Reward Hacking Challenges external behavioral evidence, the increment lies in theoretical proofs (subtask rewards reduce collapse probability when all trajectories fail) and self-play evidence without human-annotated tasks (ALFWorld +18.67, AppWorld 30B reaching 59.67% near human-data RL's 62.65%), with self-play pretraining improving subsequent human-data RL. Limitation: evaluation mainly in benchmark environments, no complete invalid run determination reported.
Offline model-authored skills, frozen after simulation qualification, run on robots with no model or a cheap caller at 98% success and under a cent per episode.
Editor's noteShould read. It systematically separates for the first time "author-executor" in embodied manipulation: skills are compiled offline by strong models, frozen, then executable by pure code or cheap cross-family callers, with a three-way contrast of learning vs selection vs inheritance, quantifying that inheritance dominates cross-session improvements with marginal contribution from development feedback. Compared to Issue 10's Athanor weight-training closed loop, the increment lies in making the "compile-reuse" boundary a measurable object and separating inheritance from feedback via preregistered experiments, reporting complete failure rates (13/20 uncompiled, 5/12 gated by safety aborts) and invalid run determination. Limitation: tasks are simulated, no direct online control baseline.
Self-evolving LLM agents trade stability for plasticity, failing as continual learners because global prompt edits cause cross-task interference and similarity-based retrieval collides on conflicting workflows.
Editor's noteShould read. It systematically evaluates for the first time non-parametric self-evolving agents (ReasoningBank, GEPA, Meta-Harness) as continual learners under drifting task streams, decomposing performance into plasticity and stability, and identifying specific interference mechanisms (pre-block interference, post-block forgetting from prompt compression, retrieval conflicts under similar queries). Compared to Issue 8's EVOHARNESSBENCH harness non-stationarity and Issue 9's Agent Plasticity artifact-level failure diagnosis, the increment lies in an explicit continual learning framework and per-group decomposition under four drift types, with the core negative result that "higher plasticity consistently comes at the cost of stability." It provides per-group decomposition, checkpoint evaluation, and failure analysis, aligning with preferences for reproducible closed loops and transparent failure reporting.
