Full text not available for this paper
Summary (Overview)
- GUARD (Grader-free UED with Adaptive Rule Distillation) is a closed-loop framework for reference-free continual evolution of agent harnesses (prompts, skills, tool rules, memory) around a frozen base LLM, without weight updates or external graders.
- The key insight is decomposing tasks into pre-execution self-checklists and guiding harness updates via paired directional regret, which separates newly broken behaviors (Δ_hurt) from unresolved capability gaps (Δ_gap).
- GUARD uses GRPO-inspired parallel candidate pooling with critic-free group-relative fitness for selection, and a Soft-HURT promotion gate that accepts edits only when gains outweigh regressions, guaranteeing non-regression on gated task buffers.
- Across benchmarks (PAST-Bench, EvoHarnessBench, Drift Gauntlet, professional suites), GUARD achieves 19/19 non-regressive promotions, zero post-block forgetting under hidden drift, and 54.5% token reduction per task, with monotonic accuracy gains.
- Three theoretical guarantees are provided: non-regressive gating with concentration bounds (Theorem 1), removal of the 1/ε passive-discovery penalty via dense checklist grading (Theorem 2), and strict regression dominance in priority ranking (Proposition 1).
Introduction and Theoretical Foundation
Background and Motivation
Modern language model agents pair a frozen base model with an editable harness of prompts, skills, tool interfaces, memory, and control logic. Recursive self-improvement (RSI) adapts this harness from experience without weight updates—a practical approach since frontier models sit behind black-box APIs and open-world tasks rarely provide ground-truth rewards.
Core problem: Self-editing without external graders often backfires. Free-form reflection and scalar scores conflate two distinct failure signals:
- Edit-induced regressions (behaviors the latest edit newly broke)
- Unresolved capability gaps (things the agent still hasn't solved)
Mixing these signals causes three failure modes:
- Local overfitting (over-triggering): broad rules written for recent failures break tasks that already worked
- Catastrophic forgetting: shifts in workflow/topic/domain overwrite earlier skills
- Memory pollution: unverified additions accumulate stale or conflicting rules that bloat the prompt
Theoretical Foundation
The framework draws on two ideas from reinforcement learning:
- Curriculum regret from Unsupervised Environment Design (UED) [7, 14, 15, 35] — prioritizing high-regret levels, transferred here to directional criterion ranking over task checklists
- Group Relative Policy Optimization (GRPO) [37, 3] — critic-free group-relative scoring, applied to non-parametric harness optimization without weight updates
Problem formulation: An agent pairs a frozen policy with an editable harness state at round . The reference-free evolver updates:
The goal is maximizing where is the unobserved task evaluator.
Reference-free means neither the solver, self-reviewer, nor distiller accesses held-out answers, test cases, hidden verifiers, oracle skills, or rubrics—but the framework still leverages observable structure: task prompt distributions, tool declarations, interaction trajectories, and criterion-level self-review judgments.
Methodology
GUARD instantiates the evolver as a four-stage closed loop:
Stage 1: Reference-Free Self-Review via Task Checklists (§4.1)
Pre-execution checklist: The self-reviewer reads the task prompt, tool declarations, and schema once at Turn 0 to derive a fixed checklist of atomic binary predicates across two classes:
- Execution-verifiable predicates: clean sandbox execution, valid entity IDs/columns, idempotent mutations (checked deterministically on an ephemeral sandbox copy)
- Semantic predicates: honoring prompt constraints, recipient, format, safety rules (graded individually from prompt, output, and trace)
Two-layer grading: Trajectory receives criterion score and strict score :
This exploits generation–verification asymmetry: checking one atomic predicate after execution requires localized attention, while long-horizon generation couples many constraints.
Stage 2: Paired Regret Ranking — Analyze (§4.2)
Definition 1 (Criterion obedience and paired regret): For trajectories grading criterion under harness :
Comparing the two most recent incumbents and :
Criteria are partitioned into three diagnostic lists: Regressed (Δ_hurt > 0, wrap offending rules in [iff] preconditions), Unsolved Gap (Δ_gap > 0), and Mastered (Priority = 0, pinned against overwriting).
Stage 3: Group-Relative Candidate Pooling — Propose (§4.3)
Inspired by GRPO, GUARD samples candidate harnesses in parallel, scores them on the same task batch, and ranks by critic-free group-standardized fitness:
ContrastEvolve contrasts the best () and worst () candidates to update strategy prompts:
Stage 4: Soft-HURT Promotion Gate — Gate (§4.4)
The group-averaged regression penalty:
Soft-HURT rule: Promote candidate over only when:
When a candidate fails solely due to isolated regressions (), GUARD triggers at most one in-round [iff] scope-guard repair. Rejected edits are logged to a rejection ledger so bad changes are not repeated.
Theoretical Guarantees (§4.5)
Theorem 1 (Non-Regressive Gating and Concentration): Under the gate of (12):
- (i) No downward drift:
- (ii) Regressions are paid for:
- (iii) Concentration: with probability ≥ 1−α,
- (iv) Factored speedup from per-group gating
Theorem 2 (Targeted Repair vs. Passive Discovery): Passive learning requires rounds to discover rare invariants (each surfacing with probability ), while dense checklist grading achieves , removing the dependence.
Proposition 1 (Strict Regression Dominance): In the single-rollout binary case, a fresh regression (1→0, priority 2) outranks every non-regressed criterion (priority at most 1). Priority iff and — no regressed or failed criterion is ever pinned.
Empirical Validation / Results
PAST-Bench: Cross-Session Memory Write Path (Table 1)
| Arm | Task | Proc. | Mech. | Upd. | W.Prec. | Recall | Ret. | Poll.↓ | Evolve |
|---|---|---|---|---|---|---|---|---|---|
| No-Evolve (W₀) | 0.372 | 0.427 | 0.454 | 0.278 | 0.000 | 0.000 | 0.990 | 0.000 | |
| Hermes+ | 0.852 | 0.808 | 0.840 | 0.791 | 0.724 | 0.854 | 1.000 | 0.167 | |
| RRSI | 0.809 | 0.714 | 0.725 | 0.901 | 0.438 | 0.811 | 0.974 | 0.500 | |
| GEPA | 0.858 | 0.815 | 0.837 | 0.777 | 0.719 | 0.858 | 0.996 | 0.167 | |
| Meta-Harness | 0.812 | 0.677 | 0.739 | 0.903 | 0.479 | 0.814 | 0.998 | 0.500 | |
| GUARD (Ours) | 0.854 | 0.830 | 0.878 | 0.924 | 0.776 | 0.856 | 0.999 | 0.167 | 0.436 |
GUARD leads procedural task (83.0%), procedural mechanism (90.0%), and update (90.1%) at 0.000 procedural pollution, with 0/48 family losses on W.Prec. or Poll. across all baselines.
EvoHarnessBench: Continual Evolution (Table 2)
| Method | ACC | Solved | BWT Score | BWT Pass | FWT | Tokens/Task↓ | Calls↓ |
|---|---|---|---|---|---|---|---|
| No-Evolve (W₀) | 71.91% | 48.84% | −6.00 pp | −5.13 pp | – | 131,032 | 14.47 |
| MemToolAgent | 68.00% | 51.16% | −4.02 pp | +2.56 pp | −4.98 pp | 82,426 | 15.84 |
| RRSI | 74.28% | 51.16% | −0.45 pp | 0.00 pp | +1.79 pp | 84,519 | 13.19 |
| GEPA | 70.69% | 48.84% | −3.97 pp | −2.56 pp | +13.61 pp | 85,156 | 17.05 |
| Meta-Harness | 74.52% | 51.16% | +5.83 pp | +5.13 pp | −13.21 pp | 78,705 | 14.19 |
| GUARD (Ours) | 80.80% | 58.14% | +0.85 pp | +2.56 pp | 0.00 pp | 59,651 (−54.5%) | 11.21 (−22.5%) |
GUARD is the only arm with no negative backward or forward transfer, achieves 0/43 budget exhaustions (vs. 6–10 on baselines), and solves 9.75 tasks per 1M tokens (1.50× the best baseline).
Drift Gauntlet: Continual RSI Under Hidden Drift (Table 3)
| Method | Final Mean | Δ_K | ¯Δ | Plast. P | Forgetting F | Stab. S | Block Drops↓ |
|---|---|---|---|---|---|---|---|
| No-Evolve (W₀) | 69.92% | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0/12 |
| Hermes+ | 70.31% | +0.40 | +1.09 | +2.67 | −5.59 | −2.66 | 5/12 |
| RRSI | 67.92% | −2.00 | −0.47 | +4.49 | −8.11 | −6.02 | 8/12 |
| GEPA | 72.40% | +2.48 | +2.18 | +2.80 | −1.09 | −0.35 | 3/12 |
| Meta-Harness | 71.21% | +1.30 | +0.15 | +3.28 | −0.07 | −2.51 | 4/12 |
| GUARD (Ours) | 73.22% | +3.30 | +2.71 | +4.55 | 0.00 | −0.93 | 0/12 |
GUARD is the only arm whose stream-mean accuracy never decreases across all 12 block transitions, achieving both top plasticity (+4.55 pp) and zero post-block forgetting.
Self-Reviewer Calibration and Gate Fidelity (Table 4)
- Layer-1 sandbox checks curb LLM-only optimism: 22.2% of execution-broken traces pass 1-layer review vs. 0.0% under 2-layer
- 19/19 execution-gated promotions satisfy Δ_test ≥ 0 (15 strictly > 0, 4 ties)
- Removing Analyze drops macro ACC by −7.33 pp; removing Soft-HURT drops it by −10.35 pp
- Parallel M=2 pooling (81.08% ACC) beats matched-budget sequential search (78.85%, p=0.0267) with 64% fewer rejected gate rounds
Theoretical and Practical Implications
Theoretical Contributions
-
Non-regressive gating guarantee: Theorem 1 proves self-review scores never decrease on gated task buffers, with concentration bounds that scale as samples
-
Removal of passive-discovery penalty: Theorem 2 shows dense per-criterion grading eliminates the dependence on rare task invariants that plagues passive end-task learners
-
Strict regression dominance: Proposition 1 proves paired priority uniquely satisfies both exact pinning (no regressed/failed criterion is pinned) and strict dominance (fresh regressions outrank static gaps), unlike frequency ranking or signed differences alone
Practical Implications
- Deployment efficiency: GUARD cuts tokens per task by 54.5% and MCP calls by 22.5% while improving accuracy—the compact playbook (2,315 chars vs. 4,538–10,013 on baselines) prevents budget exhaustion
- No catastrophic forgetting: By keying playbook sections to pre-execution observable tool/schema signatures and gating with Soft-HURT, GUARD achieves zero post-block forgetting across all four drift streams
- Inspectability: Unlike weight-space RL, GUARD produces inspectable conditional rules with explicit [iff] preconditions, aiding debugging and trust
- Teacher–student applicability: GUARD works both in strictly self-referential single-model settings (Gemini 4 for all roles) and cross-model teacher–student distillation pipelines
Conclusion
GUARD addresses the central challenge of continual harness RSI without external graders by treating the text harness as a non-parametric policy component updated via regret-guided, group-relative evolution:
- Analyze separates edit-induced regressions (Δ_hurt) from unresolved gaps (Δ_gap) while pinning mastered criteria
- Propose selects among parallel candidate edits via critic-free group-standardized fitness and contrasts best/worst candidates to rewrite proposal prompts
- Gate (Soft-HURT) promotes only net-positive updates with formal non-regression guarantees
Across four benchmark settings, GUARD improves upon or matches the base agent in held-out accuracy, achieves 19/19 non-regressive execution-gated promotions, sustains monotonic pass@1 gains with zero post-block forgetting under hidden stream drift, and cuts hosted tokens per task by 54.5%.
Future directions suggested by the work include: extending the framework to more complex harness structures (code-level control logic), exploring the interplay between signature routing and replay-buffer design, and investigating whether the theoretical guarantees can be strengthened to finite-sample certificates at benchmark task counts.
Related papers
- When Can Self-Play Fix a False Belief? Answer Keys, Pass Bands and Replacement Rules in an Execution-Grounded Loop
Self-play with execution-grounded keys cannot correct false beliefs because the generator's own code inherits its misreadings, and decorrelated keys are necessary but insufficient for correction.
- BuggyMLE: Scaling Machine Learning Engineering Difficulty with Bug Injection
BuggyMLE generates 6,600+ bug-injected ML repositories and a structural complexity metric (CCI) predicting solve rates with R²=0.951, enabling difficulty-graded benchmarks where a 9B model rivals frontier agents.
- The Resolution Hypothesis: Agent Evolution's Key Is Identifying Improvements
Agent evolution fails because noisy evaluations cannot identify helpful improvements, not because proposals are lacking, making selection nearly random.