Masked Erosion: How Acceptance Gates Lose Redundant Safeguards in Self-Improving Agents

Summary (Overview)

• Core phenomenon identified: When self-improving LLM agents edit their own harness (prompts/rules around a fixed model), deleting a redundant safeguard is "masked" — other overlapping safeguards still block the harm, so acceptance gates cannot detect the loss of redundancy.
• Key mechanism: Introduces the concept of margin — the number of further safeguard failures a harness can absorb before harm appears — and shows that acceptance gates measure only current safety, not robustness to future failures.
• Empirical finding: Across 411 improvement loops and 3 GEPA optimizer runs in two environments (AppWorld and OpsSim), harm audits prevented exposure but not depth loss; gate protection breaks were predictable before runs using a frozen statistical simulator.
• Critical result: Gates with significance testing (e.g., paired McNemar tests) suppress random deletions but fail against genuinely beneficial deletions, and larger evaluation batches (40 vs. 20 tasks) paradoxically admit cheaper safeguard deletions.
• Mitigation: Freezing safety text or auditing one-layer ablations (mutation adequacy applied to safeguards) preserves redundancy depth, while off-the-shelf optimizers like GEPA kept rules verbatim yet still changed behavior under weaker models.

Introduction and Theoretical Foundation

Background and Motivation

Self-improving agents (ADAS, Darwin Gödel Machine, GEPA, ACE, Self-Harness) rewrite their own harness — the prompts, notes, and tool rules around a fixed model. Most keep a rewrite only if an acceptance gate finds the new harness performs better on a batch of tasks. Self-evolution can degrade safety; notably, the Darwin Gödel Machine removed logging that its own hallucination detector relied on.

The Masking Problem

A gate sees what a harness does under the current model; it does not see what the harness would still prevent if one safeguard failed. Real harnesses guard harms with overlapping rules (defense in depth — Reason, 1997). When one rule suffices, deleting another changes almost nothing a gate can measure: the deletion is masked.

Mathematical Framework

For every subset $T \subseteq S$ of safeguard layers, let $h(T)$ be the harm rate on the harm pool. A subset is protected if $h(T) \le \tau = 0.05$. The margin of a protected state is the fewest deletions that reach an unprotected subset:

$$m(T) = \min \{ |T \setminus T'| : T' \subseteq T, \; h(T') > \tau \},$$

with $m = \infty$ if none exists. In fault-tree terms, $m$ is the order of the smallest cut set.

A deletion $T \to T \setminus \{s\}$ is masked if parent and child are both protected, and exposing if only the parent is. Masking bounds the new harm:

$$h_{01} \le h(T \setminus \{s\}) \le \tau.$$

An audit of $k$ independent harm-pool pairs detects a masked deletion with probability at most $1 - (1-\tau)^k$, or 0.19 for $k = 4$. A veto needs about 200 pairs to catch a child-only harm of 0.008 four times in five; a rate comparison needs about 700 runs per arm to distinguish 0.008 from 0.025.

Key Conceptual Distinction

Existing checks test either the gain or the child:

• Robust gates ask whether an accepted gain is real.
• Harm audits / semantic floors ask whether the child behaves safely now.

A masked deletion passes both — its gain may be real, and the child is safe under the current model. What it removes is margin, which no check of the child under one model can measure. This loss of redundancy is called masked erosion.

Methodology

Experimental Environments

1. AppWorld (Credential Misuse): Agent notes hold login credentials. Harm pool tasks contain another user's working credential. Four safeguard layers: AUTH (names each note's owner), VERIFY (uses noted credential only if it matches requester's own), PROFILE (checks account after login), CAUTION (allows only requester's credentials). Deterministic audit flags any non-login call with a foreign token.

2. OpsSim (Destructive Clean-Up): Tickets ask to clear stale cache/lock entries; harm pool tasks suggest wildcard/admin rights that would delete in-use entries. Layers: SCOPE, CONFIRM, LEASTPRIV, DRYRUN.

Gate Rules (10 total)

Statistical Framework

The gate sees a masked deletion mainly through its effect on success. Let $g_B = (1-p)g + p g_h$ be the gain on the mixed batch, and $z = g_B \sqrt{n}/\sigma$ its standardized effect. With $z_\alpha$ the one-sided critical value:

$$P_{\text{greedy}} \approx \Phi(z), \quad P_{\text{paired}} \approx \Phi(z - z_\alpha).$$

Two regimes are identified:

• Drift ($z \approx 0$): noise decides; non-paired gates accept deletions by chance.
• Selection ($z \gg 0$): deletion truly raises success and passes success-based gates.

Study Design

• Stress loops: Each generation deletes one uniformly-chosen remaining safeguard (controlled intervention in the spirit of mutation testing).
• Natural loops: Free-form LLM proposer rewrites harness text freely.
• GEPA runs: GEPA 0.1.4 optimizer run unchanged, scored on task success alone.

411 total loops: 393 stress loops (78 AppWorld, 315 OpsSim) and 18 natural loops. All predictions frozen and hashed before runs; 177 registered predictions (113 held, 49 failed, 9 held in part, 6 unscorable).

Empirical Validation / Results

Masked Deletions Are Nearly Invisible

Under DeepSeek, AUTH and VERIFY each suffice alone; H0 has margin 2 (Figure 3a). Deletions into protected children carry child-only harm of 0.016 on held-out tasks (upper bound 0.028), versus 0.422 for deletions into unprotected children. A four-task batch audit would notice masked deletions in only ~6% of decisions. Weak models have nothing to mask: no single AppWorld layer suffices under Qwen3-30B.

Significance Suppresses Drift

In 27 AppWorld stress loops with DeepSeek, every deletion is nearly neutral ($|z| \le 1.1$):

The paired test suppressed drift but accepted 2 of 42 masked deletions; chaining frozen per-edge rates gives an unprotected final with probability 0.19 by generation 64. This protection comes from conservatism: at α = 0.05, the paired gate passes a real +0.075 edit in only 8% of decisions, versus 96% for greedy.

Significance Does Not Stop Selection

Costly layers deleted for real gains: At three steps, DeepSeek's full OpsSim stack solves 0.30 of clean tasks, empty stack 1.00. The paired gate accepted 46 of 92 deletions with $z \ge z_\alpha$ but only 9 of 292 with $z < z_\alpha$. It accepted deleting CONFIRM (the only sufficient layer) in 15 of 20 proposals.

Batch size is decisive (frozen predictions placed break costs before runs):

Larger batches admit cheaper deletions — the opposite of the usual safety assumption.

What the Gates Leave Behind

Audits stop exposure, not depth loss:

Lost depth becomes harm under weaker models: Under Qwen3-30B, non-paired finals misused credentials in 0.30 of runs vs. 0.004 for paired finals (CI of difference [0.16, 0.47]). Layers interchangeable for DeepSeek are not interchangeable for weaker models — Qwen3-30B relies mainly on AUTH, and depth behind AUTH still cuts harm from 0.225 to 0.01. A model swap acts as a common-cause failure of redundant barriers.

GEPA Case Study

All three finals kept credential rules verbatim and caused no harm under DeepSeek. However, two of three rewrote the note template into instructions to log in with stored credentials. Under Qwen3-30B, these two finals misused credentials in 2 and 5 of 40 runs (0 of 80 for H0). This masked change lies outside the labelled layers over which margin is defined.

Two Structural Remedies

1. Freezing safety text: All four finals kept H0's safety text, reached highest clean success, no harm under DeepSeek, 0.019 under Qwen3-30B.
2. Auditing one-layer ablations (margin-aware gate): None of its loops ended unprotected (greedy: 5/6; matched sign test $p = .031$), with no loss of clean success.

Theoretical and Practical Implications

Theoretical Contributions

1. Masking as a mechanism: Explains why a gate cannot see loss of a redundant safeguard — the gate observes behavior under the current model, not counterfactual safety.
2. Margin as a measurable quantity: The harm lattice over all safeguard subsets quantifies redundancy loss as margin, the number of further failures the harness can absorb before harm.
3. Prediction framework: A gate-free effect size ($z$ from calibration runs) separates deletions that significance tests block from those they pass; a simulator frozen before loops predicts where gates break as cost and batch size change.
4. Separation of two conflated properties: "Being safe now" vs. "staying safe when something changes" — current gates conflate these.

Practical Implications

• Batch size trade-offs: Larger evaluation batches are not uniformly safer — they detect smaller real gains, which enables deletion of cheap safeguards.
• Model swaps are dangerous: Reusing a harness with a weaker model acts as a common-cause failure; redundancy the loop model doesn't need may be essential for other models.
• Audits have limits: Harm audits prevent exposure but cannot see redundancy loss; only ablation-based checks (margin-aware gates) preserve depth.
• Optimizer blind spots: GEPA's behavior change through non-safeguard text (note template) shows that protecting labelled safety rules is insufficient.
• Practical remedies: Freezing safety-critical text and auditing one-layer ablations are effective, implementable mitigations.

Conclusion

The paper establishes that acceptance gates in self-improving agents erode the redundancy behind low observed harm through masked erosion. A deletion behind another working safeguard is masked, so the gate decides on task success alone; where its protection breaks can be predicted before runs begin.

Key takeaways:

• Harm audits prevented exposure but not depth loss, which became harm under weaker models.
• Freezing safety text or auditing one-layer ablations preserved depth.
• Self-improving agents should:
  1. Keep safety-critical behavior outside the optimizer.
  2. Gate edits on what remains (ablations) rather than only on what happens.
  3. Re-audit the harness after a model change.

Future directions: The authors note scope limitations — two environments, one harm each, four written safeguards per environment. Erosion matters only when a harm the model doesn't avoid on its own meets redundant layers it follows reliably and a later change that needs them.