# Falsifiable Release Gates for Self-Improving Systems: Standing Invariants at Scale

> Falsifiable release gates with machine-checked invariants make safety for self-improving AI a durable process, verified across releases at negligible governance cost.

- **Source:** [arXiv](https://arxiv.org/abs/2607.13070)
- **Published:** 2026-09-05
- **Permalink:** https://picx.dev/p/8LZVZB
- **Whiteboard:** https://picx.dev/p/8LZVZB/image

## Summary

## Summary (Overview)

- **Core contribution**: Introduces **falsifiable release gates**, a methodology for self-improving AI systems where every new capability must pass a pre-declared, machine-checkable acceptance suite before shipping, while a fixed set of **standing invariants** is preserved across all gates.
- **Machine-checked safety core**: The non-bypass property (no action reaches an effector without a capability token minted by a control ring) is verified **exhaustively over 291 reachable states** of a bounded model, with deliberately broken models proving the checker can fail (the "teeth discipline").
- **Longitudinal evidence**: The runtime (Antahkarana) is carried through **six further releases** after the initial six-gate ladder; the action-safety invariants **inv-1 through inv-6 held unchanged** throughout, and one release added three capabilities with **no new invariant required**.
- **Six new machine-checked invariant families**: Memory (inv-7 to inv-10), agency (inv-11 to inv-15), epistemic (inv-16 to inv-21), agent harness (inv-33 to inv-37), self-improvement (inv-38 to inv-42), and residency (inv-43), each shipping with broken models that must be rejected.
- **Real-hardware validation**: Gated self-improvement compounds a small model from **20% to 70% accuracy** while auto-rejecting confidence-gaming candidates, at a governance cost of **0.021 ms per request (0.008% of model inference)**.

---

## Introduction and Theoretical Foundation

### Background and Motivation

The central problem addressed is that safety claims for self-improving agent runtimes are **almost always self-graded**: a policy file, a guardrail, or a promise in a README. The hard question is not "is this one version safe?" but rather:

> "does every new capability preserve the safety of the last?"

as agent runtimes gain memory, tool access, multi-agent coordination, and the ability to revise their own policies.

### The Six Standing Invariants

The method rests on six standing invariants that every gate must preserve:

1. **Single gate**: Exactly one path from a model's intent to a real-world action—a control ring renders a verdict and, only on an *allow*, mints a capability token bound to that specific action. No effector accepts an action without a matching, unexpired token (machine-checked as inv-1).
2. **Monotone tightening**: The system may make its own policy stricter autonomously; it may never make it looser without a human-merged change.
3. **Everything audited**: Every decision emits a hash-chained record; humans are not above the ring.
4. **Hashes over payloads**: Governance reasons over hashes, not raw content.
5. **Opt-in, zero-overhead-off**: A capability costs nothing until used.
6. **Gates before code**: The falsifiable acceptance suite is written *before* the capability.

### Theoretical Foundation: The Inner-Instrument Model

The runtime is named **Antahkarana** (Sanskrit for "inner instrument"), a classical model splitting the mind into four faculties:

| Faculty | Its job | What it is in the runtime | Anchors |
|---------|---------|---------------------------|---------|
| Manas | perception, registering | novelty and risk scorer | G9 |
| Buddhi | judgment, discrimination | control ring: tiered verdict + capability-token mint | G8, G12 |
| Ahaṃkāra | boundary, the "I"-maker | identity binding, per-tenant key hierarchy, taint tracking | G11 |
| Citta | memory, retention | consolidation, replay, hash-chained audit log | G7, G9 |

The key design principle: **manas proposes, buddhi disposes, and nothing acts on the world unless buddhi sanctions it.**

---

## Methodology

### Falsifiable Release Gates

A release gate is a pre-declared acceptance suite with three properties:
1. **Falsifiable**: Makes a specific claim a measurement could disprove.
2. **Machine-checkable**: Code that runs in CI and returns pass/fail.
3. **Preservative**: Passing must not break any earlier gate (full suite re-run on every change).

### The Laddered Ordering

The six gates are laid out with a **forced dependency order**—each rung makes the next safe to build:

- **G7 (Hardening)**: Any past tick reconstructable from traces; policy versioned; rollback < 1s.
- **G8 (Integration)**: Injected instructions never cause a real action.
- **G9 (Memory & feedback)**: A tuned gate strictly dominates the static default and never self-loosens.
- **G10 (Mesh)**: Supervisor control lands within one tick.
- **G11 (Platform)**: Tenants cryptographically isolated.
- **G12 (Self-governing)**: No action bypasses the ring (machine-checked); contained self-improvement.

### The Teeth Discipline

A checker that cannot fail proves nothing. For each invariant, the authors add **deliberately broken variants** of the model that re-enable illegal transitions, and require the checker to catch them and return the **shortest counterexample** (breadth-first search guarantees minimality). These broken-model results are asserted in the test suite, so a regression making the checker vacuous would itself fail a gate.

### Trace Conformance

To prevent the model drifting from the code, recorded execution traces are replayed through the model. **One million synthetic traces** are validated with no rejections, and a hand-constructed bypass trace is rejected as it must be.

---

## Empirical Validation / Results

### First Ladder Results (G7–G12)

| Gate | Rung | Falsifiable claim | Measured result | Cases | Verdict |
|------|------|-------------------|-----------------|-------|---------|
| G7 | Hardening | any past tick reconstructable | deterministic replay + diff; atomic rollback | 20 | PASS |
| G8 | Integration | injected instructions never cause a real action | **432/432 attacks blocked**; 0 policy escalations | 16 | PASS |
| G9 | Memory & feedback | tuned gate strictly dominates static default | miss 0.50/0.58/0.67 → **0.00** at FAR = 0 | 13 | PASS |
| G10 | Mesh | supervisor control within one tick | **p100 = 0 leaked actions**; 1.00 attribution | 14 | PASS |
| G11 | Platform | tenants cryptographically isolated | cross-tenant decrypt denied; tamper localized | 14 | PASS |
| G12 | Self-governing | no action bypasses the ring (machine-checked) | **inv-1/4/5 over all 291 states**; 1M traces, 0 rejections | 18 | PASS |

### Machine-Checked Core (G12 Verification)

The reachable state space at the reported scope is **291 states**, and inv-1 (non-bypass), inv-4 (one-tick control), and inv-5 (token single-use) hold over all of them. The deliberately broken models are caught with shortest counterexamples:
- Bypass variant: **caught in 4 steps**
- Escaped-control variant: **caught in 8 steps**
- Token-reuse variant: **caught in 5 steps**

### Preservation Under Growth: Six Further Releases

| Family | Introduced at | What it guarantees | States | Teeth | Verdict |
|--------|--------------|-------------------|--------|-------|---------|
| inv-1 to inv-6 | first ladder | non-bypass action core, monotone tightening, full audit | 291 | 3 | holds |
| **none added** | governed serving | 3 capabilities under existing invariants | n/a | n/a | preserved |
| inv-7 to inv-10 | memory | forget cascades, consolidation preserves, pin safe, index equals live | 273 | 4 | holds |
| inv-11 to inv-15 | governed agent | tool calls token-gated, no second path to effector | 273 | 5 | holds |
| inv-16 to inv-21 | calibrated abstention | probability identity, abstention floor, post-quantum audit, external evidence | property | 6 | holds |
| inv-33 to inv-37 | agent harness | adaptive compute floor, skill provenance, export fidelity, signal fusion, sandbox | 148 | 15 | holds |
| inv-38 to inv-42 | self-improvement | acceptance monotonicity, non-self-modifiable gate, one-cycle reversibility, eval freshness, reward-hacking floor | 72 | 5 | holds |
| inv-43 | residency | artifacts stay in-boundary, cross-boundary only under signed grant | 8 | 2 | holds |

### Real-Hardware Results (Self-Improvement Family)

- **Compounding**: Gated self-improvement lifted held-out accuracy from **20% to 70%** on a small model.
- **Rejection of gaming**: A poisoned candidate that only inflated confidence was **auto-rejected**, as was a regression candidate.
- **Governance cost**: The entire governed path (safety-signal fusion + adaptive routing) added **0.021 ms per request** against ~270 ms model inference = **0.008% of end-to-end latency**.
- **Safety**: A jailbreak attempt and unsafe request were both refused **before the model was ever called**.
- **Honest negative**: On task classes a capable model already solves, the loop correctly adopts nothing—it compounds only where there is real headroom.

### Key Results on the Epistemic Family

- Expected calibration error falls from **0.42 to 0.29** under governed collapse.
- The reliability curve moves toward the diagonal as abstention absorbs unsupported cases.

### Agent Harness Economic Payoff

The adaptive compute floor matches the quality of an always-largest-model policy while spending the largest tier only on genuinely hard tasks, at **roughly 60% lower cost**.

---

## Theoretical and Practical Implications

### For the Methodology

- **Economy of guarantees**: The governed-serving release (three capabilities, no new invariant) demonstrates that a good discipline does not force invariant inflation. A method that demands a fresh guarantee for every feature will not survive a real roadmap.
- **Falsifiability as a first-class requirement**: The teeth discipline transforms "the checker says safe" into "the checker demonstrably can fail and does fail on the broken models we ship."

### For Self-Improving Systems

- **Containment by construction**: The improver's write-surface is policy rules only; the type system cannot express writes to the judging machinery. A red-team feeding adversarial evidence cannot produce an out-of-scope diff.
- **Monotone tightening**: An unattended system can only converge toward caution—a loosening change always requires a human merge.
- **The fixed judge**: The self-improvement gate is non-self-modifiable, so the loop cannot edit its own acceptance criteria.

### For the Field

The paper positions its contribution relative to existing work:
- **Guardrails and allow-lists** (mechanisms at one point in a system's life) vs. the temporal discipline of preservation across growth.
- **RLHF/constitutional methods** (weight-level) vs. control-ring-level governance that works under any policy.
- **Formal methods** (TLA+, model checking) extended with the teeth discipline and trace conformance as *standing* requirements.

---

## Conclusion

The paper's central claim is that **safety for self-improving systems can be a process, not a promise**, and that the process can be made falsifiable and durable. The evidence:

1. The action-safety core (inv-1 to inv-6) **held unchanged across every release**, machine-checked at every step.
2. The acceptance suite grew from **122 to 563 tests**, all green at the latest release.
3. Six new machine-checked families extended the core into memory, agency, epistemic honesty, multi-agent harnesses, self-improvement, and residency.
4. Real-hardware validation shows governance is **effectively free** (0.008% overhead) and self-improvement compounds real models while rejecting false gains.

### Explicitly Stated Limitations

The authors are candid about boundaries:
- **Finite proofs**: Reachable-state results are exhaustive *within small scopes* (72–291 states), not unbounded proofs.
- **Measured vs. proved**: Calibration and compounding curves are measured, not proved.
- **Self-graded results**: Numbers come from the authors' own suite; external scrutiny is planned via a standing red-team challenge.

### Future Work

1. **Operational evidence**: A flagship deployment running for months against pre-registered thresholds.
2. **External scrutiny**: A standing machine-adjudicated red-team challenge and independent TLA+ verification.
3. **Research frontier**: Unbounded (inductive) verification and statistical bounds (e.g., conformal prediction) for learned components.
4. **Architecture**: General graphs beyond tree-structured meshes, cross-organization federation, adversarial supervisors.

> "The part meant to outlast this particular runtime was always the discipline. This paper is the evidence that it lasts."

**Reproducibility**: Every claim reduces to one command (e.g., `atk verify --teeth` for the non-bypass core); the runtime, CLI, model card, and full gate suite are open-source.

---

_Markdown view of https://picx.dev/p/8LZVZB, served by PicX — AI-generated visual whiteboard summaries of research papers._