# Zetta ζ: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical Intelligence

> Zetta evolves code-based runtime critics and recovery skills around frozen VLA policies, achieving 90.8% success on LIBERO-Pro and 93.6% on RoboCasa without policy retraining.

- **Source:** [arXiv](https://arxiv.org/abs/2608.16590)
- **Published:** 2026-08-21
- **Permalink:** https://picx.dev/p/EA5MqG
- **Whiteboard:** https://picx.dev/p/EA5MqG/image

## Summary

# Zetta ζ: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical Intelligence

## Summary (Overview)

- **Zetta** introduces a closed-loop embodied agent harness that evolves code-based runtime critics and recovery skills online while keeping the base policy model (VLA/WAM) frozen, enabling sustained same-task improvement without additional policy training.
- The system achieves **state-of-the-art success rates** of **90.8% on LIBERO-Pro** and **93.6% on RoboCasa** under the current rollout budget, with an **11.1× inference speedup** compared to the RPent baseline.
- **Key contributions** include: (1) the first self-evolving embodied agent through learnable code-based critics and three coordinated evolution loops; (2) the first rollout infrastructure (Z-Infra) designed specifically for embodied self-evolution; (3) demonstrated zero-shot skill transfer across tasks and clear robotic "Aha Moments."
- **Performance scaling**: Success increases from 34.5% → 90.8% on LIBERO-Pro and from 73.6% → 93.6% on RoboCasa within several evolution iterations.
- **Zero-shot transfer**: Skills discovered on source tasks (e.g., PnP-Stove) improve success on related tasks (PnP-Sink: 58% → 82%, PnP-Cabinet: 62% → 80%, PnP-Toaster: 72% → 90%) without any target-specific optimization.

## Introduction and Theoretical Foundation

### Background and Motivation

Physical intelligence is advancing along two complementary paths:
1. **End-to-end policy models** (VLAs, WAMs) trained on large demonstration corpora—but these are brittle under real-world distribution shifts.
2. **LLM-based embodied agents** that orchestrate policy models, code, tools, and control primitives—but these remain *open-loop* during execution.

The fundamental bottleneck is that physical tasks require decisions at **millisecond-level frequency**, which large agentic models cannot achieve. Existing systems perform reflection only at episode level, which:
- Cannot govern execution as it unfolds
- Cannot test alternative actions to verify reflections
- Makes credit assignment over entire trajectories difficult
- Lacks access to precise failure states

### Theoretical Foundation: Dual-Agent Governance

The framework formalizes long-horizon robotic manipulation as a governance and evolution process with:

**Fixed Runtime Components**:
- **Action Policy (π)**: Generates low-level actions $a_t = \pi(s_t, g; \theta)$ with constraint $\nabla\theta = 0$
- **Orchestrator Agent** ($\mathcal{A}_{orch}$): Fixed multimodal reasoning operator for adjudication

**Evolvable Harness** (H):

$$\mathcal{H} = \{C, R, \mathcal{T}\} \tag{1}$$

- **Runtime Critic (C)**: High-frequency monitoring functions that generate structured proposals:
$$P_t = C(\tau_{0:t}) = \langle e_t, \hat{\sigma}_t \rangle \tag{2}$$
- **Recovery Playbook (R)**: Structured library of strategies mapped to failure mechanisms
- **Heterogeneous Toolset (T)**: Executable tools and operators that can be generated and refined

**Adjudication Logic** (online execution):
$$\sigma_t = \mathcal{A}_{orch}(P_t, R, \mathcal{T}, \mathcal{K}) \tag{3}$$

**Offline Optimization** (evolutionary agents):
$$\mathcal{H}^{(k+1)} \leftarrow \mathcal{A}_{evo}(\mathcal{D}_{fail}^{(k)}, \mathcal{H}^{(k)}) \tag{4}$$

**Optimization Objective**:
$$\max_{\mathcal{H}} J(\mathcal{H}) = \mathbb{E}_{g, s_0 \sim \mathcal{D}} [\text{Success}(\tau) \mid \pi, \mathcal{A}_{\text{orch}}, \mathcal{H}] \tag{5}$$

### Three Core Challenges

1. **Open-Loop Semantics-Physics Gap**: VLA/WAM models lack closed-loop sensory-motor perception → **Solution**: High-frequency state governance via critics
2. **Generalization Pitfall of Over-Parameterized Repair**: Overfitting repairs corrupt action distributions → **Solution**: Top-down hierarchical causal diagnosis with minimal intervention
3. **Scalability of Expert-in-the-Loop Debugging**: Manual debugging is unscalable → **Solution**: Fully automated evolutionary learning loop

## Methodology

### Three Coordinated Evolution Loops

**Loop 1: Critic-Governed Action Loop** — Executes learned critics at action frequency, triggers appropriate recovery skills when necessary

**Loop 2: Rollout-Batch Candidate Optimization Loop** — Clusters and diagnoses failures, proposes candidate critics and recoveries using an SGD-like optimization in code space (drawing on SkillOpt and EmbodiSkill)

**Loop 3: Validation-Gated Skill Update Loop** — Admits only candidates that improve success rate and generalize across rollouts into skill memory

### Phase I: Empirical Failure Profiling

- Large-scale sampling of the action policy across development seed set $\mathcal{D}_{dev}$
- **Deterministic Scheduling**: Centralized resource scheduler routes tasks with identical containers/hardware
- **Validity Determination**: Valid rollout set $\mathcal{V}$ includes only complete trajectories with full sensor/video evidence
- **Multi-Dimensional Evidence**: Each trajectory is a multimodal time series:
$$\boldsymbol{\tau}^{(j)} = \{(s_t, a_t, \mu_t, \phi_t)\}_{t=0}^T \tag{7}$$
  - $s_t$: state, $a_t$: action, $\mu_t$: milestone completion status, $\phi_t$: physio-auxiliary signals

### Phase II Stage 1: Failure Clustering & Causal Diagnosis

**First Missing Milestone** ($m^*$): The first milestone not observed in trajectory history:
$$m^* = \min\{m_k \in M \mid m_k \notin \{\mu_t\}_{t=0}^T\} \tag{9}$$

**Earliest Observable Divergence** ($t_{EOD}$): First time where state deviates from healthy distribution:
$$t_{EOD} = \min\{t \mid \text{dist}(s_t, s_t^{ref}) > \epsilon, s_t^{ref} \in I_{succ}(\mu_t)\} \tag{10}$$

**Top-Down Hierarchical Diagnosis**: Traverses layers in priority order: Evaluation → Critic → State → Planning/Control → Recovery → Parameter

### Phase II Stage 2: Harness Repair & Validation

**VLA Re-entry Contract**: Defines when control returns to base policy:
$$\Psi(s_t) = \mathbb{1}(\text{FailureCleared}) \wedge \mathbb{1}(\text{Stability}(s_t) > \gamma) \tag{12}$$

**Validation Success Criteria**:
$$\text{Success}(\mathcal{H}_{\text{patch}}) = \mathbb{1}(\mu_{T,\text{new}} = m_{\text{goal}} \wedge \forall t \in \text{Intv}, \text{Adjudicated by} \mathcal{A}_{\text{orch}}) \tag{13}$$

### Phase III: Harness Consolidation & Generalization

- **Historical Regression**: All failed seeds in cluster must succeed (100% success rate)
- **Held-out Evaluation**: Success rate increment over baseline:
$$\Delta SR = SR(\mathcal{D}_{\text{held-out}} \mid \mathcal{H}_{\text{merged}}) - SR(\mathcal{D}_{\text{held-out}} \mid \pi_{VLA}) \tag{16}$$
- **Dynamic Transition**: New failure mechanisms trigger reclassification of held-out seeds as development seeds

### Z-Infra: Three-Layer Rollout Infrastructure

1. **Control Plane**: Gateway with unified API, session registry, heartbeat monitoring, fault tolerance
2. **Environment Worker**: Manages simulation environments with session-based lifecycle, resource-sharing groups (ModelTemplate + fork), high-performance C++ controller
3. **Rollout Worker**: GPU-resident model serving with batched inference, dynamic scheduling, model partitioning (VLM/AE separate processes, 53% latency reduction), quantization runtime (W8A8 for prefix MLP, 1.18×–1.32× speedup)

## Empirical Validation / Results

### "Aha" Moments (Discontinuous Capability Gains)

| Task | v0 (baseline) | v1 (symptomatic) | v2 (root-cause) | Key Insight |
|------|---------------|-------------------|-----------------|--------------|
| LIBERO-Pro Goal-T2 | 10% | 15% | **95%** | Grasp retention before transport |
| LIBERO-Pro Goal-T8 | 5% | 10% | **60%** | Full contact–grasp–retention sequence |
| LIBERO-Pro Goal-S6 | 5% | 5% | **90%** | Pre-contact semantic approach |
| RoboCasa TurnOnElectricKettle | 88% | ~88% | **94%** | EEF re-alignment |
| RoboCasa SlideDishwasherRack | 76% | ~76% | **94%** | Centered contact restoration |
| RoboCasa CloseToasterOvenDoor | 82% | ~82% | **96%** | Contact geometry restoration |

### Scaling Results

**LIBERO-Pro**: Average success increases from 31.0% → 92.5% (Goal-T) and 38.0% → 89.0% (Goal-S) through cumulative Critic–Recovery mechanisms.

**RoboCasa**: Macro-average success increases from 73.56% → 93.56% across four cumulative global repair rounds.

### Final Benchmark Results

**RoboCasa Atomic-Seen Tasks** (18 tasks):

| Method | T1 | T2 | T3 | T4 | T5 | T6 | T7 | T8 | T9 | **Avg.** |
|--------|----|----|----|----|----|----|----|----|----|---------|
| Pure VLA (GR00T) | 78 | 74 | 78 | 58 | 48 | 62 | 72 | 74 | 70 | **73.56** |
| Zetta | 96 | 92 | 94 | 96 | 86 | 80 | 96 | 96 | 86 | **93.56** |

**LIBERO-Pro** (40 task-setting pairs):

| Setting | Method | Task 0–9 | **Average** |
|---------|--------|----------|-------------|
| Goal (T) | $\pi_{0.5}$ | 0,95,10,0,100,0,20,80,5,0 | **31.0** |
| Goal (T) | Zetta | 80,100,95,80,100,100,95,95,80,100 | **92.5** |
| Goal (S) | $\pi_{0.5}$ | 0,60,0,45,0,0,0,100,100,75 | **38.0** |
| Goal (S) | Zetta | 90,65,80,85,95,95,100,100,100,80 | **89.0** |
| LIBERO-10 (T) | $\pi_{0.5}$ | 5,95,95,0,0,80,85,75,65,0 | **50.0** |
| LIBERO-10 (T) | Zetta | 35,95,100,0,25,100,95,80,100,0 | **63.0** |
| LIBERO-10 (S) | $\pi_{0.5}$ | 0,35,0,0,5,50,0,0,0,0 | **9.0** |
| LIBERO-10 (S) | Zetta | 90,50,95,75,15,65,5,0,0,5 | **40.0** |

### Z-Infra Performance

- **Throughput**: 35.1 episodes/min at concurrency 64 (vs. 2.88 ep/min for without Z-Infra, 1.72 ep/min for RPent)
- **Latency**: 95s per episode at concurrency 64 vs. 513s for RPent (11.9× reduction)
- **Inference**: 11.1× speedup, 53% latency reduction from model partitioning, 1.18×–1.32× from quantization

## Theoretical and Practical Implications

### Theoretical Significance

1. **Closed-loop governance paradigm**: The work demonstrates that frozen VLA/WAM policies can be made reliable through evolvable code-based critics, challenging the assumption that policy improvement requires weight updates.

2. **Scaling unit insight**: The "scalable unit of physical intelligence" is identified as **state variables** (grasp stability, approach geometry) rather than task-specific trajectories — offering a new theoretical framework for understanding what constitutes reusable knowledge in embodied AI.

3. **Three-timescale evolution**: The separation of action-level (milliseconds), rollout-batch-level (seconds/minutes), and iteration-level (hours) loops provides a principled decomposition for self-evolving agent systems.

### Practical Implications

1. **Deployment-time self-improvement**: Harness self-evolution enables robots to improve through experience in real-world deployment without requiring model retraining, which is significant for applications where data collection is expensive or safety-critical.

2. **Infrastructure as enabler**: Z-Infra demonstrates that rollout throughput is the rate-limiting factor for intelligence scaling — faster rollouts directly accelerate the evolution loop, making infrastructure design a primary research problem.

3. **Zero-shot transfer**: Skills learned from a single task can be transferred to related tasks, suggesting that harness-based skill acquisition could replace task-specific training for common manipulation primitives.

## Conclusion

### Main Takeaways

Zetta establishes a new paradigm for physical intelligence scaling: instead of scaling end-to-end policy models, it scales the **governance harness** around frozen policies. By evolving code-based runtime critics and recovery skills through three coordinated loops, the system achieves:
- **Sustained same-task improvement** with frozen policy weights
- **Zero-shot capability transfer** across tasks
- **Discontinuous "Aha" moments** when the true physical bottleneck is identified
- **Accelerated execution** through efficient rollout infrastructure

### Future Directions

The paper identifies clear next steps:
1. **Real robot deployment**: Extending Zetta and Z-Infra to real robots by bridging the sim-to-real gap
2. **First-class real-robot environments**: Integrating real-robot environments as first-class workers alongside simulated ones in Z-Infra
3. **Massively parallel rollout collection**: Enabling fast, parallel rollout collection for self-evolution directly with real machines

The work establishes that **harness self-evolution is a viable path toward reliable embodied intelligence**, complementing the end-to-end policy scaling approach with a more sample-efficient, safer alternative that leverages existing frozen models while continuously improving execution reliability.

---

_Markdown view of https://picx.dev/p/EA5MqG, served by PicX — AI-generated visual whiteboard summaries of research papers._