# EnvHarness: Awakening Static Worlds for Agent Learning

> EnvHarness wraps static environments with plug-in components to dynamically customize training tasks, boosting agent performance by up to 9 points while using fewer steps across five benchmarks.

- **Source:** [arXiv](https://arxiv.org/abs/2608.19880)
- **Published:** 2026-08-22
- **Permalink:** https://picx.dev/p/Q5MumR
- **Whiteboard:** https://picx.dev/p/Q5MumR/image

## Summary

## Summary (Overview)

- **EnvHarness** is a programmable layer that transforms static, hand-built environments into dynamically customizable ones without modifying the underlying environment logic, using three plug-in components: **Stage**, **Contract**, and **Chain**.
- **EnvRigger** is an automated loop that treats the target policy as a black box, diagnoses weaknesses from execution trajectories, and synthesizes/validates EnvHarness components targeting those flaws.
- Across five benchmarks in four domains (ALFWorld, WebArena, SWE-bench Verified, OfficeQA, SpreadsheetBench), EnvHarness achieves up to **9.0-point improvement** on held-out instances with **9.8% fewer execution steps**.
- EnvHarness provides a superior optimization signal for reinforcement learning (up to **6.5 points improvement**) and enables continuous co-evolution of policy and environment with compounding gains.
- The framework is domain-agnostic, preserving original verifiers while outperforming domain-specific environment generation pipelines (GenEnv, VeriEnv, SWE-smith).

---

## Introduction and Theoretical Foundation

LLM agents learn by interacting with environments, yet these environments are **hand-built and static**—they behave identically regardless of the agent's weaknesses or improvement level. This rigidity limits learning in two ways:
1. **Lack of targeted signal**: Environments fail to address specific agent weaknesses.
2. **Premature saturation**: Once an agent solves existing tasks, the environment has nothing more to teach.

Existing automated environment generation approaches suffer from two major limitations:
- **Domain specificity**: Pipelines built for one setting (web, programming, tool use) cannot transfer to others.
- **Correctness issues**: LLM-generated environments and verifiers require over-generation and heavy filtering, which is costly and unreliable.

**Key Insight**: The paper draws an analogy between *agent harnesses* (which wrap frozen LLMs with tools, memory, and skills to create capable agents) and *environment harnesses* (which wrap frozen environments with plug-in components to create customizable environments).

> **Table 1 | Analogy between Agent Harness and EnvHarness**

| | Agent Harness | EnvHarness (Ours) |
|---|---|---|
| Base System | Frozen LLM | Static environment |
| Designed to Solve | Lack of action, memory, or loops | Hardcoded interaction logic |
| Harness Layer | Capabilities (tools, memory) | Customization (states, rules, observations) |
| Unified Output | Autonomous agent | Customized environment |

**Formal Definition**: An environment is modeled as a tuple $E = (\mathcal{S}, \mathcal{A}, \mathcal{O}, T, R, s_0)$ where $\mathcal{S}$ is the state space, $\mathcal{A}$ the action space, $\mathcal{O}$ the observation space, $T: \mathcal{S} \times \mathcal{A} \to \mathcal{S}$ the transition function, $R$ the reward induced by the verifier, and $s_0$ the initial state. An EnvHarness component is an environment-agnostic transformation $w$:

$$
E^{\prime} = w(E), \qquad E^{\prime} = (\mathcal{S}^{\prime}, \mathcal{A}^{\prime}, \mathcal{O}^{\prime}, T^{\prime}, R^{\prime}, s_0^{\prime}).\tag{1}
$$

The transformation reshapes the environment **strictly at the interface level**, preserving the ground-truth evaluation logic.

---

## Methodology

### Three EnvHarness Components

**1. Stage: Changing the Initial State**

A Stage, $w_{\text{stage}, \delta}$, is specified by a sequence of state-manipulation actions $\delta = (a_1, \ldots, a_k)$ applied to the initial state $s_0$:

$$
E^{\prime} = w_{\text{stage}, \delta}(E) = (\mathcal{S}, \mathcal{A}, \mathcal{O}, T, R, s_0^{\prime}), \quad \text{where } s_0^{\prime} = T\left(\dots T\left(T(s_0, a_1), a_2\right) \dots, a_k\right).\tag{2}
$$

*Example*: On ALFWorld task "put a clean mug on the desk," a Stage hides the mug in a drawer, forcing the agent to search first.

**2. Contract: Rewriting the Interaction**

A Contract, $w_{\text{contract}, r}$, is specified by a triplet of transformation maps $r = (f_A, f_T, f_O)$, each defaulting to the identity:

$$
E^{\prime} = w_{\text{contract}, r}(E) = \left(\mathcal{S}, \mathcal{A}^{\prime}, \mathcal{O}^{\prime}, T^{\prime}, R, s_0\right), \quad \text{where } \left(\mathcal{A}^{\prime}, \mathcal{O}^{\prime}, T^{\prime}\right) = \left(f_A(\mathcal{A}), f_O(\mathcal{O}), f_T(T)\right).\tag{3}
$$

*Examples*: Enforce action preconditions, augment/mask observations, or attach structured feedback to steer learning.

**3. Chain: Extending the Environment**

A Chain, $w_{\text{chain}, \ell}$, is specified by a pair $\ell = (E_{\text{ext}}, g)$ where $E_{\text{ext}}$ is an additional environment and $g$ is a composition logic:

$$
E^{\prime} = w_{\text{chain}, \ell}(E) = (\mathcal{S}^{\prime}, \mathcal{A}^{\prime}, \mathcal{O}^{\prime}, T^{\prime}, R^{\prime}, s_0^{\prime}), \quad \text{where } E^{\prime} = g(E, E_{\text{ext}}).\tag{4}
$$

The new spaces are the union of base environments, and $R^{\prime}$ acts as the composite reward. Composition logic $g$ is unrestricted (concatenation, interleaving, branching).

**Composition**: Components share a standard interface and compose freely (though noncommutatively):

$$
E^{\prime} = w_{\text{chain}, \ell}\left(w_{\text{contract}, r}\left(w_{\text{stage}, \delta}(E)\right)\right).\tag{5}
$$

### EnvRigger: Automating Customization

Given a base environment $E$ and target policy $\pi$, the objective is to generate a modified environment $E^{\prime}$ tailored to task $t$:

$$
E^{\prime} = \mathcal{H}(E, t; \pi) = (w_k \circ w_{k-1} \circ \dots \circ w_1)(E),\tag{6}
$$

EnvRigger operates through **four stages**:

1. **Observe**: Run policy $\pi$ on task $t$ to collect rollout trajectories (both successes and failures).
2. **Diagnose**: Analyze trajectories to identify root causes (repetitive loops, parsing failures, misread constraints) and determine customization direction (scaffold struggling policies; harden overly successful ones).
3. **Write**: Synthesize EnvHarness components targeting diagnosed flaws.
4. **Validate**: Wrap the environment with candidates, run fresh rollouts, and decide: **accept**, **reject** (unsolvable/non-challenging), or **refine** (poorly scaled signals). Accepted components are added to the EnvHarness.

---

## Empirical Validation / Results

### Experimental Setup

- **Benchmarks**: ALFWorld (embodied), WebArena (web), SWE-bench Verified (software engineering), OfficeQA + SpreadsheetBench (office automation).
- **Models**: Gemini-3.1-Flash-Lite (ALFWorld, WebArena), Gemini-3.5-Flash (elsewhere)—the same backbone is used for both EnvRigger and policy to avoid distillation effects.
- **Baselines**: No Skills, Original Envs, GenEnv (ALFWorld), VeriEnv (WebArena), SWE-smith (SWE-bench).
- **Skill extraction**: ReasoningBank pipeline; training/evaluation episodes strictly disjoint.

### Main Results

**Table 2 | Performance on ALFWorld and WebArena** (mean over 3 runs, std as subscripts)

| Skill Source | ALFWorld In-Dist | ALFWorld OOD | ALFWorld Avg. | WebArena Reddit | WebArena Shopping | WebArena Shop Admin | WebArena GitLab | WebArena Avg. |
|---|---|---|---|---|---|---|---|---|
| No Skills | 62.6 | 60.7 | 61.7 | 39.6 | 35.2 | 44.1 | 35.8 | 38.7 |
| Original Envs | 63.3 | 61.4 | 62.4 | 38.7 | 35.2 | 44.6 | 35.4 | 38.5 |
| GenEnv | 63.3 | 61.9 | 62.6 | – | – | – | – | – |
| VeriEnv | – | – | – | 39.6 | 30.2 | 49.7 | 38.9 | 39.6 |
| **EnvHarness Envs** | **66.2** | **70.4** | **68.3** | **40.6** | **37.4** | **50.8** | **37.7** | **41.6** |
| **Improvement** | **+2.9** | **+9.0** | **+5.9** | **+1.9** | **+2.2** | **+6.2** | **+2.3** | **+3.1** |

**Table 3 | Performance on SWE-bench Verified, OfficeQA, and SpreadsheetBench**

| Skill Source | SWE-verified SR (↑) | SWE-verified Avg Step (↓) | OfficeQA EM (↑) | OfficeQA F1 (↑) | SpreadsheetBench Pass@1 (↑) | SpreadsheetBench Mean Score (↑) |
|---|---|---|---|---|---|---|
| No Skills | 47.67 | 53.58 | 54.23 | 55.77 | 46.44 | 61.32 |
| Original Envs | 49.88 | 55.01 | 54.40 | 55.77 | 45.88 | 61.47 |
| SWE-smith | 50.12 | 54.72 | – | – | – | – |
| **EnvHarness Envs** | **52.58** | **49.61** | **56.20** | **57.73** | **49.15** | **62.48** |
| **Improvement** | **+2.70** | **+5.40** | **+1.80** | **+1.96** | **+3.27** | **+1.01** |

### Key Findings

1. **Consistent gains**: EnvHarness outperforms original environments on every benchmark; static environments can actually degrade performance (e.g., SpreadsheetBench skills fall below no-skill baseline).
2. **Domain-agnostic**: One interface works across all domains, while specialized baselines are constrained to their benchmarks.
3. **Efficiency gains**: On SWE-bench, EnvHarness reduces average steps from 53.6 to 49.6, while original environments increase it to 55.0.

### Reinforcement Learning Results

**Table 4 | RL on ALFWorld and WebShop (GRPO with Qwen3-8B-base)**

| Training Set | ALFWorld In-Dist | ALFWorld OOD | ALFWorld Avg. | WebShop Score | WebShop SR |
|---|---|---|---|---|---|
| Original Envs | 81.4 | 89.6 | 85.5 | 75.6 | 66.0 |
| **EnvHarness Envs** | **87.9** | 88.8 | **88.4** | **79.2** | **67.4** |

### Long-Horizon Tasks (Chain Component)

**Table 5 | Performance on long-horizon environments**

| | SR (%) ↑ | AS ↓ |
|---|---|---|
| No Skills | 47.67 | 53.58 |
| Original Envs | 49.88 | 55.01 |
| EnvHarness (Stage/Contract Only) | 52.58 | 49.61 |
| EnvHarness (Chain Only) | 49.63 | **41.96** |
| **Combined Skills (Stage/Contract + Chain)** | **54.30** | **43.12** |

### Environment Scaling

Under an identical environment budget on SWE-bench Verified, EnvHarness climbs from 47.67 to **54.79** (a 7.12-point gain) and maintains an upward trajectory at 300 environments. In contrast, the same budget yields only 52.13 on original environments and 50.37 on generated ones. EnvHarness environments co-evolve with the policy, targeting its current capability boundary.

### Cross-Model Generalization

EnvHarness skills outperform real environment skills across all four tested models (Gemini 3.1 Flash-Lite, Qwen3.6 27B, Gemini 3.5 Flash, Claude Sonnet 4.6), with gains of **2.7 to 3.7 absolute points**, independent of underlying policy strength (skill-free success rates span 30.7–67.2).

### On-Demand Environment Generation

EnvRigger can accept explicit user-defined constraints. Example: Given the weakness *"The policy submits a patch without running the failing test, so the fix stays unverified,"* EnvRigger generates a Contract that rejects code submissions unless tests are run:

```python
class _Contract(Contract):
    def modify_transition(self, action, response, env_state):
        cmd = bash_command(action)
        if "pytest" in cmd or "runtests.py" in cmd:
            env_state.extras["ran_tests"] = True
        if is_submission(cmd) \
            and not env_state.extras.get("ran_tests"):
            return failed(response,
                "githook: pre-commit hook 'verify-tests' "
                "failed. Run the test suite before submitting.")
        return response
```

This yields the distilled skill: **"Verification-Driven Development Loop"**—run the relevant test suite before finalizing any change to confirm the failure exists, then run it again after the patch to verify the fix.

---

## Theoretical and Practical Implications

### Theoretical Contributions

1. **Reframing environment construction**: EnvHarness reframes environment building as a *wrapping problem* rather than an *authoring problem*, making it possible to customize environments that were never designed for the target task.

2. **Interface-level transformation**: By operating strictly at the reset/step interface, EnvHarness provides a formal framework ($E^{\prime} = w(E)$) for environment customization that preserves verifier integrity—a key correctness guarantee absent in generated environments.

3. **Task-policy-conditioned customization**: The formulation $\mathcal{H}(E, t; \pi)$ makes explicit that environment design should be conditioned on both the task and the learner's diagnosed weaknesses, rather than being policy-agnostic.

### Practical Implications

- **Scalability**: EnvHarness scales better than both human-built and generated environments—performance keeps improving with environment count where baselines flatten out.
- **Cost efficiency**: Requires no domain-specific pipelines; one implementation works across diverse domains.
- **Safety**: Preserving original verifiers ensures that reshaped environments maintain trusted evaluation logic.
- **Continuous co-evolution**: The EnvRigger loop enables ongoing policy-environment co-evolution, with each round targeting newly exposed weaknesses.

---

## Conclusion

EnvHarness introduces a programmable layer that turns static environments into controllable ones through three plug-in components—**Stage** (initial state), **Contract** (interaction rules), and **Chain** (extended episodes)—operating entirely through standard reset/step interfaces. The accompanying **EnvRigger** automates customization by diagnosing policy weaknesses from trajectories and synthesizing targeted components validated via fresh rollouts.

Key takeaways:
- Consistent gains across five benchmarks in four domains (up to +9.0 points on held-out tasks, 9.8% fewer steps).
- Superior RL optimization signal and efficient environment scaling with compounding gains.
- Domain-agnostic design that outperforms specialized generation baselines while preserving original verifiers.
- Generalizes across model families and strengths, and accepts explicit user constraints.

**Future directions** (from Appendix I): Extending Chain to handle multi-environment internal state observation, exploring additional component types beyond the three introduced, and investigating the interaction between EnvHarness and other training paradigms beyond skill-based learning and GRPO-based RL.

**Limitations** (from Appendix H): The Chain component is excluded from the automated EnvRigger pipeline due to difficulty observing internal states of joined environments; deterministic reset assumption for Stage validation; reliance on prompt templates for domain adaptation.

---

_Markdown view of https://picx.dev/p/Q5MumR, served by PicX — AI-generated visual whiteboard summaries of research papers._