Summary (Overview)

• EnvHarness is a programmable layer that transforms static, hand-built environments into dynamically customizable ones without modifying the underlying environment logic, using three plug-in components: Stage, Contract, and Chain.
• EnvRigger is an automated loop that treats the target policy as a black box, diagnoses weaknesses from execution trajectories, and synthesizes/validates EnvHarness components targeting those flaws.
• Across five benchmarks in four domains (ALFWorld, WebArena, SWE-bench Verified, OfficeQA, SpreadsheetBench), EnvHarness achieves up to 9.0-point improvement on held-out instances with 9.8% fewer execution steps.
• EnvHarness provides a superior optimization signal for reinforcement learning (up to 6.5 points improvement) and enables continuous co-evolution of policy and environment with compounding gains.
• The framework is domain-agnostic, preserving original verifiers while outperforming domain-specific environment generation pipelines (GenEnv, VeriEnv, SWE-smith).

---

Introduction and Theoretical Foundation

LLM agents learn by interacting with environments, yet these environments are hand-built and static—they behave identically regardless of the agent's weaknesses or improvement level. This rigidity limits learning in two ways:

1. Lack of targeted signal: Environments fail to address specific agent weaknesses.
2. Premature saturation: Once an agent solves existing tasks, the environment has nothing more to teach.

Existing automated environment generation approaches suffer from two major limitations:

• Domain specificity: Pipelines built for one setting (web, programming, tool use) cannot transfer to others.
• Correctness issues: LLM-generated environments and verifiers require over-generation and heavy filtering, which is costly and unreliable.

Key Insight: The paper draws an analogy between agent harnesses (which wrap frozen LLMs with tools, memory, and skills to create capable agents) and environment harnesses (which wrap frozen environments with plug-in components to create customizable environments).

Table 1 | Analogy between Agent Harness and EnvHarness

Formal Definition: An environment is modeled as a tuple $E = (\mathcal{S}, \mathcal{A}, \mathcal{O}, T, R, s_0)$ where $\mathcal{S}$ is the state space, $\mathcal{A}$ the action space, $\mathcal{O}$ the observation space, $T: \mathcal{S} \times \mathcal{A} \to \mathcal{S}$ the transition function, $R$ the reward induced by the verifier, and $s_0$ the initial state. An EnvHarness component is an environment-agnostic transformation $w$:

$$E^{\prime} = w(E), \qquad E^{\prime} = (\mathcal{S}^{\prime}, \mathcal{A}^{\prime}, \mathcal{O}^{\prime}, T^{\prime}, R^{\prime}, s_0^{\prime}).\tag{1}$$

The transformation reshapes the environment strictly at the interface level, preserving the ground-truth evaluation logic.

---

Methodology

Three EnvHarness Components

1. Stage: Changing the Initial State

A Stage, $w_{\text{stage}, \delta}$, is specified by a sequence of state-manipulation actions $\delta = (a_1, \ldots, a_k)$ applied to the initial state $s_0$:

$$E^{\prime} = w_{\text{stage}, \delta}(E) = (\mathcal{S}, \mathcal{A}, \mathcal{O}, T, R, s_0^{\prime}), \quad \text{where } s_0^{\prime} = T\left(\dots T\left(T(s_0, a_1), a_2\right) \dots, a_k\right).\tag{2}$$

Example: On ALFWorld task "put a clean mug on the desk," a Stage hides the mug in a drawer, forcing the agent to search first.

2. Contract: Rewriting the Interaction

A Contract, $w_{\text{contract}, r}$, is specified by a triplet of transformation maps $r = (f_A, f_T, f_O)$, each defaulting to the identity:

$$E^{\prime} = w_{\text{contract}, r}(E) = \left(\mathcal{S}, \mathcal{A}^{\prime}, \mathcal{O}^{\prime}, T^{\prime}, R, s_0\right), \quad \text{where } \left(\mathcal{A}^{\prime}, \mathcal{O}^{\prime}, T^{\prime}\right) = \left(f_A(\mathcal{A}), f_O(\mathcal{O}), f_T(T)\right).\tag{3}$$

Examples: Enforce action preconditions, augment/mask observations, or attach structured feedback to steer learning.

3. Chain: Extending the Environment

A Chain, $w_{\text{chain}, \ell}$, is specified by a pair $\ell = (E_{\text{ext}}, g)$ where $E_{\text{ext}}$ is an additional environment and $g$ is a composition logic:

$$E^{\prime} = w_{\text{chain}, \ell}(E) = (\mathcal{S}^{\prime}, \mathcal{A}^{\prime}, \mathcal{O}^{\prime}, T^{\prime}, R^{\prime}, s_0^{\prime}), \quad \text{where } E^{\prime} = g(E, E_{\text{ext}}).\tag{4}$$

The new spaces are the union of base environments, and $R^{\prime}$ acts as the composite reward. Composition logic $g$ is unrestricted (concatenation, interleaving, branching).

Composition: Components share a standard interface and compose freely (though noncommutatively):

$$E^{\prime} = w_{\text{chain}, \ell}\left(w_{\text{contract}, r}\left(w_{\text{stage}, \delta}(E)\right)\right).\tag{5}$$

EnvRigger: Automating Customization

Given a base environment $E$ and target policy $\pi$, the objective is to generate a modified environment $E^{\prime}$ tailored to task $t$:

$$E^{\prime} = \mathcal{H}(E, t; \pi) = (w_k \circ w_{k-1} \circ \dots \circ w_1)(E),\tag{6}$$

EnvRigger operates through four stages:

1. Observe: Run policy $\pi$ on task $t$ to collect rollout trajectories (both successes and failures).
2. Diagnose: Analyze trajectories to identify root causes (repetitive loops, parsing failures, misread constraints) and determine customization direction (scaffold struggling policies; harden overly successful ones).
3. Write: Synthesize EnvHarness components targeting diagnosed flaws.
4. Validate: Wrap the environment with candidates, run fresh rollouts, and decide: accept, reject (unsolvable/non-challenging), or refine (poorly scaled signals). Accepted components are added to the EnvHarness.

---

Empirical Validation / Results

Experimental Setup

• Benchmarks: ALFWorld (embodied), WebArena (web), SWE-bench Verified (software engineering), OfficeQA + SpreadsheetBench (office automation).
• Models: Gemini-3.1-Flash-Lite (ALFWorld, WebArena), Gemini-3.5-Flash (elsewhere)—the same backbone is used for both EnvRigger and policy to avoid distillation effects.
• Baselines: No Skills, Original Envs, GenEnv (ALFWorld), VeriEnv (WebArena), SWE-smith (SWE-bench).
• Skill extraction: ReasoningBank pipeline; training/evaluation episodes strictly disjoint.

Main Results

Table 2 | Performance on ALFWorld and WebArena (mean over 3 runs, std as subscripts)

Table 3 | Performance on SWE-bench Verified, OfficeQA, and SpreadsheetBench

Key Findings

1. Consistent gains: EnvHarness outperforms original environments on every benchmark; static environments can actually degrade performance (e.g., SpreadsheetBench skills fall below no-skill baseline).
2. Domain-agnostic: One interface works across all domains, while specialized baselines are constrained to their benchmarks.
3. Efficiency gains: On SWE-bench, EnvHarness reduces average steps from 53.6 to 49.6, while original environments increase it to 55.0.

Reinforcement Learning Results

Table 4 | RL on ALFWorld and WebShop (GRPO with Qwen3-8B-base)

Long-Horizon Tasks (Chain Component)

Table 5 | Performance on long-horizon environments

Environment Scaling

Under an identical environment budget on SWE-bench Verified, EnvHarness climbs from 47.67 to 54.79 (a 7.12-point gain) and maintains an upward trajectory at 300 environments. In contrast, the same budget yields only 52.13 on original environments and 50.37 on generated ones. EnvHarness environments co-evolve with the policy, targeting its current capability boundary.

Cross-Model Generalization

EnvHarness skills outperform real environment skills across all four tested models (Gemini 3.1 Flash-Lite, Qwen3.6 27B, Gemini 3.5 Flash, Claude Sonnet 4.6), with gains of 2.7 to 3.7 absolute points, independent of underlying policy strength (skill-free success rates span 30.7–67.2).

On-Demand Environment Generation

EnvRigger can accept explicit user-defined constraints. Example: Given the weakness "The policy submits a patch without running the failing test, so the fix stays unverified," EnvRigger generates a Contract that rejects code submissions unless tests are run:

class _Contract(Contract):
    def modify_transition(self, action, response, env_state):
        cmd = bash_command(action)
        if "pytest" in cmd or "runtests.py" in cmd:
            env_state.extras["ran_tests"] = True
        if is_submission(cmd) \
            and not env_state.extras.get("ran_tests"):
            return failed(response,
                "githook: pre-commit hook 'verify-tests' "
                "failed. Run the test suite before submitting.")
        return response

This yields the distilled skill: "Verification-Driven Development Loop"—run the relevant test suite before finalizing any change to confirm the failure exists, then run it again after the patch to verify the fix.

---

Theoretical and Practical Implications

Theoretical Contributions

1. Reframing environment construction: EnvHarness reframes environment building as a wrapping problem rather than an authoring problem, making it possible to customize environments that were never designed for the target task.

2. Interface-level transformation: By operating strictly at the reset/step interface, EnvHarness provides a formal framework ($E^{\prime} = w(E)$) for environment customization that preserves verifier integrity—a key correctness guarantee absent in generated environments.

3. Task-policy-conditioned customization: The formulation $\mathcal{H}(E, t; \pi)$ makes explicit that environment design should be conditioned on both the task and the learner's diagnosed weaknesses, rather than being policy-agnostic.

Practical Implications

• Scalability: EnvHarness scales better than both human-built and generated environments—performance keeps improving with environment count where baselines flatten out.
• Cost efficiency: Requires no domain-specific pipelines; one implementation works across diverse domains.
• Safety: Preserving original verifiers ensures that reshaped environments maintain trusted evaluation logic.
• Continuous co-evolution: The EnvRigger loop enables ongoing policy-environment co-evolution, with each round targeting newly exposed weaknesses.

---

Conclusion

EnvHarness introduces a programmable layer that turns static environments into controllable ones through three plug-in components—Stage (initial state), Contract (interaction rules), and Chain (extended episodes)—operating entirely through standard reset/step interfaces. The accompanying EnvRigger automates customization by diagnosing policy weaknesses from trajectories and synthesizing targeted components validated via fresh rollouts.

Key takeaways:

• Consistent gains across five benchmarks in four domains (up to +9.0 points on held-out tasks, 9.8% fewer steps).
• Superior RL optimization signal and efficient environment scaling with compounding gains.
• Domain-agnostic design that outperforms specialized generation baselines while preserving original verifiers.
• Generalizes across model families and strengths, and accepts explicit user constraints.

Future directions (from Appendix I): Extending Chain to handle multi-environment internal state observation, exploring additional component types beyond the three introduced, and investigating the interaction between EnvHarness and other training paradigms beyond skill-based learning and GRPO-based RL.

Limitations (from Appendix H): The Chain component is excluded from the automated EnvRigger pipeline due to difficulty observing internal states of joined environments; deterministic reset assumption for Stage validation; reliance on prompt templates for domain adaptation.