# Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application

> This paper presents the first systematic survey of agentic environment engineering for LLMs, defining a taxonomy and synthesis paradigms to guide agent-environment co-evolution.

- **Source:** [arXiv](https://arxiv.org/abs/2606.12191)
- **Published:** 2026-06-12
- **Permalink:** https://picx.dev/p/N6hho4

## Summary

## Summary (Overview)

- This paper presents the first systematic survey on **Agentic Environment Engineering** for Large Language Models (LLMs), covering the full lifecycle of environment modeling, synthesis, evaluation, and application.
- The authors define agentic environments as dynamic, interactive systems designed for evaluating and training LLM agents, formalized as Partially Observable Markov Decision Processes (POMDPs).
- The paper introduces a comprehensive taxonomy of environments across **8 attributes** (symbolic vs. neural, open-loop vs. closed-loop, online vs. offline, MDP vs. POMDP, deterministic vs. nondeterministic, discrete vs. continuous, unimodal vs. multimodal, single-agent vs. multi-agent) and **8 domains** (GUI, Deep Research, Embodied, Game, Tool, Code, Domain-Specific, Cross-Domain).
- Two main paradigms for automated environment synthesis are identified: **Symbolic Synthesis** (code-driven, rule-based) and **Neural Synthesis** (world model-based, parameterized by neural networks).
- The paper characterizes agent-environment co-evolution through **4 agent evolution pathways** (memory-centric, orchestration-centric, trajectory-centric offline, exploration-centric online) and **3 environment evolution paradigms** (neural-driven, difficulty-driven, scaling-driven).

## Introduction and Theoretical Foundation

The paper addresses a critical gap in LLM agent research: while environments serve as the interactive systems that drive agent capability evolution, existing work lacks systematic categorization and deep analysis of agentic environments.

**Key Motivation:**
- Real-world interactions for training agents are often infeasible due to high costs, safety risks, and data privacy concerns
- Real-world environments are irreproducible, making operational failures exceptionally costly
- Manually engineering simulated environments is resource-intensive and suffers from constrained scenario coverage
- Agentic environments address these challenges by incorporating sophisticated tools and reward signals that closely approximate real-world conditions

**Theoretical Foundation:**
The paper formally defines the **Environment** $E$ as a Partially Observable Markov Decision Process (POMDP), represented by the tuple $\langle S, A, P, R, \Omega, O, \gamma \rangle$:

- **State Space** $S$: The set of all possible latent states of the environment
- **Action Space** $A$: The set of all possible actions available to the agent
- **Transition Function** $P$: $S \times A \rightarrow \Delta(S)$, governing the environment's physical dynamics
- **Reward Function** $R$: $S \times A \rightarrow \mathbb{R}$, providing scalar feedback $r_t$
- **Observation Space** $\Omega$ and **Observation Function** $O$: $S \times A \rightarrow \Delta(\Omega)$, the agent's perception of the latent state
- **Discount Factor** $\gamma$: A constant $\in [0,1)$

The **Agent** is defined as an autonomous decision-making entity with policy $\pi: H \rightarrow \Delta(A)$, where $H$ is the history space. The agent's objective is to maximize the Expected Discounted Return:

$$J(\pi) = \mathbb{E}_{\pi, P, O}\left[\sum_{k=0}^{\infty} \gamma^k r_{t+k}\right]$$

**Key Shift from Data Engineering to Environment Engineering:**
1. **From Passive Learning to Collaborative Evolution**: Environments dynamically adjust task complexity based on agent performance, unlike static datasets
2. **From Single-turn Q&A to Multi-turn Interaction**: Environments support multi-turn engagement and external tool integration
3. **From Open-loop System to Closed-loop System**: Environments establish continuous coupling between agent actions and resulting state changes

## Methodology

The paper organizes its analysis around three research questions (RQ1-RQ3) and employs a systematic literature review methodology.

### Environment Attribute Analysis (RQ1)

The paper defines **8 attribute pairs** for characterizing environments:

| Attribute | Description |
|-----------|-------------|
| Symbolic vs. Neural | Whether transition dynamics use explicit code/programmed logic or neural network parameters |
| Open-Loop vs. Closed-Loop | Whether agent uses only initial observation or continuously adapts based on feedback |
| Online vs. Offline | Whether agent actively interacts with the environment or uses static trajectories |
| MDP vs. POMDP | Whether environment is fully observable (state = observation) or partially observable |
| Deterministic vs. Nondeterministic | Whether actions produce predictable outcomes or involve randomness |
| Discrete vs. Continuous | Whether action space is finite/countable or spans real-valued vectors |
| Unimodal vs. Multimodal | Whether observation space uses single modality (text/image) or multiple modalities |
| Single-Agent vs. Multi-Agent | Whether one or multiple agents interact with the environment |

### Environment Domain Classification

The paper categorizes environments into **8 domains**:

1. **GUI**: Desktop GUI (OSWorld, WindowsAgentArena), Mobile GUI (AitW, AndroidWorld), Web GUI (WebShop, WebArena, VisualWebArena)
2. **Deep Research**: Information Search (SimpleQA, WideSearch), Multi-Source Reasoning (GAIA, BrowseComp), Research Report Writing (DeepResearch Bench, DR.BENCH)
3. **Embodied**: Spatial Navigation (Habitat, Room-to-Room), Physical Manipulation (RLBench, Robocasa), Long-Horizon Planning (ALFRED, ALFWorld)
4. **Game**: Open World Games (MineDojo), Puzzle Reasoning Games (Baba Is AI), Social Deduction Games (AvalonBench, Werewolf), Adventure Quest Games (BALROG, GameArena), Strategy Management Games (CivRealm, Factorio Learning Environment)
5. **Tool**: Conventional Tool Use (API-Bank, ToolBench), User-Simulated Tool Use ($\tau$-bench), MCP-based Tool Use (MCPVerse, MCP-Bench)
6. **Code**: Code Generation, Code Understanding, Code Verification, Code Debugging (SWE-Bench, InterCode)
7. **Domain-Specific**: Biomedical/Healthcare, Science/Technology, Finance/Investment
8. **Cross-Domain**: OpenAI Gym, AgentBench, GEM

### Environment Synthesis Methods (RQ2)

**Symbolic Synthesis** (code-driven environments):
- **Task-Driven Synthesis**: Environments built from predefined task specifications (e.g., SWE-Gym, Text2World, WorldCoder)
- **Real-World-Driven Synthesis**: Environments constructed from real-world data sources (e.g., AgentSynth, EnvGen, OSWorld-MCP)
- **De Novo Synthesis**: Environments generated from scratch without predefined templates (e.g., AutoEnv, LOGIGEN, InfiniteWeb)

**Neural Synthesis** (model-driven environments):
- **Pixel-Level Modeling**: Neural networks predict pixel-level state transitions (e.g., EVA, DIAMOND, NeuralOS, Cosmos)
- **Word-Level Modeling**: LLMs simulate environment dynamics through text generation (e.g., RAP, WebDreamer, CWM, SWE-World)
- **Latent-Level Modeling**: Neural networks operate in learned latent spaces (e.g., V-JEPA 2, I-JEPA, DINO-world, DINO-Foresight)

### Environment Evaluation

The paper discusses quality control across four dimensions:
- **Correctness**: Well-established with existing evaluation frameworks
- **Diversity**: Under-researched but crucial for broad agent training
- **Complexity**: Under-researched but important for progressive learning
- **Fidelity**: Under-researched but essential for sim-to-real transfer

## Empirical Validation / Results

The paper provides extensive tables cataloging environments across domains. Key findings include:

**Table 1: GUI and Deep Research Environments** (partial excerpt)

| Domain | Name | Size | Modality | Observability | Multi-agent | Continuity | Online |
|--------|------|------|----------|---------------|-------------|------------|--------|
| GUI | WebShop [15] | 500 | T I | Partially | ✗ | Discrete | ✓ |
| GUI | WebArena [60] | 812 | T I | Partially | ✗ | Discrete | ✓ |
| GUI | OSWorld [28] | 369 | T I | Partially | ✗ | Mixed | ✓ |
| Deep Research | GAIA [30] | 466 | T | Partially | ✗ | Discrete | ✓ |
| Deep Research | BrowseComp [31] | 1,266 | T | Partially | ✗ | Discrete | ✓ |

**Table 2: Embodied and Game Environments** (partial excerpt)

| Domain | Name | Size | Modality | Observability | Multi-agent | Continuity | Online |
|--------|------|------|----------|---------------|-------------|------------|--------|
| Embodied | ALFRED [32] | 1,529 | T I | Partially | ✗ | Discrete | ✗ |
| Embodied | Habitat [79] | – | I | Partially | ✗ | Discrete | ✓ |
| Game | MineDojo [91] | 3,142 | T I | Partially | ✗ | Discrete | ✓ |
| Game | AvalonBench [100] | 1,140 | T | Partially | ✓ | Discrete | ✓ |

**Key Observations:**
- Most existing environments are **single-agent** (multi-agent environments are underrepresented)
- The majority operate in **partially observable** settings
- **Discrete action spaces** dominate, with fewer continuous action space environments
- **Online interaction** is the predominant paradigm
- **Text-only** environments are common, but multimodal environments (text+image) are growing

### Agent Evolution Pathways

The paper identifies **4 agent evolution categories**:

1. **Memory-Centric Experience Evolution**: Accumulating and leveraging past interaction experiences
   - Instance Trajectory Experience (e.g., OpenAgent, CoPS)
   - Abstract Scripts Experience (e.g., Reasoning Bank, Agent-Pro)
   - Structured Skill Experience (e.g., SAGE, SkillWeaver, SkillRL)

2. **Orchestration-Centric Workflow Evolution**: Dynamic coordination of task sequences
   - Fixed Workflow (e.g., MetaGPT, SWE-agent)
   - Automated Workflow (e.g., HuggingGPT, AutoFlow)
   - Evolving Workflow (e.g., AFlow, DSPy, GPTSwarm)

3. **Trajectory-Centric Offline Evolution**: Generating high-quality trajectories for supervised fine-tuning
   - Task Synthesis (e.g., OS-Genesis, Insta)
   - Trajectory Synthesis (e.g., ToolAlpaca, Aguvis)
   - Trajectory Refinement (e.g., Toolformer, ETO, Self-Improvement)

4. **Exploration-Centric Online Evolution**: Using reinforcement learning for dynamic adaptation
   - Reasoning Structure (e.g., DeepRetrieval, Search-R1)
   - Reward Shaping (e.g., Agent-R1, ToolRL, GDPO)
   - Algorithmic Optimization (e.g., RAGEN, GiGPO, ARPO)

### Environment Evolution Paradigms

1. **Neural-Driven Evolution**: Adjusting environment parameters via neural networks
   - Self-Play (e.g., Absolute Zero, Self-Challenging, Active Zero)
   - World Model (e.g., WebDreamer, UI-Simulator, Code2World)

2. **Difficulty-Driven Evolution**: Adapting task complexity to match agent capabilities
   - Explicit Curriculum Signals (e.g., POET, AgentGen, SEC)
   - Implicit Curriculum Mechanisms (e.g., PAIRED, DCD, ACCEL, EnvGen)

3. **Scaling-Driven Evolution**: Expanding environment diversity and scope
   - Scenario-Level Scaling (e.g., FTRL, AgentScaler, AutoForge)
   - Environment-Level Scaling (e.g., ARE, AutoEnv)

## Theoretical and Practical Implications

**Theoretical Implications:**
- The paper establishes a formal POMDP-based framework for agentic environments, providing a unified mathematical foundation for future research
- The distinction between data engineering and environment engineering highlights a fundamental paradigm shift in how LLM agents should be trained and evaluated
- The identification of environment attributes (8 pairs) provides a systematic framework for analyzing and comparing environments
- The concept of **Environment-Agent Alignment** formalizes the optimization objective of maximizing expected returns through environment interaction

**Practical Implications:**
- **Environment-as-a-Service (EaaS)**: The paper proposes a paradigm for standardized, scalable, and reproducible environment deployment
- **Multi-Agent Environments**: Future designs should incorporate cooperative and competitive multi-agent settings to realize collective intelligence
- **Neural-Symbolic Environments**: Current agents face a deployment bottleneck between rigid symbolic systems and uncontrollable neural models; future infrastructures must bridge this gap
- **Sim-to-Real Alignment**: Environments must better approximate real-world conditions for reliable agent training
- **Agent-Environment Co-Evolution**: The closed-loop interaction between agent and environment evolution is crucial for continuous improvement

**Key Theoretical Contributions:**
- The paper introduces **Agentic Reinforcement Learning** algorithms (PPO, GRPO, DAPO) as foundational methods for environment-based agent training
- The **GRPO** objective eliminates the need for an explicit value network:
  $$\mathcal{L}_{GRPO}(\theta) = \mathbb{E}_q\left[\frac{1}{G}\sum_{i=1}^G \frac{1}{|y_i|}\sum_{t=1}^{|y_i|}\left(\mathcal{L}_{i,t}^{CLIP}(\theta) - \beta D_{KL}(\pi_\theta \|\pi_{ref})\right)\right]$$
- The **DAPO** objective introduces decoupled clipping for more stable optimization:
  $$\mathcal{L}_{i,t}^{D-CLIP}(\theta) = \min\left(r_{i,t}(\theta)\hat{A}_i, \text{clip}(r_{i,t}(\theta), 1-\epsilon_{low}, 1+\epsilon_{high})\hat{A}_i\right)$$

## Conclusion

**Main Takeaways:**
1. Agentic environments are essential for evaluating and training LLM agents, providing interactive systems that enable multi-turn engagement, tool integration, and closed-loop feedback
2. The paper provides a comprehensive taxonomy covering 8 environment attributes and 8 domains, offering a systematic framework for analyzing and comparing environments
3. Two main synthesis paradigms exist: symbolic (code-driven, reliable but limited scalability) and neural (model-driven, scalable but less controllable)
4. Agent-environment co-evolution follows four agent evolution pathways and three environment evolution paradigms
5. Environment quality evaluation across correctness, diversity, complexity, and fidelity remains an under-researched area

**Future Directions:**
1. **Environment-as-a-Service**: Standardized, scalable, and reproducible environment deployment
2. **Multi-Agent Environments**: Cooperative and competitive settings for collective intelligence
3. **Neural-Symbolic Environments**: Combining the reliability of symbolic systems with the scalability of neural models
4. **Dynamic, Long-Horizon, Open-Ended Environments**: Requiring fundamentally new agent capabilities
5. **Sim-to-Real Alignment**: Ensuring environments accurately approximate real-world conditions
6. **Environment Scaling Laws**: Establishing scientific foundations for understanding how environment size and complexity relate to agent capability development
7. **Environment-Capability Relationships**: Understanding how different environment properties affect the development of specific agent capabilities

**Critical Open Challenges:**
- Evaluating diversity, complexity, and fidelity of environments remains under-researched
- Most existing frameworks are limited to single-agent scenarios
- Current agents face a deployment bottleneck between rigid symbolic frameworks and fully generative world models
- The field lacks systematic understanding of how environment properties scale with agent capabilities

---

_Markdown view of https://picx.dev/p/N6hho4, served by PicX — AI-generated visual whiteboard summaries of research papers._