Summary (Overview)

• The paper challenges the assumption that simply scaling the number of multimodal environments improves agent training, showing that performance can fluctuate or even degrade as environment types increase.
• Through controlled experiments, the authors demonstrate that multimodal environments are more prone to negative transfer than their text-symbolic counterparts, with a 10.7% performance drop under mixed training versus only 1.3% for text versions.
• The authors propose Ability-aware Environment Selection (AES), which selects diverse, low-redundancy, low-conflict environments based on meta-ability profiles derived from agent trajectories, achieving full core-ability coverage with only 30 out of 200 environments.
• They introduce Hierarchical Difficulty Curriculum (HDC), a two-level curriculum combining harness weakening (gradually removing textual scaffolds) and state-scale progression (increasing instance complexity), targeting multimodal-specific bottlenecks: visual state extraction and world modeling.
• Combined, AES + HDC achieve a 143.2% average relative gain across different settings, outperforming both full-environment training and random selection by substantial margins.

---

Introduction and Theoretical Foundation

Background and Motivation

Recent advances in multimodal large language models (MLLMs) and agents have shifted model training from static datasets to dynamic environments. A growing body of work constructs large-scale multimodal environment pools and trains agents jointly on them. However, existing efforts primarily emphasize sample-level quality—verifying that each environment is executable and provides reliable rewards—while neglecting the effectiveness of the training environment distribution itself.

Key Preliminary Findings

The authors build a pool of 200 multimodal environments and conduct three sets of controlled experiments:

1. Naive scaling is not always beneficial (Section 2.1): As the number of environment types increases under a fixed compute budget, model performance does not monotonically improve and may even decrease, indicating redundancy or conflicts.

2. Multimodal environments exhibit stronger negative transfer (Section 2.2): For the same environment set, the multimodal version shows a 10.7% drop under mixed training versus 1.3% for the text-symbolic version. Gradient cosine similarity analysis reveals more polarized similarities, including strongly negative correlations in the multimodal case (Figure 4).

3. Multimodal-specific failure modes (Section 2.3): Error analysis of 200 failure trajectories identifies two dominant bottlenecks:

   • Visual state extraction: models fail to reliably extract environment states from visual observations.
   • World modeling: models fail to correctly understand environment rules and action effects.

Theoretical Framework

The paper argues that environment distribution effectiveness should be evaluated along two core dimensions:

• Diversity: The environment set should cover broad abilities while avoiding redundancy and optimization conflicts.
• Difficulty structure: The distribution should provide a meaningful difficulty progression for stable, continuous improvement.

This framing moves beyond simple environment scaling to a more principled design of training distributions.

---

Methodology

1. Diversity: Ability-aware Environment Selection (AES)

Meta-Ability Profile Construction

For each environment, the authors:

• Collect 40 agent trajectories (20 from Qwen3-VL-4B, 20 from Gemini-3-Flash) to capture both successful strategies and failure patterns.
• Use GPT-5 to decompose trajectories into atomic abilities (e.g., "identifying the target position").
• Merge semantically equivalent abilities and build a meta-ability profile $P_e$ for each environment, containing meta-abilities, transition edges, and frequencies.
• Divide abilities into core (stable, task-critical) and soft (occasional, weakly related) categories, yielding 72 core meta-abilities across all 200 environments.

AES Selection Algorithm

For each candidate environment $e$, define:

New Coverage:

$$\operatorname{NewCoverage} (e, S) = \sum_ {a \in C (e) \backslash C (S)} w _ {a},$$

where $w_a$ is the weight of ability $a$, and $C(S)$ is the union of abilities covered by selected set $S$.

Redundancy:

$$\mathrm{Redundancy} (e, S) = \max _ {e ^ {\prime} \in S} \mathrm{sim} (P _ {e}, P _ {e ^ {\prime}}),$$

where $\mathrm{sim}(\cdot, \cdot)$ is the weighted profile similarity.

Conflict:

$$\operatorname{Conflict} (e, S) = \max _ {e ^ {\prime} \in S} \max (0, - \cos (g _ {e}, g _ {e ^ {\prime}})),$$

where $g_e$ and $g_{e'}$ are training gradients.

Selection Gain:

$$\begin{array}{r l} \mathrm{Gain} (e \mid S) = & \lambda_ {1} \mathrm{NewCoverage} (e, S) \\ & - \lambda_ {2} \mathrm{Redundancy} (e, S) \\ & - \lambda_ {3} \mathrm{Conflict} (e, S). \end{array}$$

The algorithm greedily selects the highest-gain environment until all core abilities are covered, yielding a subset of 30 environments.

---

2. Difficulty: Hierarchical Difficulty Curriculum (HDC)

Harness Weakening (Outer Curriculum)

Four types of textual harnesses are designed to scaffold agent learning:

The harness weakening axis has 5 difficulty levels $H_0$ to $H_4$: $H_0$ provides full harnesses, while $H_4$ retains only raw visual observations and task descriptions. Each environment maintains a current harness frontier $r_e \in \{0,1,2,3,4\}$, and sampling follows:

$$h \sim D _ {e} (h \mid r _ {e}),$$

with earlier levels sampled with exponential decay:

$$w _ {h} = \exp \big (- \alpha (r _ {e} - h) \big), \quad D _ {e} (h \mid r _ {e}) = (1 - p _ {\mathrm{cur}}) \frac {w _ {h}}{\sum_ {j < r _ {e}} w _ {j}}, \quad h < r _ {e}.$$

State-Scale Progression (Inner Curriculum)

Each environment has state-scale difficulty levels $s \in \{0, 1, \ldots, S_e\}$ (e.g., grid size). Sampling uses a sliding window:

$$s \sim \operatorname{Uniform} \bigl (\{\ell_ {e}, \ell_ {e} + 1, \dots , u _ {e} \} \bigr), \quad \ell _ {e} = \operatorname* { m a x } ( 0 , u _ {e} - \Delta d ).$$

Hierarchical Update

• Advance inner curriculum ($u_e \leftarrow u_e + 1$) when scale threshold $\tau_s$ is reached.
• Advance outer curriculum ($r_e \leftarrow r_e + 1$) when $u_e$ reaches the target level and harness threshold $\tau_h$ is satisfied, then reset state-scale.

Each environment maintains its own curriculum state tuple $(r_e, \ell_e, u_e)$, allowing independent progress.

---

Empirical Validation / Results

Main Results

The table below presents the main results with $K=30$ environments:

ST = single-turn success rate, MT = normalized return in multi-turn environments, Rel. = relative gain over base model.

Key findings:

• AES alone achieves a 95.6% average relative gain versus 43.4% for All Envs. and 44.6% for Random-K.
• AES + HDC achieves a 143.2% average relative gain across ID/OOD groups and both model scales.
• AES-selected environments generalize across model scales (effective for both 4B and 8B models).

Ablation on AES

Removing conflict control drastically reduces relative gain from 40.3% to 2.8%, highlighting the critical importance of gradient-based conflict mitigation.

Ablation on HDC

Both curriculum axes contribute, with harness weakening providing a larger gain than state-scale progression alone, and their combination being optimal.

Harness Effectiveness

Text State is the most effective harness, while all harnesses significantly outperform no-harness training.

---

Theoretical and Practical Implications

Theoretical Implications

1. Environment distribution matters more than environment count: The paper demonstrates that training effectiveness depends on how environments are distributed, not just how many there are. This challenges the prevailing "scale is all you need" assumption in environment-based training.

2. Multimodal-specific challenges require multimodal-specific solutions: The finding that multimodal environments amplify gradient conflicts and negative transfer suggests that methods designed for text environments cannot be directly transferred. The identification of visual state extraction and world modeling as key bottlenecks provides a theoretical basis for curriculum design.

3. Ability-based diversity is superior to surface-level diversity: Measuring diversity through agent behaviors and meta-ability profiles captures the actual learning requirements better than task descriptions or code embeddings.

Practical Implications

1. Efficient training: AES selects only 30 out of 200 environments while achieving full core-ability coverage, substantially reducing training costs without sacrificing—and even improving—performance.

2. Generalizable method: AES's environment selection, based on Qwen3-VL-4B trajectories, remains effective for Qwen3-VL-8B, suggesting the method captures model-agnostic ability requirements.

3. Stabilized training: HDC mitigates the instability of mixed-environment training by providing reliable learning signals through scaffolds, making joint training on heterogeneous environments more robust.

4. Benchmark improvements: Trained models show modest but positive gains on general multimodal benchmarks (MathVision, MMMU, MMStar), indicating that environment-based training can complement static dataset training.

---

Conclusion

Main Takeaways

This paper demonstrates that simply scaling multimodal environments is not always effective for agent training. The effectiveness of an environment distribution depends on two critical dimensions:

• Diversity: measured by ability coverage, redundancy, and optimization conflicts (addressed by AES).
• Difficulty structure: organized through hierarchical curriculum learning that targets multimodal-specific bottlenecks (addressed by HDC).

The combined AES + HDC framework achieves a 143.2% average relative gain across settings, showing that principled environment distribution design is a promising direction for multimodal agent research.

Limitations and Future Directions

1. Environment synthesis: The environment pool is built from existing works; large-scale environment synthesis remains unexplored.
2. Compute budget: Comparisons are made under a unified compute budget; larger budgets may yield different conclusions.
3. Conflict estimation efficiency: AES's gradient-based conflict analysis adds offline computation; more efficient estimation methods are needed.
4. Safety considerations: Environment-trained agents should only be deployed in controlled, verifiable settings, not open real-world systems without proper safeguards.