Summary (Overview)

• This paper presents a systematic empirical study of test-time future modeling in World Action Models (WAMs), addressing the debate between explicit WAMs (which denoise future frames during inference) and latent WAMs (which discard future generation for efficiency).
• The authors find that while latent WAMs match explicit WAMs on in-distribution tasks, they suffer significant degradation across three generalization axes: environmental perturbation, data efficiency, and task generalization.
• A key finding is that almost all the generalization benefit comes from a single forward pass over fully noised future video tokens, not from iterative denoising into clean frames.
• Based on this, the authors propose Simple-WAM, which conditions on future video tokens left at Gaussian noise with a single video expert pass, achieving explicit-level generalization at latent-level efficiency.
• Simple-WAM demonstrates the best of both worlds across simulation (LIBERO, RoboTwin 2.0, LIBERO-Plus) and real-world tasks, outperforming explicit WAMs in generalization while maintaining latency comparable to latent WAMs.

Introduction and Theoretical Foundation

World Action Models (WAMs) jointly generate future dynamics and actions conditioned on observations and language instructions, providing supervision in observation space beyond sparse action labels. The central research question is whether inference-time video generation is necessary for generalization, or whether future modeling serves only as a training objective.

The paper identifies two competing paradigms:

• Explicit WAMs: Denoise future frames into clean frames alongside every action chunk, conditioning actions on the generated future. These report success rates tracking the quality of generated futures.
• Latent WAMs: Retain video supervision during training but skip heavy video generation at inference, reporting comparable in-distribution performance at a fraction of the cost.

The authors argue that previous evidence was collected in distribution, leaving the generalization question open. They consolidate generalization into three complementary axes:

1. Environmental perturbation: Robustness to changes in lighting, object layout, and background.
2. Data efficiency: Performance with fewer demonstrations.
3. Task generalization: Generalization to tasks unseen during action training.

Methodology

Model Formulation

A WAM couples a video expert (parameters $\theta$) and an action expert (parameters $\psi$). The video expert reads a sequence of latents spanning current and future frames, with future frames carrying a flow time $\tau \in [0, 1]$ (with $\tau = 0$ at clean data and $\tau = 1$ at Gaussian noise). The action expert regresses its own velocity field $v_\psi$ over an independent flow time $\sigma$ and interpolant $A_t^\sigma$. Training minimizes:

$$\bar{\mathcal{L}} = \mathcal{L}_{\mathrm{act}} + \lambda \mathcal{L}_{\mathrm{vid}}$$

At inference, the action chunk is produced by integrating the action velocity field from $\sigma_1 = 1$ to $\sigma_{K+1} = 0$ over $K$ steps.

Controlled Comparison Design

The explicit and latent paradigms differ only in a structured attention mask controlling whether action tokens attend to future video tokens. Key cost analysis:

• Explicit paradigm: $\bar{K} C_{\mathrm{vid}} + K C_{\mathrm{act}}$ (denoising over the whole schedule)
• Latent paradigm: $C_{\mathrm{vid}} + K C_{\mathrm{act}}$ (single video pass)

Simple-WAM Design

Simple-WAM makes two changes:

1. Drops iterative denoising: The video expert makes one forward pass with future video tokens left at Gaussian noise ($\tau = 1$).
2. Adapts training noise schedule: A mixture of flow-time sampling that emphasizes $\tau = 1$ (the only flow time used at inference) while preserving original training distribution.

The inference cost becomes $C_{\mathrm{vid}} + K C_{\mathrm{act}}$, identical to latent WAMs.

Empirical Validation / Results

Finding 1: Generalization Deteriorates without Inference-time Future Modeling

Table 1: Generalization Deteriorates without Inference-time Future Modeling

• In distribution, the two paradigms are within 0.9 points (97.75 vs. 96.85), reproducing prior parity results.
• Under environmental perturbation, the explicit paradigm leads by 13.97 points; under data efficiency, by 8.45 points.

Finding 2: A Single Forward Pass Recovers Most of the Gap

• The generalization gain comes from the intermediate representation formed by a single forward pass, not from the denoised future itself.
• Future video tokens left at pure noise ($\tau = 1$) recover almost all of the gap, without iterative denoising.

Main Results Across All Axes

Table 2: Main results across the three generalization axes

• Simple-WAM averages 79.5 over seven LIBERO-Plus perturbation factors, against 67.7 (explicit) and 53.8 (latent).
• On real-world tasks, in-distribution performance is within 2.3 points across models, but generalization gaps persist.

Ablations

• Noise level robustness: Performance is stable across mixture probabilities $p = 0.25, 0.5, 0.75$ (within 1.3 points), showing robustness to the choice of $p$.
• The design preserves original training strategy, benefiting most from video pretraining and emphasizing pretraining-aligned conditioning.

Theoretical and Practical Implications

Theoretical implications:

• Future modeling is a test-time requirement, not merely a training objective—challenging the latent paradigm's core claim.
• The mechanism of benefit is feature preparation, not future generation per se: the action expert conditions on informative representations of "what could happen" without needing clean predictions.
• The finding that a single noisy forward pass suffices suggests WAMs learn to extract useful spatiotemporal structure from partially observed futures.

Practical implications:

• Simple-WAM offers a practical recipe: keep future tokens in context at pure noise, use a single video pass, and align the training noise schedule with inference behavior.
• This achieves explicit-level generalization with latent-level efficiency, making WAMs more viable for real-time embodied applications.
• The three-axis evaluation framework provides a reusable protocol for future WAM research.

Conclusion

This paper demonstrates that inference-time future modeling is necessary for WAM generalization, but the full denoising schedule is not. The key insight is that a single forward pass over fully noised future tokens recovers almost all generalization benefit. Simple-WAM operationalizes this by conditioning on noisy future tokens with one video expert pass and adapting the training noise schedule accordingly.

Future directions:

• Scaling to larger models and more diverse tasks remains open.
• Investigating whether findings hold at larger scale or with different video backbones.
• Exploring whether the "feature preparation" insight generalizes to other conditional generation architectures.

The paper's controlled methodology and three-axis evaluation framework provide a solid foundation for continued investigation into what makes world action models generalize.