Summary (Overview)

• Core contribution: The paper introduces FFDC-WAM, a framework for adaptive action execution in World Action Models (WAMs) that treats execution as a future–reality verification problem—the robot executes longer when the WAM-predicted future remains consistent with reality and replans earlier when deviations occur.
• Key innovation: The Future Forward Dynamics Causal Attention (FFDC) verifier is a lightweight Transformer that jointly reasons over predicted future actions, predicted visual dynamics, real observations, and language instructions to output a confidence score for the remaining action rollout.
• Adaptive chunking emergent behavior: The effective action chunk size becomes an emergent consequence of prediction–observation consistency rather than a fixed hyperparameter, enabling long execution in predictable phases and short execution in contact-rich or difficult phases.
• Mixture-of-Horizon Training: A training strategy that improves long-horizon trajectory coverage by uniformly sampling conditioning timesteps across the entire episode, reducing bias toward early-episode prefixes.
• Key results: On RoboTwin benchmark, FFDC-WAM reduces WAM forward passes by 69.10% and execution time by 34.02% while improving success rate by 2.54% over the short-chunk baseline. In real-world experiments, it improves success rate by 35% (from 45% to 80%).

---

Introduction and Theoretical Foundation

Background and Motivation

Humans do not execute actions blindly; they constantly compare predicted future feedback with actual observations. When predictions match reality, we act smoothly over long horizons; when mismatches occur, we slow down, correct, or replan. This prediction–observation comparison is central to robust physical interaction, especially in contact-rich or uncertain environments.

World Action Models (WAMs) extend conventional vision-language-action (VLA) policies by jointly predicting future visual observations and future actions. Unlike VLAs that primarily model the action modality, WAMs benefit from dense supervision in video space, capturing multiple control-relevant distributions within a unified framework:

• Forward dynamics: $p(o' | o, a)$
• Inverse dynamics: $p(a | o, o')$
• Marginal action distribution: $p(a | o)$
• Marginal image distribution: $p(o' | o)$

Problem Statement

Current WAMs typically execute a fixed number of predicted actions after each model inference, leaving the robot blind to whether the imagined future remains consistent with the physical rollout. This is fundamentally suboptimal because:

• Simple/predictable dynamics (e.g., approaching a rigid cup): predictions remain accurate over long horizons; frequent replanning wastes computation.
• Deformable/contact-rich/stochastic interactions (e.g., folding cloth): predictions quickly become unreliable; long open-loop execution causes failures.

Key Insight

The defining property of WAMs—predicting future visual observations alongside actions—creates a new form of self-verification: during execution, the robot can compare real observations with WAM-predicted observations and jointly reason over them with the action sequence to assess whether the remaining rollout is still compatible with reality.

---

Methodology

3.1 Preliminary: World Action Model with Action Chunking

The framework builds on Motus [1], a WAM that jointly predicts future actions and future visual observations conditioned on the current observation and language instruction. The training loss combines rectified flow-matching losses for both action and video prediction:

$$\mathcal{L}_{WAM} = \mathcal{L}_{act} + \mathcal{L}_{vid}$$

At inference time, given the current observation $o_t$ and instruction $\ell$, the WAM predicts:

$$(\hat{A}_{t+1:t+H}, \hat{O}_{t+1:t+H}) = \pi_\theta(o_t, \ell)$$

where $\hat{A}_{t+1:t+H}$ is the predicted action chunk of length $H$, and $\hat{O}_{t+1:t+H}$ is the predicted latent visual sequence.

Adaptive Action Execution with Verifier

A verifier $\mu_\phi$ decides whether the remaining predicted rollout is trustworthy:

$$e_t = \mu_\phi(o_t, \hat{A}_{t:t+k}, \hat{O}_{t:t+k}, \ell)$$

where $e_t \in [0,1]$ is a confidence score. The decision rule is:

$$\text{execute if } e_t \geq \tau, \quad \text{replan if } e_t < \tau$$

with threshold $\tau = 0.5$ in this paper.

3.2 Future Forward Dynamics Causal Attention (FFDC)

Verifier Input Sequence

At verification step $t$, the verifier takes:

• Current real observation tokens $O_t$
• Semantic tokens $L$ from the Understanding expert
• WAM-predicted historical video tokens $\hat{O}_t^p$
• WAM-predicted future video tokens $\hat{O}_t^f$
• Future action segment $\hat{A}_t$
• A learnable [CLS] token

The input sequence is:

$$X_t = [L, \hat{O}_t^p, O_t, \hat{O}_t^f, \hat{A}_t, [CLS]]$$

Causal Attention Mechanism

FFDC uses a structured Boolean visibility matrix $M$ where $M(i,j) = 1$ means token $x_i$ can attend to token $x_j$. The mask enforces temporally aligned causal interaction:

• Each future visual token $\hat{O}^{(j)}_{t_f}$ attends only to $\{\hat{O}^{(\leq j)}_{t_f}, \hat{A}^{(\leq t+jr)}_t\}$
• Each future action token attends only to $\{\hat{O}^{(\leq j)}_{t_f}, \hat{A}^{(\leq t+jr)}_t\}$
• A local window restricts attention to nearby aligned action/visual tokens for computational efficiency

Efficiency via KV Cache

WAM-predicted tokens are produced once and stored as a KV cache. During execution, the verifier only encodes the latest real observation and performs lightweight attention against cached tokens—avoiding full WAM reruns.

Score Computation

The [CLS] token output passes through an MLP head:

$$z_t = g_\psi(\text{FFDC}(X_t)[CLS])$$ $$e_t = \sigma(z_t) \in [0,1]$$

3.3 Training Strategy and Dataset Construction

Mixture-of-Horizon Training

For an episode of length $T$, a conditioning timestep is sampled uniformly: $s \sim U\{1, \ldots, T\}$. Given horizon $H$, action and video indices are:

$$\tau_i = \min(s + i, T), \quad \upsilon_j = \min(s + jr, T)$$

yielding sequences:

$$A_s = [a_{\tau_0}, \ldots, a_{\tau_{H-1}}], \quad O_s = [o_{\upsilon_0}, \ldots, o_{\upsilon_{H/r-1}}]$$

Out-of-range positions are padded by repeating the final valid action/frame.

Verifier Dataset Construction

A binary dataset $\mathcal{D}_{ver} = \{(X, y)\}$ is constructed where $y \in \{0,1\}$ indicates executability:

• Positive samples ($y=1$): valid segments from demonstrations and successful rollouts
• Negative samples ($y=0$): failed rollouts and synthetically corrupted action segments

Data augmentation methods include:

1. Temporal swap: randomly swaps two pairs of actions within a horizon
2. Gripper flip: negates designated gripper dimensions
3. Late-stage Gaussian noise: perturbs the second half of the sequence
4. Tail scaling: shrinks a randomly sampled suffix by a random scale factor

Verifier Training Loss

$$\mathcal{L}_{ver} = -\left[ y \log \sigma(z) + (1-y) \log(1-\sigma(z)) \right]$$

---

Empirical Validation / Results

Experimental Setup

• Backbone: Motus [1] as the WAM, trained on four NVIDIA A100 GPUs (80GB each)
• FFDC verifier: trained on a single A100 GPU
• Benchmarks: RoboTwin simulator [4] (50 manipulation tasks, clean and random settings) and real-world Astribot S1 robot (34 DoF, two pick-and-place tasks)
• Baselines: Base-Motus (chunk size 16), LC-16/32/48/64 (long-chunk backbones trained with chunk size 64, executing first 16/32/48/64 actions)

Main Results on RoboTwin

Key findings:

• FFDC-WAM achieves the highest average SR across all settings
• On hard tasks: SR improves from 54.20% → 76.40% (Rand.hard) and 57.80% → 76.00% (Clean.hard)
• On easy tasks: completion time drops from 23.5s → 15.7s (Rand.easy) and 20.4s → 12.9s (Clean.easy)
• Reduces model calls by 69.10% vs. Base-Motus under random setting

Qualitative Analysis

Move can pot (simple task): Base-Motus requires 3 WAM inferences due to fixed short-horizon execution; FFDC-WAM completes with only 1 inference (consistently high confidence scores).

Hanging mug (hard task): Base-Motus needs 7 inferences; FFDC-WAM executes long chunks in the predictable transport stage, then switches to frequent replanning in the final precision-critical stage when confidence drops. Without FFDC, long-chunk execution leads to accumulated error and failure.

Real-World Results

FFDC-WAM improves average success rate from 45% to 80% (+35%). The slightly higher execution time reflects additional computation spent on online correction, which substantially improves robustness in real-world settings with perception noise and contact uncertainty.

Ablation Study

Ablation insights:

• Removing predicted visual tokens causes the largest SR drop (76.4% → 71.6%): imagined future observations are the most informative signal
• Removing real observation also causes a clear drop (72.4%): comparing predicted dynamics against actual state is crucial
• Removing action input reduces SR to 73.4%: predicted control sequence provides complementary information
• Removing language conditioning causes a smaller drop (74.8%): task semantics help assess rollout validity

---

Theoretical and Practical Implications

Theoretical Significance

1. New paradigm for WAM execution: The paper reframes execution from fixed-horizon chunking to reliability-aware control, where the effective chunk size emerges from prediction–observation consistency rather than being a manually tuned hyperparameter.

2. Self-verification capability: WAMs' joint video-action prediction creates a unique opportunity for internal self-verification that action-only policies cannot provide. This is a fundamental advantage of the WAM paradigm over conventional VLAs.

3. Causal attention design: The structured Boolean visibility matrix enforces temporally aligned causal interaction between action and visual modalities, providing a principled way to detect task-critical mismatches without information leakage.

Practical Implications

1. Efficiency–robustness trade-off: The method achieves both higher success rates AND lower inference costs simultaneously—a rare combination. On easy tasks it approaches the efficiency of long-chunk baselines while on hard tasks it maintains the robustness of short-chunk execution.

2. Difficulty-aware inference allocation: FFDC-WAM automatically allocates more computation to hard tasks and less to easy ones, adapting inference frequency to task difficulty without explicit task classification.

3. Real-world applicability: The 35% improvement in real-world success rate demonstrates practical value beyond simulation, particularly in handling perception noise, actuation error, and contact uncertainty.

4. KV-cache efficiency: Storing WAM-predicted tokens as a KV cache makes the verifier lightweight enough for high-frequency verification during execution without rerunning the full WAM.

---

Conclusion

Main Takeaways

The paper reformulates adaptive WAM execution as a future–reality verification problem: rather than selecting a single execution length, the system verifies whether the WAM's imagined future can still be trusted during rollout. The proposed FFDC-WAM framework achieves this through:

1. FFDC verifier: A lightweight Transformer with structured causal attention that jointly models predicted actions, predicted visual dynamics, real observations, and language instructions
2. Adaptive trust: Long execution when predictions remain reliable, replanning when future–reality consistency breaks down
3. Mixture-of-Horizon Training: Improved long-horizon trajectory coverage for robust adaptive execution

Key Results Summary

• RoboTwin: 69.10% reduction in WAM forward passes, 34.02% reduction in execution time, 2.54% improvement in success rate over short-chunk baseline
• Real world: 35% improvement in success rate (45% → 80%)
• Best overall performance across all settings compared to fixed-chunk baselines

Limitations and Future Directions

1. Binary supervision: FFDC is trained with binary supervision from successful, failed, and synthetically corrupted segments, which may not cover the full diversity of real-world execution deviations
2. Fixed threshold: The detection threshold of 0.5 is fixed; a systematic study of threshold effects on robustness–efficiency trade-off could improve performance
3. Lightweight design: Further exploration of the trade-off between FFDC parameter scale and verification capability is needed
4. Future work: Extending the verifier to learn from richer failure modes and more diverse real-world data