# BadWAM: When World-Action Models Dream Right but Act Wrong

> Adversarial perturbations can cause task failure in world-action models while keeping imagined futures visually plausible, a vulnerability called World-Action Drift.

- **Source:** [arXiv](https://arxiv.org/abs/2607.15207)
- **Published:** 2026-07-18
- **Permalink:** https://picx.dev/p/xmxc61
- **Whiteboard:** https://picx.dev/p/xmxc61/image

## Summary

## Summary (Overview)

- **BadWAM** identifies a new class of adversarial attacks specific to **World-Action Models (WAMs)**: small visual perturbations can cause **task failure** while the model's **imagined future remains visually plausible** — a phenomenon called *World-Action Drift*.
- Two attack instantiations are proposed: an **action-only attack** (maximizes action deviation) and an **imagination-preserving attack** (maintains future predictions close to clean rollouts while shifting actions). Both operate under **black-box query access**.
- Evaluated on **LIBERO** and **RoboTwin** benchmarks, the action-only attack reduces task success from 96.5% to 43.1% on the Action-only WAM; the imagination-preserving attack achieves comparable disruption while keeping future prediction drift low.
- Attacks transfer across different WAM architectures, and simple detection baselines (e.g., augmentation-consistency) achieve only **≤21.4% recall at 5% false-positive rate**, highlighting the challenge of defending against such attacks.
- The work demonstrates that **future prediction alone is an insufficient safety signal** for WAMs; the *alignment* between action and imagination is the critical security property.

---

## Introduction and Theoretical Foundation

### Background and Motivation

World-Action Models (WAMs) are a promising paradigm for embodied AI: they **couple action generation with future world prediction**. A conventional policy maps observation and instruction directly to actions:
$$ a_{t:t+H-1} \sim \pi_\theta(o_t, g). $$
By contrast, a WAM additionally imagines future states (latent or decoded) before or jointly with action prediction:
$$ z_{t+1:t+K} \sim W_\theta(o_t, g),\quad a_{t:t+H-1} \sim A_\theta(o_t, g, z_{t+1:t+K}). $$
or as a joint distribution:
$$ (z_{t+1:t+K}, a_{t:t+H-1}) \sim p_\theta(z_{t+1:t+K}, a_{t:t+H-1} | o_t, g). $$

This coupling is frequently argued to improve **robustness, interpretability, and safety** — e.g., imagined futures can be inspected by a safety monitor:
$$ \text{safe} = M(z_{t+1:t+K}, g). $$

### The Security Gap

The paper questions whether that promise holds under **adversarial observation perturbations**. The central finding: **action generation and future imagination can fail asynchronously**. An attacker can desynchronize the action from the imagined future, causing task failure while the model still produces a plausible-looking future. This is a **WAM-specific vulnerability**, distinct from attacks on classifiers, world models, or reactive policies.

Empirical motivation (cf. Figure 1) shows that in closed-loop traces, **failed episodes have larger action shifts** than successful ones, but **predicted-future shifts overlap** — indicating that action and imagination decouple under attack.

### Related Work

- **WAMs** (e.g., Fast-WAM, OA-WAM, ABot-M0.5, VT-WAM) increasingly rely on world-prediction signals. The paper notes that even WAMs that skip test-time future generation may inherit vulnerabilities from the world-action coupling trained into their representations.
- **Adversarial attacks on embodied AI** previously targeted reactive policies, perception modules, or world models (e.g., BadWorld, JailWAM). BadWAM targets the *alignment* between action and imagination, not each output independently.
- This complements prior work on oracle-level integrity attacks (e.g., corrupting imagination for downstream safety gates): BadWAM shows that **even if the imagination looks clean, the action may be hijacked**.

---

## Methodology

### Threat Model

The adversary perturbs the **visual observation** at each replanning step:
$$ \tilde{o}_t = \text{clip}(o_t + \delta_t),\quad \|\delta_t\|_\infty \le \varepsilon. $$
The adversary cannot modify instructions, robot state, model parameters, or environment. Access levels:
- **Action-only**: observes only the output action chunk.
- **Imagination-visible**: also sees the predicted future (latent or video).

The objective is **untargeted task failure**. For stealth, the adversary may also want the **imagined future to remain close** to the clean prediction.

### BadWAM Framework

BadWAM treats the WAM as a **queryable input-output system** and optimizes a perturbation at each step to maximize a scalar objective $J$. It defines two distances:
- $D_{\text{act}}$: deviation between clean and attacked action chunks.
- $D_{\text{img}}$: drift between clean and attacked imagined futures.

The **stealth attack** is formulated as:
$$ \max_{\|\delta\|_\infty \le \varepsilon} D_{\text{act}}(a^\delta, a) \quad \text{s.t.} \quad D_{\text{img}}(z^\delta, z) \le \tau_{\text{img}}. $$

#### Action-Only Attack
Maximizes action deviation only:
$$ \delta_t^\star = \arg\max_{\|\delta_t\|_\infty \le \varepsilon} D_{\text{act}}\left(a_{t:t+H-1}^\delta, a_{t:t+H-1}\right). $$
No constraint on imagination; used as the high-strength endpoint.

#### Imagination-Preserving Attack
Solves a Lagrangian relaxation:
$$ \delta_t^\star = \arg\max_{\|\delta_t\|_\infty \le \varepsilon} D_{\text{act}}(a^\delta_{t:t+H-1}, a_{t:t+H-1}) - \lambda\, D_{\text{img}}(z^\delta_{t+1:t+K}, z_{t+1:t+K}). $$
When decoded future video is available, $D_{\text{img}}$ is an average frame-level distance:
$$ D_{\text{img}}(v^\delta_{t+1:t+K}, v_{t+1:t+K}) = \frac{1}{K}\sum_{k=1}^K d_{\text{frame}}(v^\delta_{t+k}, v_{t+k}). $$

### Query-Based Online Optimization

BadWAM uses **zeroth-order finite-difference** optimization (no gradients needed). At each replan:
1. Sample random directions $u_i$.
2. Estimate gradient:
$$ \hat{\nabla} J(\delta_t) = \frac{1}{m}\sum_{i=1}^m \frac{J(\delta_t + c u_i) - J(\delta_t - c u_i)}{2c} u_i. $$
3. Update and project:
$$ \delta_t \leftarrow \Pi_{[-\varepsilon,\varepsilon]}\left(\delta_t + \eta \hat{\nabla}J(\delta_t)\right). $$
The objective $J$ is either action-only or imagination-preserving. The best perturbation within the budget is used for that step.

### Closed-Loop Execution Metrics

Three groups:
- **Task-level**: success rate, induced failures.
- **Action disruption**: action distance, channel-level and horizon-level shifts.
- **Stealthiness** (imagination-preserving attacks): predicted-future distance, adversarial score.

---

## Empirical Validation / Results

### Experimental Setup

- **Benchmarks**: LIBERO (4 suites: Spatial, Object, Goal, Long-horizon) and RoboTwin (dual-arm manipulation).
- **WAM variants**: Action-only WAM (direct mapping), Joint WAM (jointly predicts futures and actions), IDM WAM (first imagines, then decodes action).
- **Default attack parameters**: $\varepsilon=0.06$, 8 optimization iterations per replan (17 total queries incl. clean reference).
- **Metrics**: closed-loop success rate (lower = stronger attack), action distance, future distance.

### RQ1: Attack Effectiveness

**Table 1** summarizes main results:

| Model | Clean | Action-only | Img-pres. | Clean (RoboTwin) | Action-only (RoboTwin) | Img-pres. (RoboTwin) |
|-------|-------|-------------|-----------|------------------|------------------------|----------------------|
| Action-only WAM | 96.5 | 43.1 (↓53.4) | – | 92.1 | 84.4 (↓7.7) | – |
| Joint WAM | 98.1 | 61.5 (↓36.6) | 63.0 (↓35.1) | 90.9 | 84.4 (↓6.5) | 85.2 (↓5.7) |
| IDM WAM | 98.4 | 66.1 (↓32.3) | 68.1 (↓30.3) | 91.4 | 83.7 (↓7.7) | 85.1 (↓6.3) |

- **Action-only attack** reduces success dramatically (e.g., Action-only WAM from 96.5% to 43.1%).
- **Imagination-preserving attack** achieves comparable disruption (Joint WAM: 63.0% vs 61.5% for action-only).
- On **RoboTwin** the drops are smaller but still consistent (~6-8 percentage points).  
- **Comparison with random noise** (Figure 6): Random ε=0.06 leaves success at 71.0%/75.2% (Joint/IDM), far higher than BadWAM's 61.5%/66.1%. White-box gradient-based attacks achieve lower success (49.2%/52.8%) – BadWAM is weaker but operates with black-box access.

### RQ2: Systematicity of Failures

- **Figure 8 (per-suite)**: Spatial and Long-horizon tasks suffer most (e.g., Action-only WAM Spatial: 96.5% → 16.0%); Object tasks remain relatively robust (93.0%).
- **Figure 7 (pass@k)**: The success gap between clean and attacked persists across 20 trials per task, indicating systematic failure rather than unlucky seeds.
- **Task-level failure bins (Figure 5)**: 42.5% of tasks fall into the 0-25% success bin for the action-only WAM under attack.

### RQ3: Failure Mechanism

- **Figure 9** shows a qualitative example: the attacked WAM initially acts plausibly but gradually drifts (knocks objects, fails to grasp) — closed-loop compounding rather than one bad action.
- **Figure 10**: per-replan action shifts persist across execution; failed episodes accumulate significantly larger cumulative shifts.
- **Figure 11**: search dynamics show the optimizer steadily improves action distance while future-video distance changes only mildly (+1.5% vs +8.3% action distance).
- **Figure 2**: action shifts concentrate on continuous channels (XYZ, rotation) and later portions of the action horizon, not uniform noise.

### RQ4: Stealth Under Matched Strength

- **Figure 12**: Under similar perturbation budgets, the imagination-preserving attack reduces average predicted-future distance from 14.01 to 13.04, with lower future distance on 39 of 40 tasks.
- This supports the core claim: action-imagination decoupling is possible while maintaining substantial attack strength.

### RQ5: Sensitivity and Efficiency

- **Future-preserving weight λ (Figure 13)**: Moderate λ (0.005–0.015) improves the action-future tradeoff; excessive λ (>0.1) weakens attack.
- **Perturbation budget ε (Figure 14)**: Strong scaling – ε=0.01 leaves success >90%; ε=0.20 collapses to 0%.
- **Query budget B (Figure 14)**: Increasing B from 1 to 16 improves attack; B=32 offers no further gain despite higher runtime (from ~2.5s to ~28s per replan). Moderate budget captures most benefit.

### RQ6: Transferability and Defenses

**Table 2 (Transferability)**:

| Attack direction | Target clean | Target under attack | Drop (pp) |
|-----------------|--------------|---------------------|-----------|
| Action-only WAM → Joint WAM | 98.3† | 64.2 | 34.2 |
| Joint WAM → IDM WAM | 100.0† | 59.2 | 40.8 |
| IDM WAM → Joint WAM | 98.3† | 61.7 | 36.7 |
| Joint WAM → IDM WAM (img-pres.) | 100.0† | 60.8 | 39.2 |
| IDM WAM → Joint WAM (img-pres.) | 98.3† | 63.3 | 35.0 |

Perturbations transfer across architectures, indicating shared observation-space directions affecting action generation.

**Table 3 (Non-adaptive defenses)** on LIBERO subset:

| Model | Defense | Clean success | Attack success |
|-------|---------|---------------|----------------|
| Joint WAM | None | 98.3† | 57.1 |
| Joint WAM | Gaussian blur | 98.3† | 85.0 |
| Joint WAM | Resize-crop ens. | 52.5† | 0.0 |
| Joint WAM | JPEG-noise ens. | 94.2† | 89.2 |
| IDM WAM | None | 100.0† | 54.6 |
| IDM WAM | Gaussian blur | 98.3† | 89.2 |
| IDM WAM | Resize-crop ens. | 54.2† | 8.3 |
| IDM WAM | JPEG-noise ens. | 93.3† | 90.0 |

- **Gaussian blur** and **JPEG-noise ensemble** recover most of the success (e.g., 89.2% attack success for Joint WAM with JPEG-noise), but these defenses are **non-adaptive**.
- **Resize-crop** harms clean performance unacceptably.
- **Augmentation-consistency detection (Figure 15)** achieves AUROC of 0.675 (Joint) and 0.725 (IDM), but at **5% FPR** detects only **13.4% and 21.4%** of attacked replans — insufficient for robot safety.

---

## Theoretical and Practical Implications

- **World-action drift** is a fundamentally new attack surface: the adversary exploits the *interface* between two coupled capabilities (imagining and acting) rather than corrupting a single output.
- **Future-based monitors are overconfident**: a WAM can produce plausible imagined futures while executing failing actions. A safety monitor that only checks future plausibility will miss such attacks.
- **Evaluation must be multi-metric**: task success alone insufficient; action distance, future distance, horizon-level shifts, and transfer behavior are needed to distinguish overt hijacking from stealthy desynchronization.
- **Defenses are challenging**:
  - Simple preprocessing (blur, JPEG noise) helps against non-adaptive attacks but is not a principled solution; an adaptive attacker can circumvent them.
  - Detection via consistency suffers from low recall at acceptable FPR.
  - A robust defense must **restore action-imagination synchronization** under adversarial perturbations, not just suppress one attack pattern.
- **Transferability** indicates the vulnerability is architectural rather than implementation-specific, implying that security evaluations should cover multiple WAM families.

---

## Conclusion

- **BadWAM** demonstrates that world-action models are vulnerable to **World-Action Drift Attacks**: small black-box visual perturbations can cause task failure while the imagined future remains plausible.
- Two concrete attack instantiations cover a spectrum from **overt action hijacking** (action-only) to **stealthy decoupling** (imagination-preserving).
- Extensive closed-loop experiments on LIBERO and RoboTwin show substantial success drops (up to 53.4 percentage points), systematic failure patterns, and a tunable tradeoff between attack strength and stealthiness.
- The results highlight a **critical security gap**: coupling action generation with future prediction, often seen as a safety asset, can itself become an attack surface.
- Future directions: developing synchronization-aware defenses, extending the framework to physical-world attacks (adversarial patches), and integrating BadWAM into the safety evaluation pipeline for deployed WAM-based robots.

---

_Markdown view of https://picx.dev/p/xmxc61, served by PicX — AI-generated visual whiteboard summaries of research papers._