Wave Forcing: Towards Speed-of-Light Streaming Video Generation

Summary (Overview)

• Novel generation paradigm: Introduces Wave Forcing (WF), a causal wavefront-based approach that enables continuous video generation with 1F1C (one output chunk per concurrent forward round) after pipeline fill, using multi-step denoising with full-model replicas per stage.
• Three-stage training recipe: Combines mixed-noise initialization (DMD on RF/SF rollouts), off-policy DMD with teacher-guided motion supervision (LPIPS loss), and causal adaptation using self-generated history across RF/WF/SF branches, achieving quality comparable to Rolling Forcing while substantially improving motion dynamics.
• Wave Runtime (WaveRT): Implements Wave Parallelism (WP) with early layer-wise K/V delivery, communication–computation overlap, direct paged writes into attention buffers, and load-balanced VAE pipeline parallelism for sustained end-to-end throughput.
• Significant efficiency gains: 4-step WF achieves 147.6 DiT FPS and 134.8 end-to-end FPS on eight H200 GPUs (7.07× single-GPU speedup); WP outperforms Ulysses/Ring context-parallel baselines by up to 2.53× under Sage+W8A8 across model scales from 1.3B to 14B.
• Improved motion with stability: 5-step WF raises VBench-Long Dynamic Degree from 36.63 (RF) to 91.45 while improving Subject Preservation (97.10 vs. 92.75) and maintaining low Camera Drift (0.37 vs. 0.17).

Introduction and Theoretical Foundation

Real-time streaming video generation requires continuous output with consistent appearance and natural motion. Existing chunk-wise autoregressive methods face two fundamental challenges:

1. Efficiency bottleneck: Under the clean-history caching convention, sequential execution requires both denoising and a separate "store" forward pass to refresh K/V history from clean results, yielding NF1C (N Forward Rounds, 1 Chunk Output). Even one-step denoising requires two forward passes.

2. Quality degradation: Longer rollouts can degrade quality and cause camera drift, as models struggle with accumulated errors and inconsistent conditioning.

The key theoretical insight of Wave Forcing is that earlier chunks execute later denoising stages while newer chunks execute earlier ones, forming a moving wavefront (inspired by wavefront scheduling from Jin et al., 2016). This staggered execution pattern is enabled by chunk-wise causal attention, where each chunk attends to:

• Full attention within itself
• Cached history
• Live context from earlier, cleaner chunks

The mask is defined as:

$$M_{\mathrm{WF}}(s, j) = \mathbf{1}[j \geq s], \quad s, j \in \mathcal{A}_k$$

This directional conditioning accommodates differences in stage-wise progress and enables overlapped execution without dependence on later, noisier chunks.

Methodology

1. Causal Wavefront Generation

The denoising process for chunk $i$ at step $s$ follows:

$$Z_i^{(s+1)} = \Psi_{\nu_{s+1}}\Big(G_\theta(Z_i^{(s)}, \nu_s \mid \mathcal{C}_{i,s}), \epsilon_{i,s}\Big), \quad 0 \leq s < S$$

where $G_\theta$ predicts clean latents, $\Psi_\nu$ re-noises using draw $\epsilon_{i,s}$, and $\mathcal{C}_{i,s}$ denotes layer-wise K/V context.

At filled runtime tick $k$, the denoising inputs form:

$$\mathcal{F}_k = \{Z_k^{(0)}, Z_{k-1}^{(1)}, \ldots, Z_{k-S+1}^{(S-1)}\}$$

Adding store at $s = S$ gives $\mathcal{W}_k = \mathcal{F}_k \cup \{B_{k-S}\}$, of width at most $S+1$.

The ordered context at Transformer layer $\ell$ is:

$$\mathcal{C}_{k,s}^{\mathrm{WF},\ell} = A_k^{\ell}\oplus H_k^{\ell}\oplus U_k^{\ell}\oplus \Bigl[\bigoplus_{j\in\mathcal{A}_k, j\geq s}(K_{k,j}^{\ell},V_{k,j}^{\ell})\Bigr]$$

where $A_k^\ell$ is the anchor, $H_k^\ell$ is bounded clean history, $U_k^\ell$ is live store, and the last term concatenates denoising K/V in increasing $j$ (self first).

2. Three-Stage Training

Stage 1: Mixed-noise initialization — Adopts RF's mixed-noise joint-denoising setup with bidirectional attention across chunks at different noise levels. Samples RF or SF rollouts with equal probability, trained with distribution matching distillation (DMD).

Stage 2: Off-policy DMD — Re-noises consecutive teacher chunks at staggered timesteps with preceding teacher chunks as clean K/V history. The loss combines DMD and LPIPS:

$$\mathcal{L}_2 = \mathbb{E}\left[\mathcal{D}_{\mathrm{DMD}}(\widehat{Y}_\theta; \mathbf{c}) + \lambda_p \operatorname{LPIPS}\left(\mathcal{T}_{\mathrm{perc}}(\widehat{Y}_\theta), \mathcal{T}_{\mathrm{perc}}(Y_{\mathrm{win}})\right)\right], \quad \lambda_p = 0.25$$

Stage 3: Causal context and self-generated history — Uniformly samples self-generated RF, WF, or SF rollouts for on-policy DMD:

$$\mathcal{L}_3 = \mathbb{E}_{b \sim \mathrm{Unif}\{\mathrm{RF}, \mathrm{WF}, \mathrm{SF}\}}[\mathcal{L}_b] = \frac{1}{3}\mathcal{L}_{\mathrm{RF}} + \frac{1}{3}\mathcal{L}_{\mathrm{WF}} + \frac{1}{3}\mathcal{L}_{\mathrm{SF}}$$

3. Wave Runtime (WaveRT) with Wave Parallelism

Each denoising stage and store owns a full-model GPU replica. Within a round, dependencies are:

$$P_{s,\ell} \to C_{s,\ell}, \quad C_{s,\ell} \to P_{s,\ell+1} \quad (\ell < L)$$ $$P_{j,\ell} \to C_{s,\ell}, \quad j > s, \quad s, j \in \mathcal{A}_k$$

Key optimization: rank $s$ reads $N_s$ keys where:

$$N_s = N_{\mathrm{prefix}} + (S - s)N_C$$

Cleaner ranks read fewer keys, permitting later-layer producer lead. WaveRT publishes K/V after QKV projection and RoPE, then checks readiness immediately before attention. The exposed wait for edge $e$ is:

$$\Delta_e = t_e^{\mathrm{need}} - t_e^{\mathrm{ready}}, \quad w_e = \max(0, \tau_e - \Delta_e)$$

Delivery is fully hidden when every $\tau_e \leq \Delta_e$.

Empirical Validation / Results

Generation Quality (Table 1)

Key findings:

• 5-step WF achieves the best VBench Total (81.64) and Quality (84.40)
• Causal Forcing has severe stability issues: Subject Preservation of only 71.11 and Camera Drift of 11.14 (subject disappears at 30 and 75 seconds)
• WF raises Dynamic Degree from 36.63 (RF) to 91.45 on 100-second videos while maintaining low Camera Drift

Training Ablations (Table 2)

All three training branches (RF, WF, SF) and off-policy DMD are essential for the best balance of quality, motion, and stability.

System Efficiency

• 4-step WF: 147.6 DiT FPS, 134.8 E2E FPS on 8×H200 (7.07× single-GPU speedup)
• 14B model: 28.74 DiT FPS vs. 5.51 FPS single-GPU reference (6.08× speedup)
• WaveRT vs. Sync: Paged delivery reduces BF16 tick p50 from 91.96 to 78.88 ms (14.2%) at 1.3B and from 464.27 to 413.80 ms (10.9%) at 14B
• WP vs. CP baselines: WP achieves 1.025–2.530× the throughput of the faster Ulysses/Ring baseline under Sage+W8A8 across all six model scales

Theoretical and Practical Implications

Theoretical Contributions

1. Causal wavefront generation: WF demonstrates that chunk-wise causal attention with mixed-noise contexts can replace reverse refinement (RF's approach) while maintaining quality, fundamentally changing how streaming video generation can be parallelized.

2. Training-execution codesign: The paper shows that the training recipe must be adapted to the deployment graph—removing access to later, noisier chunks requires explicit training with mixed-noise contexts and self-generated history.

3. Parallelism beyond context splitting: WP introduces a new parallelization dimension (stage-level chunk parallelism) complementary to traditional context parallelism, with different scaling characteristics.

Practical Implications

1. Real-time streaming: At 134.8 E2E FPS, WF enables real-time video generation at 832×480 resolution, making interactive applications feasible.

2. Stability metrics: The paper identifies that high benchmark scores can coexist with subject loss and camera drift, motivating new diagnostic metrics (Subject Preservation, Camera Drift) for long-video evaluation.

3. Scalability trade-offs: WP's advantage over CP narrows for larger models, and stage parallelism is bounded by the denoising step count, suggesting hybrid approaches may be needed for extreme scales.

Conclusion

Wave Forcing jointly designs causal context, training strategies, and multi-GPU pipelining to denoise multiple video chunks concurrently. The key achievements are:

• 1F1C throughput: One completed chunk per concurrent forward round after pipeline fill, retaining all denoising updates
• Quality preservation: Overall quality comparable to Rolling Forcing with substantially improved motion dynamics (Dynamic Degree 91.45 vs. 36.63 on long videos)
• Efficient execution: WaveRT translates stage concurrency into sustained end-to-end throughput via early K/V delivery and communication–computation overlap

Future Directions

1. Simplified training: Reduce the complexity of the multi-stage training procedure while preserving quality
2. Hybrid parallelism: Combine WP with context parallelism to overcome the fixed-step scaling limit
3. Interactive generation: Support prompt updates during generation, camera/action control, and closed-loop tasks
4. Richer evaluation: Integrate streaming-interaction benchmarks and understanding-based quality judges
5. Serving concurrent streams: Develop latency-aware batching and admission control for multi-stream scenarios

The paper demonstrates that jointly designing generation context, training, and execution can translate cross-chunk parallelism into practical gains in video generation efficiency, opening a promising direction toward speed-of-light streaming video generation.