Summary (Overview)

• SolarWM is a fully open, unified foundation for interactive video world models, integrating a reconfigurable multi-source data engine with a scalable backbone-native adaptation framework.
• The data infrastructure processes 1.43 million canonical clips (25.85 TB) from 10 datasets spanning real-world, synthetic, and game environments, normalizing them into a consistent frame-aligned training contract.
• Four models (5B–33B parameters) are instantiated across heterogeneous backbones (Wan2.2, LTX-2.5, MiniMax-H3) using a unified three-stage training recipe: bidirectional adaptation, teacher-forced AnyFlow autoregressive initialization, and DMD-based causal training.
• The resulting causal models support real-time interaction and continuous hour-scale rollouts after training on only 5-second sequences, without attention-sink mechanisms or long-sequence fine-tuning.
• The complete stack—data, pipeline, recipes, weights, and framework—is released to enable reproducible research and systematic cross-backbone comparison.

Introduction and Theoretical Foundation

Interactive video world models transform passive video generators into explorable environments conditioned on camera motion, player actions, or semantic instructions. The transition from short-clip generation to interactive long-horizon rollout requires models to maintain visual quality and temporal coherence while remaining responsive to control.

The paper identifies two closely coupled challenges:

1. Data heterogeneity: Existing datasets differ in temporal scale, visual quality, motion distribution, captioning styles, and camera conventions. Naive mixing introduces inconsistent supervision, causing models to deteriorate under multi-source training.

2. Model heterogeneity: Video generators differ substantially in architecture—latent representations, attention structures, and conditioning mechanisms. A shared adaptation strategy risks compromising pretrained capabilities, while backbone-specific implementations limit systematic comparison.

The paper argues that existing open systems overlook this coupling, relying on restricted data sources and model-specific implementations. SolarWM addresses this gap by coordinating multi-source data construction and backbone adaptation while providing a standardized training protocol across backbones.

Methodology

Data Engine

The data component processes every canonical clip before applying training-time selection criteria, decoupling source preprocessing from mixture construction. Each sample follows a unified schema:

$$\mathcal {S} _ {i} = \big (V _ {i}, P _ {i}, K _ {i}, C _ {i}, m _ {i}, q _ {i}, \pi_ {i} \big)$$

where $V_i$ is video, $P_i \in \mathbb{R}^{N \times 4 \times 4}$ contains metric camera-to-world transforms, $K_i \in \mathbb{R}^{N \times 4}$ stores per-frame intrinsics $(f_x, f_y, c_x, c_y)$, $C_i$ is the dense caption, $m_i$ contains metadata, $q_i$ is the metric record, and $\pi_i$ stores provenance.

Key processing stages include:

• Metric-scale camera annotation: Pi3X estimates scene structure fused with MoGe-2 depth anchors, then VIPE SLAM estimates 6-DoF trajectories with per-frame intrinsics
• LTX Clean Plate processing: Removes dynamic people/vehicles while preserving static scene layout (543k clean clips)
• Dense captioning: Kimi-K2.6 captions all clips, excluding dynamic entities and camera-motion information from training captions
• Quality assessment: Multi-axis metrics covering camera integrity, visual quality, motion, temporal consistency, and semantic suitability

Training Pipeline

Stage 1: Bidirectional Adaptation minimizes:

$$\mathcal {L} _ {\mathrm{bid}} = \mathbb {E} _ {\mathbf {z} _ {0}, t, \epsilon} \left[ \| f _ {\theta} (\mathbf {z} _ {t}, t, \mathbf {c}) - \mathbf {u} _ {t} \| _ {2} ^ {2} \right]$$

using bidirectional attention, with camera control injected through fused-PRoPE (projective rotations applied directly to query, key, and value tensors in self-attention).

Stage 2: Teacher-Forced AnyFlow Initialization introduces causality:

$$\mathcal {L} _ {\mathrm{TF-AF}} = \mathbb {E} \left[ \sum_ {k} \ell_ {\mathrm{AF}} \left(f _ {\phi}; \mathbf {z} _ {t} ^ {k}, t, r, \mathbf {z} _ {0} ^ {< k}, \mathbf {c}\right) \right]$$

This directly produces a few-step autoregressive initializer, removing the need for separate Causal ODE or Causal CD stages.

Stage 3: DMD-based Causal Training addresses the exposure gap:

$$\nabla_ {\psi} \mathcal {L} _ {\mathrm{DMD}} = \mathbb {E} \left[ J _ {G _ {\psi}} ^ {\top} w (t) \left(s _ {\mathrm{fake}} (\mathbf {z} _ {t}, t, \mathbf {c}) - s _ {\mathrm{real}} (\mathbf {z} _ {t}, t, \mathbf {c})\right) \right]$$

where a frozen bidirectional teacher estimates the target distribution while a trainable fake-distribution model tracks the student's rollout distribution.

Empirical Validation / Results

Bidirectional Pretrained Models

All four backbone routes generate consistent 10-second videos from OOD first frames (created with GPT Image 2 or Krea) under translation, rotation, and mixed 6-DoF camera paths, demonstrating that the unified data and camera-conditioning contract generalizes beyond training domains.

Distilled Causal Generation

In-domain validation: 10-second third-person rollouts maintain subject appearance and spatial relationships; minute-scale rollouts sustain scene consistency across real-world, synthetic, and game domains.

OOD initialization: The model converts externally synthesized still images into navigable video while retaining initial style, palette, and scene elements. Critically, newly revealed content must be inferred by the world model, not copied from the image generator.

Long-horizon generation: Continuous autoregressive generation at minute and hour scales without restarting from the input image, injecting reference frames, or using attention sinks. The 60-minute endpoints remain recognizable and visually coherent.

Corpus Statistics

876k clips are retained (471k high + 404k xhigh); 549k fully processed clips are published in the rejected partition with complete annotations and rejection reasons.

Theoretical and Practical Implications

• Scalable long-horizon generation without complex pipelines: Strong long-horizon world models can be obtained without specialized ODE or consistency-distillation initialization. Most optimization should occur during bidirectional training; AR adaptation converges rapidly, and DMD requires even fewer steps.

• Short-sequence training suffices: Training on only 5-second sequences enables open-ended rollouts over minutes-to-hours horizons, provided data construction, backbone adaptation, and training stages are properly aligned.

• Cross-backbone transferability: The unified recipe works across four distinct backbone families (Wan2.2, LTX-2.5, MiniMax-H3), demonstrating it is a reusable framework rather than a model-specific procedure.

• Reproducible research infrastructure: The release of complete data, pipeline, recipes, and weights enables controlled reconfiguration across data filters, mixtures, backbones, schedules, and stage settings—addressing a critical gap in the field where most systems release only checkpoints or partial code.

Conclusion

SolarWM provides a fully open foundation for interactive video world models, integrating a reconfigurable multi-source data engine with backbone-native adaptation. The key contributions are:

1. A reconfigurable multi-source data infrastructure processing 1.43M clips from 10 datasets with full annotations, metrics, and provenance
2. A scalable backbone-native adaptation framework with four 5B–33B models trained via a unified three-stage recipe
3. State-of-the-art performance with open-ended rollouts over minutes-to-hours horizons after training solely on 5-second sequences

The work demonstrates that scalable long-horizon generation does not require complex training pipelines or computationally intensive long-video training when data construction, backbone adaptation, and training stages are properly aligned. Future directions include extending to new data sources and backbones, exploring alternative recipe configurations, and leveraging the released infrastructure for embodied learning and interactive content creation applications.