Summary (Overview)

• Omni-WorldBench is introduced as the first benchmark specifically designed to evaluate the interactive response capabilities of world models in 4D settings (spatial structure + temporal evolution), addressing a critical gap in existing evaluation methodologies.
• The benchmark comprises two key components: Omni-WorldSuite (a systematic prompt suite spanning three hierarchical interaction levels and diverse scene types) and Omni-Metrics (an agent-based evaluation framework quantifying the causal impact of interaction actions on both final outcomes and intermediate state evolution).
• The benchmark evaluates 18 representative world models across multiple paradigms (Text-to-Video, Image-to-Video, and camera-conditioned generation), revealing that current models excel at visual fidelity but show significant limitations in causally grounded interaction dynamics.
• A novel AgenticScore aggregation mechanism uses MLLMs to adaptively weight three evaluation dimensions (Interaction Effect Fidelity, Generated Video Quality, Camera-Object Controllability) based on prompt semantics.
• The benchmark covers 1,068 evaluation prompts across general scenes (physics, optics, fluid dynamics, thermodynamics) and task-oriented scenarios (autonomous driving, embodied robotics, gaming), with the broadest capability coverage compared to existing benchmarks like VBench, WorldScore, and WorldModelBench.

Introduction and Theoretical Foundation

Background and Motivation

World models aim to characterize the temporal evolution of environmental states under given interaction conditions, providing foundations for counterfactual reasoning, planning, and decision-making. Two dominant paradigms have emerged:

1. Video generation - leveraging high-quality video representations to model world dynamics
2. 3D reconstruction - focusing on spatial geometry and viewpoint consistency

However, existing evaluation benchmarks suffer from critical limitations:

• Generative model benchmarks (e.g., VBench) focus narrowly on visual fidelity and text-video alignment
• 3D reconstruction metrics (e.g., WorldScore) fundamentally neglect temporal dynamics

The paper argues that the future of world modeling lies in 4D generation, which jointly models spatial structure and temporal evolution. In this paradigm, the core capability is interactive response: the ability to faithfully reflect how interaction actions drive state transitions across space and time.

Theoretical Foundation

The evaluation framework is built on a three-level interaction hierarchy that characterizes the scope of interaction effects:

• Level 1: Actions confined to the acting object, without altering other objects or the environment
• Level 2: Localized interactions where one object directly affects another
• Level 3: Complex interactions influencing multiple objects and leading to broader environmental changes

This hierarchy imposes progressively increasing representational and dynamic modeling requirements on world models.

Methodology

Omni-WorldSuite Construction

The prompt suite is constructed along two primary dimensions:

1. Scene Coverage: Both general daily-life scenarios and task-oriented environments (autonomous driving, embodied AI, gaming), covering physical laws, commonsense reasoning, causality, camera motion, closed-loop dynamics, and spatial constraints.

2. Interaction Hierarchy: Three levels as described above.

Construction Strategies

Dataset-grounded Prompt Generation (Fig. 2a):

• Extracts camera motion trajectories and first frames from open-source datasets (DriveLM for driving, InternData-A1 for robotics, Sekai for gaming)
• Uses Qwen-VL for caption generation with manual verification and refinement

Concept-driven Prompt Generation (Fig. 2b):

• Builds prototype concepts spanning scene domains, target objects, and actions
• Uses ChatGPT-5.2 for prompt generation, cross-checked by Gemini and DeepSeek-R1
• Employs FLUX.1-dev for image generation with CFG scale of 3.5 and 50 sampling steps
• All outputs undergo human verification and refinement

Omni-Metric Evaluation Framework

The evaluation protocol comprises three pivotal dimensions:

1. Generated Video Quality

Leverages established metrics including imaging quality, temporal flickering, motion smoothness, dynamic degree (from VBench), and content alignment (from WorldScore).

2. Camera-Object Controllability

Three independent metrics:

• Camera Control: Evaluates rotational and translational discrepancies in camera trajectories
• Object Control: Reframed as a direct VQA problem—a multimodal model answers whether each target object is present in uniformly sampled frames
• Transitions Detect: Uses PySceneDetect's ContentDetector to detect scene transitions:

$$s_{trans} = \begin{cases} 1, & N = 1 \\ 0, & N > 1 \end{cases} \tag{1}$$

3. Interaction Effect Fidelity

Four comprehensive metrics:

InterStab-L (long-horizon temporal coherence):

$$s(i,j) = \frac{1}{2}\left(\text{SSIM}_{gray}(I_i, I_j) + \cos(\phi(I_i), \phi(I_j))\right) \tag{2}$$ $$\text{InterStab-L} = \frac{1}{|\mathcal{R}|} \sum_{(t_a,t_b) \in \mathcal{R}} s(i(t_a), i(t_b)) \cdot \mathbb{I}_{dynamic} \tag{3}$$

where SSIM is the Structural Similarity Index, $\phi(\cdot)$ is a pre-trained vision encoder (CLIP), and $\mathbb{I}_{dynamic}$ is a dynamics gating indicator that penalizes trivial static sequences.

InterStab-N (stability of non-target regions):

$$E_{non}(s) = \frac{1}{T} \sum_{t=1}^{T} \frac{1}{|\mathcal{N}|} \sum_{x \in \mathcal{N}} \| \text{Flow}_t(x) \| \tag{4}$$ $$\text{InterStab-N}(s) = \exp\left(-\frac{E_{non}(s)}{\beta \times \min(H,W)}\right) \tag{5}$$

where $\text{Flow}_t(x)$ is the optical flow vector at location $x$ in frame $t$, and $\beta$ is a scaling factor.

InterCov (object-level causal faithfulness):

$$\text{InterCov} = \frac{1}{|\mathcal{O}|} \sum_{o \in \mathcal{O}} \mathbb{I}(v_o = 1) \tag{6}$$

where $v_o \in \{0,1\}$ is a VLM-based binary validity signal indicating whether entity behavior aligns with prescribed interaction logic.

InterOrder (temporal event ordering):

$$\text{InterOrder} = \frac{2K_s}{K(K-1)} \tag{7}$$

where $K_s$ is the number of consistent event pairs out of all possible pairs.

AgenticScore Aggregation

The final score adaptively weights the three evaluation dimensions using an MLLM conditioned on the evaluation prompt:

$$\text{AgenticScore} = w_1 A_I + w_2 A_G + w_3 A_C \tag{8}$$

where $A_I$, $A_G$, and $A_C$ are the interaction effect fidelity, generated video quality, and camera-object controllability scores respectively, and $w_1, w_2, w_3$ are prompt-dependent weights.

Empirical Validation / Results

Evaluated Models

A total of 18 representative world models were evaluated across three paradigms:

Key Results (Table 1 Summary)

Major Findings

1. IT2V paradigm leads overall: Image-to-Video models with richer conditional inputs demonstrate the highest performance potential. Wan2.2 achieves the highest AgenticScore (75.92%), closely followed by Cosmos (75.42%).

2. Interaction fidelity is the weakest dimension: Even the best models score only ~67% on interaction effect fidelity, revealing substantial room for improvement in causally grounded dynamics.

3. Trade-offs in camera-conditioned models: WonderWorld scores 84.96% on InterStab-L but drops sharply to 24.89% on InterStab-N, showing that maintaining interaction logic while introducing complex camera scheduling remains challenging.

4. Visual quality is largely saturated: Most models exceed 95% on temporal flickering and motion smoothness. The key differentiators are now Dynamic Degree and content alignment.

5. Qualitative analysis confirms quantitative findings: Wan2.2 successfully synthesizes anatomically reasonable pitching motions, while Matrix-Game2.0 suffers from catastrophic temporal degradation including complete disappearance of the human figure.

Theoretical and Practical Implications

Theoretical Significance

• Paradigm shift in evaluation: The paper argues for moving beyond passive video quality assessment toward agent-centric interactive generation evaluation, establishing interaction as the core capability of world modeling.
• Hierarchical interaction framework: The three-level interaction taxonomy provides a principled way to characterize the scope and complexity of interaction effects, from object-confined actions to global environmental changes.
• Causal evaluation metrics: The introduction of InterCov and InterOrder provides quantitative tools for assessing causal faithfulness and temporal event logic, moving beyond purely appearance-based metrics.

Practical Implications

• Benchmark coverage: Omni-WorldBench supports the most comprehensive set of prompt modalities (text, image, trajectory) and evaluates both task-oriented and general scenes, providing a reliable testbed for next-generation 4D world models.
• Diagnostic value: The benchmark reveals that current models are strong in conventional video quality but show clear limitations in action-conditioned world evolution, causal interaction consistency, and joint camera-object control.
• Actionable insights: Results highlight specific areas for improvement, including maintaining interaction logic under complex camera scheduling and improving dynamic responsiveness.

Conclusion

Main Takeaways

Omni-WorldBench represents the first comprehensive benchmark dedicated to evaluating the interactive response capabilities of video world models. The systematic evaluation of 18 models reveals substantial gaps between visual realism and true interactivity: although many models achieve strong visual fidelity and motion smoothness, their ability to maintain causally grounded interaction dynamics remains limited.

The benchmark's two components work synergistically:

• Omni-WorldSuite provides a diverse, hierarchical prompt suite spanning physical principles, task-oriented scenarios, and interaction levels
• Omni-Metric offers quantitative measurement of action impacts on both final outcomes and intermediate state transitions

Future Directions

• Expansion of coverage: The current suite cannot fully capture the complexity of open-world interactive environments, especially long-horizon and highly dynamic settings.
• Human-aligned evaluation: The authors plan to release human-aligned evaluation results to further complement and validate the assessment of interaction quality.
• Community refinement: Omni-WorldBench is designed to be continuously refined and extended through community feedback, serving as a standardized testbed for diagnosing current limitations and advancing research on more interactive and causally consistent world models.

The benchmark will be publicly released to foster progress in interactive 4D world modeling.