SenseNova-U1.5: Towards Native Unified Visual Intelligence

Summary (Overview)

SenseNova-U1.5 is an 8B-MoT (Mixture-of-Transformers) native unified multimodal model that achieves visual understanding, reasoning, and generation within a single encoder-free and VAE-free architecture.
The model introduces a major architectural shift from independent patch prediction to spatially joint reconstruction, enabling native generation at resolutions up to 4K while maintaining compact visual sequences.
A multi-expert post-training pipeline is employed: four specialized experts (aesthetics, bilingual text rendering, infographic generation, and image editing) are trained with task-specific rewards, then consolidated via multi-expert on-policy distillation into a single unified policy.
Extensive evaluations show substantial improvements in image fidelity, bilingual text rendering, complex composition, multi-reference editing, and interleaved generation, while preserving or improving understanding benchmarks and language competence.
A key finding: despite limited exposure to structured formats during training, the model generalizes effectively to long, complex, and structured visual instructions, suggesting that multimodal understanding capabilities transfer naturally to visual creation.

Introduction and Theoretical Foundation

Background and Motivation

Traditional multimodal models typically rely on separate visual encoders (for perception) and diffusion-based decoders or VAEs (for generation), creating a fragmented pipeline where understanding and generation are disconnected. SenseNova-U1 [29] established the feasibility of a native unified paradigm through the encoder-free and VAE-free NEO-unify [101] architecture, bringing perception, reasoning, and pixel-space generation into a single end-to-end model.

Core Theoretical Basis

The central hypothesis is that a single native visual representation can serve as a shared substrate for understanding, reasoning, generation, and editing, eliminating the need for parallel visual pathways. The architecture builds on:

NEO [26] lightweight native visual interface: Directly transforms raw images or noise-corrupted visual inputs into compact token sequences without an external visual encoder or VAE.
Mixture-of-Transformers (MoT) design [29]: Integrates understanding and generation within a single Transformer backbone rather than separating them into two independent networks. Clean image-text context and noise-conditioned visual states are interleaved within a unified sequence, allowing semantic representations, visual evidence, and generative dynamics to interact directly through shared self-attention.

Methodology

Architecture

Visual Interface Refinements

Compared with SenseNova-U1, U1.5 refines the near-lossless visual interface on both encoding and decoding:

Resolution-aware noise conditioning is extended to the 4096 × 4096 range.
The original patch-wise MLP head is replaced with a spatially joint reconstruction head, which improves spatial coherence while preserving the efficiency of compact visual sequences (each 32 × 32-pixel region compressed into a single visual token).

Unified Sequence Design

Clean image-text context and noise-conditioned visual states are interleaved within a unified sequence, allowing semantic representations, visual evidence, and generative dynamics to interact directly through shared self-attention.

Training Stages

The model is trained progressively in three stages (summarized in Table 2 of the paper):

Stage 1: Generation Pre-training — Extends from low-resolution to high-resolution text-to-image data (512 × 512 up to 4096 × 4096), strengthening native 4K generation capability.
Stage 2: Unified Mid-training — Introduces image-editing and interleaved-generation tasks (185K additional steps). The training mixture comprises:
- 60% text-to-image
- 30% image-editing
- 10% interleaved image-text data
A mixed corpus is constructed with 30% text-only and multimodal-understanding data, 40% text-to-image data, 20% image-editing data, and 10% interleaved data.
Stage 3: Unified Supervised Fine-Tuning — Fine-tuning on curated high-quality instruction-following data with a task composition similar to Stage 2.

Data Curation

The training corpus substantially expands SenseNova-U1's dataset with emphasis on:

Quality-aware filtering and distribution rebalancing
Multi-granularity captions: from long detailed descriptions to concise captions and lightweight semantic tags
Bilingual (Chinese-English) coverage with explicit verification of language-text consistency
Structured prompt enhancement and chain-of-thought examples for challenging multi-target or multi-constraint edits

The editing corpus comprises:

∼43% general editing
∼42% spatially controlled infographic editing
∼15% reference-conditioned subset (supporting single- and multi-reference editing)

Infographic data contributes ∼29% of the corpus, emphasizing structured composition and dense textual content.

Post-Training: Multi-Expert Reinforcement Learning and On-Policy Distillation

Stage 4: Specialized Expert Training

Four experts are trained, each with task-specific data, rewards, sampling, and regularization:

Aesthetic Expert: Uses HPSv3++ [76] reward to evaluate perceptual quality and prompt-image alignment.
OCR/Text Rendering Expert: Uses a bilingual OCR reward based on PaddleOCR [32] to measure text fidelity.
Editing Expert: Uses a VLM-based reward framework evaluating five dimensions: instruction fulfillment, edit execution, overall visual quality, text-editing quality (when applicable), and preservation of unedited regions.
Infographic Expert: Undergoes a complete training process from infographic-oriented mid-training to task-specific RL, targeting small-text rendering and dense layout capabilities.

Training details: The first and final stages use group relative policy optimization (GRPO) with 16 candidates per prompt over 30-step trajectories using coefficients-preserving sampling (CPS) [121].

Stage 5: Multi-Expert On-Policy Distillation

Given a text-only or text-image condition $c$ from capability $m$ , the student first generates its own trajectory $\{ \hat { \mathbf { x } } _ { \theta , t } \} _ { t = 0 } ^ { 1 }$ . The distillation uses scale $s = 4$ for the aesthetic, OCR, and infographic experts (with global-norm clipping), and scale $s = 1$ for the editing expert.

Key design choices:

Aesthetic and OCR prompts use their respective rewards and are interleaved at the epoch level, avoiding the need to combine heterogeneous reward scales within each sample.
No additional infographic-specific reward model is introduced.
The text-editing criterion is omitted for instructions that do not involve text modification; the edit-execution score is set to zero when no requested change is visibly realized.

This design preserves task-dependent conditioning, guidance, and resolution policies of each expert while consolidating their specialized capabilities into a unified model.

Empirical Validation / Results

General Text-to-Image Generation

Evaluated on Qwen-Image-Bench [63], GenEval [42], GenEval2 [54], DPG-Bench [49], and OneIG-Bench [12], covering visual quality, prompt alignment, compositional generation, dense instruction following, and text rendering.

Key findings:

SenseNova-U1.5 consistently improves over SenseNova-U1 even without prompt enhancement.
Remains competitive with larger open-source baselines and narrows the gap with leading proprietary systems.
Shows improved compositional consistency and the ability to satisfy multiple visual constraints within a single prompt.

Complex Composition and Instruction Following

SenseNova-U1.5 demonstrates stronger compositional reasoning under complex constraints, particularly for:

Faithfully translating structured, relation-intensive prompts into coherent visual compositions
Handling multiple simultaneous constraints

The model exhibits a balanced generation profile across English and Chinese evaluations while maintaining competitive alignment, reasoning, and style scores despite its compact 8B scale.

Text Rendering

Under demanding text-centric settings (CVTG-2K [30] and LongText-Bench [41]):

Accuracy remains robust as text length and layout complexity increase
Demonstrates improved stability in dense, structured, and bilingual text rendering

Infographic and Commercial Content Generation

Evaluated on IGenBench [110] and BizGenEval [66], SenseNova-U1.5 substantially improves over SenseNova-U1 on both easy and hard splits. With prompt enhancement, it achieves leading open-source performance. Gains are particularly evident in semantic consistency — reliably identifying the intended edit, modifying correct visual content, and preserving semantic context.

Image Editing

Notable improvements in:

Style consistency and background preservation
Reference fidelity with strong subject and pose consistency
Better balance among reference utilization, transformation accuracy, and preservation of the original scene

The results suggest improved coordination between semantic editing and visual reconstruction.

Interleaved Generation and Reasoning

Table 21 (OpenING): SenseNova-U1.5 with CoT achieves leading overall performance, improving over SenseNova-U1 and outperforming compared proprietary pipelines. Strong results in image-text coherency, human alignment, and multi-step consistency demonstrate reliable open-ended interleaved generation.

Table 22: SenseNova-U1.5 achieves new state-of-the-art results across cognitive faculties, showing its ability to reason directly through interleaved text-image generation. This indicates that generation functions not only as an output modality but also as an intermediate representation supporting multimodal reasoning.

Understanding and Language Competence

Preserves or improves performance on most understanding benchmarks compared to SenseNova-U1.
Compares favorably with the encoder-free Gemma4-12B [113] across various visual understanding evaluations.
Retains strong language understanding and instruction-following performance, indicating that visual generation capabilities do not significantly dilute underlying language competence.

Theoretical and Practical Implications

Theoretical Significance

Unified Visual Representation: The results demonstrate that a single native visual representation can serve as a shared substrate for understanding, reasoning, generation, and editing, eliminating parallel visual pathways.
Transfer from Understanding to Generation: The most striking finding is that structural knowledge and planning capabilities acquired through multimodal understanding and reasoning transfer naturally to visual creation. Despite limited reliance on fixed generation templates during training, the model generalizes to long, compositional, and highly structured visual instructions.
Generation as Reasoning: Generation can function as an intermediate representation that supports multimodal reasoning and understanding, suggesting a deeper unification of cognitive faculties within a single computational substrate.

Practical Implications

Efficiency: The compact representation (one token per 32 × 32-pixel region) supports efficient inference while delivering strong performance across diverse tasks.
Scalability: The multi-expert RL + on-policy distillation pipeline provides a practical recipe for consolidating specialized capabilities into a unified model without forcing all tasks into a shared distillation recipe.
Open Source: The authors will release training code including supervised fine-tuning, reinforcement learning, and on-policy distillation, enabling reproducibility and further research.

Conclusion

SenseNova-U1.5 represents a significant step toward native multimodal systems in which visual understanding and generation are learned within a common computational substrate rather than connected through separate perceptual and generative pathways. The key contributions are:

Architectural innovation: Spatially joint reconstruction replacing independent patch prediction, enabling 4K native generation with compact visual sequences.
Training methodology: A comprehensive pipeline spanning generation pre-training, unified mid-training, supervised fine-tuning, multi-expert reinforcement learning, and on-policy distillation.
Empirical validation: Demonstrated state-of-the-art performance across image fidelity, bilingual typography, complex composition, multi-reference editing, interleaved generation, instruction following, visual preservation, and fine-grained control.

The results position native unified modelling as a promising path toward systems that perceive, reason, and create within a fully end-to-end framework — multimodal models that do not merely combine modalities but develop a unified mechanism for interpreting, imagining, and acting upon visual worlds.

Future directions implied by this work include: further scaling of native unified architectures, deeper exploration of the transfer between understanding and generation capabilities, and extension of the multi-expert distillation framework to additional modalities and task families.