# SenseNova-U1.5: Towards Native Unified Visual Intelligence

> SenseNova-U1.5 is an 8B native unified multimodal model achieving state-of-the-art visual understanding and 4K generation via spatially joint reconstruction and multi-expert on-policy distillation.

- **Source:** [arXiv](https://arxiv.org/abs/2609.11929)
- **Published:** 2026-09-12
- **Permalink:** https://picx.dev/p/P4eOe4
- **Whiteboard:** https://picx.dev/p/P4eOe4/image

## Summary

# SenseNova-U1.5: Towards Native Unified Visual Intelligence

## Summary (Overview)

- **SenseNova-U1.5** is an 8B-MoT (Mixture-of-Transformers) native unified multimodal model that achieves visual understanding, reasoning, and generation within a single encoder-free and VAE-free architecture.
- The model introduces a **major architectural shift** from independent patch prediction to **spatially joint reconstruction**, enabling native generation at resolutions up to 4K while maintaining compact visual sequences.
- A **multi-expert post-training pipeline** is employed: four specialized experts (aesthetics, bilingual text rendering, infographic generation, and image editing) are trained with task-specific rewards, then consolidated via **multi-expert on-policy distillation** into a single unified policy.
- Extensive evaluations show substantial improvements in image fidelity, bilingual text rendering, complex composition, multi-reference editing, and interleaved generation, while preserving or improving understanding benchmarks and language competence.
- A key finding: despite limited exposure to structured formats during training, the model **generalizes effectively to long, complex, and structured visual instructions**, suggesting that multimodal understanding capabilities transfer naturally to visual creation.

---

## Introduction and Theoretical Foundation

### Background and Motivation

Traditional multimodal models typically rely on separate visual encoders (for perception) and diffusion-based decoders or VAEs (for generation), creating a fragmented pipeline where understanding and generation are disconnected. SenseNova-U1 [29] established the feasibility of a **native unified paradigm** through the encoder-free and VAE-free NEO-unify [101] architecture, bringing perception, reasoning, and pixel-space generation into a single end-to-end model.

### Core Theoretical Basis

The central hypothesis is that a **single native visual representation** can serve as a shared substrate for understanding, reasoning, generation, and editing, eliminating the need for parallel visual pathways. The architecture builds on:

- **NEO [26] lightweight native visual interface**: Directly transforms raw images or noise-corrupted visual inputs into compact token sequences without an external visual encoder or VAE.
- **Mixture-of-Transformers (MoT) design [29]**: Integrates understanding and generation within a single Transformer backbone rather than separating them into two independent networks. Clean image-text context and noise-conditioned visual states are interleaved within a unified sequence, allowing semantic representations, visual evidence, and generative dynamics to interact directly through shared self-attention.

### Related Work on Post-Training

The paper builds on recent advances in:
- **Visual reinforcement learning** with capability-specific rewards: preference models for perceptual quality and text-image alignment [76, 86, 133, 140], and specialized rewards for typography and editing [47, 52].
- **On-policy distillation (OPD) [1, 81]**: Mitigates the distribution mismatch of conventional knowledge distillation by training the student on its own generated trajectories while receiving dense supervision from a teacher. The authors extend this by retaining four task-specialized experts and hard-routing their supervision into a single native pixel-space model.

---

## Methodology

### Architecture

#### Visual Interface Refinements

Compared with SenseNova-U1, U1.5 refines the near-lossless visual interface on both encoding and decoding:
- **Resolution-aware noise conditioning** is extended to the 4096 × 4096 range.
- The original patch-wise MLP head is replaced with a **spatially joint reconstruction head**, which improves spatial coherence while preserving the efficiency of compact visual sequences (each 32 × 32-pixel region compressed into a single visual token).

#### Unified Sequence Design

Clean image-text context and noise-conditioned visual states are interleaved within a unified sequence, allowing semantic representations, visual evidence, and generative dynamics to interact directly through shared self-attention.

### Training Stages

The model is trained progressively in three stages (summarized in Table 2 of the paper):

1. **Stage 1: Generation Pre-training** — Extends from low-resolution to high-resolution text-to-image data (512 × 512 up to 4096 × 4096), strengthening native 4K generation capability.
2. **Stage 2: Unified Mid-training** — Introduces image-editing and interleaved-generation tasks (185K additional steps). The training mixture comprises:
   - 60% text-to-image
   - 30% image-editing
   - 10% interleaved image-text data

   A mixed corpus is constructed with 30% text-only and multimodal-understanding data, 40% text-to-image data, 20% image-editing data, and 10% interleaved data.
3. **Stage 3: Unified Supervised Fine-Tuning** — Fine-tuning on curated high-quality instruction-following data with a task composition similar to Stage 2.

### Data Curation

The training corpus substantially expands SenseNova-U1's dataset with emphasis on:
- **Quality-aware filtering** and distribution rebalancing
- **Multi-granularity captions**: from long detailed descriptions to concise captions and lightweight semantic tags
- **Bilingual (Chinese-English) coverage** with explicit verification of language-text consistency
- **Structured prompt enhancement and chain-of-thought examples** for challenging multi-target or multi-constraint edits

The editing corpus comprises:
- ∼43% general editing
- ∼42% spatially controlled infographic editing
- ∼15% reference-conditioned subset (supporting single- and multi-reference editing)

Infographic data contributes ∼29% of the corpus, emphasizing structured composition and dense textual content.

### Post-Training: Multi-Expert Reinforcement Learning and On-Policy Distillation

#### Stage 4: Specialized Expert Training

Four experts are trained, each with task-specific data, rewards, sampling, and regularization:

1. **Aesthetic Expert**: Uses HPSv3++ [76] reward to evaluate perceptual quality and prompt-image alignment.
2. **OCR/Text Rendering Expert**: Uses a bilingual OCR reward based on PaddleOCR [32] to measure text fidelity.
3. **Editing Expert**: Uses a VLM-based reward framework evaluating five dimensions: instruction fulfillment, edit execution, overall visual quality, text-editing quality (when applicable), and preservation of unedited regions.
4. **Infographic Expert**: Undergoes a complete training process from infographic-oriented mid-training to task-specific RL, targeting small-text rendering and dense layout capabilities.

**Training details**: The first and final stages use **group relative policy optimization (GRPO)** with 16 candidates per prompt over 30-step trajectories using **coefficients-preserving sampling (CPS)** [121].

#### Stage 5: Multi-Expert On-Policy Distillation

Given a text-only or text-image condition $c$ from capability $m$, the student first generates its own trajectory $\{ \hat { \mathbf { x } } _ { \theta , t } \} _ { t = 0 } ^ { 1 }$. The distillation uses scale $s = 4$ for the aesthetic, OCR, and infographic experts (with global-norm clipping), and scale $s = 1$ for the editing expert.

Key design choices:
- Aesthetic and OCR prompts use their respective rewards and are **interleaved at the epoch level**, avoiding the need to combine heterogeneous reward scales within each sample.
- No additional infographic-specific reward model is introduced.
- The text-editing criterion is omitted for instructions that do not involve text modification; the edit-execution score is set to zero when no requested change is visibly realized.

This design preserves task-dependent conditioning, guidance, and resolution policies of each expert while consolidating their specialized capabilities into a unified model.

---

## Empirical Validation / Results

### General Text-to-Image Generation

Evaluated on Qwen-Image-Bench [63], GenEval [42], GenEval2 [54], DPG-Bench [49], and OneIG-Bench [12], covering visual quality, prompt alignment, compositional generation, dense instruction following, and text rendering.

Key findings:
- SenseNova-U1.5 **consistently improves over SenseNova-U1** even without prompt enhancement.
- Remains **competitive with larger open-source baselines** and narrows the gap with leading proprietary systems.
- Shows improved **compositional consistency** and the ability to satisfy multiple visual constraints within a single prompt.

### Complex Composition and Instruction Following

SenseNova-U1.5 demonstrates stronger compositional reasoning under complex constraints, particularly for:
- Faithfully translating **structured, relation-intensive prompts** into coherent visual compositions
- Handling **multiple simultaneous constraints**

The model exhibits a **balanced generation profile** across English and Chinese evaluations while maintaining competitive alignment, reasoning, and style scores despite its compact 8B scale.

### Text Rendering

Under demanding text-centric settings (CVTG-2K [30] and LongText-Bench [41]):
- Accuracy remains robust as **text length and layout complexity increase**
- Demonstrates improved stability in **dense, structured, and bilingual text rendering**

### Infographic and Commercial Content Generation

Evaluated on IGenBench [110] and BizGenEval [66], SenseNova-U1.5 substantially improves over SenseNova-U1 on both easy and hard splits. With prompt enhancement, it achieves **leading open-source performance**. Gains are particularly evident in **semantic consistency** — reliably identifying the intended edit, modifying correct visual content, and preserving semantic context.

### Image Editing

Notable improvements in:
- **Style consistency and background preservation**
- **Reference fidelity** with strong subject and pose consistency
- Better balance among **reference utilization, transformation accuracy, and preservation of the original scene**

The results suggest improved coordination between semantic editing and visual reconstruction.

### Interleaved Generation and Reasoning

**Table 21 (OpenING)**: SenseNova-U1.5 with CoT achieves leading overall performance, improving over SenseNova-U1 and outperforming compared proprietary pipelines. Strong results in image-text coherency, human alignment, and multi-step consistency demonstrate reliable open-ended interleaved generation.

**Table 22**: SenseNova-U1.5 achieves **new state-of-the-art results across cognitive faculties**, showing its ability to reason directly through interleaved text-image generation. This indicates that generation functions not only as an output modality but also as an **intermediate representation supporting multimodal reasoning**.

### Understanding and Language Competence

- Preserves or improves performance on most understanding benchmarks compared to SenseNova-U1.
- Compares favorably with the encoder-free **Gemma4-12B [113]** across various visual understanding evaluations.
- Retains strong **language understanding and instruction-following** performance, indicating that visual generation capabilities do not significantly dilute underlying language competence.

---

## Theoretical and Practical Implications

### Theoretical Significance

1. **Unified Visual Representation**: The results demonstrate that a single native visual representation can serve as a shared substrate for understanding, reasoning, generation, and editing, eliminating parallel visual pathways.

2. **Transfer from Understanding to Generation**: The most striking finding is that structural knowledge and planning capabilities acquired through multimodal understanding and reasoning **transfer naturally to visual creation**. Despite limited reliance on fixed generation templates during training, the model generalizes to long, compositional, and highly structured visual instructions.

3. **Generation as Reasoning**: Generation can function as an **intermediate representation** that supports multimodal reasoning and understanding, suggesting a deeper unification of cognitive faculties within a single computational substrate.

### Practical Implications

1. **Efficiency**: The compact representation (one token per 32 × 32-pixel region) supports efficient inference while delivering strong performance across diverse tasks.

2. **Scalability**: The multi-expert RL + on-policy distillation pipeline provides a practical recipe for consolidating specialized capabilities into a unified model without forcing all tasks into a shared distillation recipe.

3. **Open Source**: The authors will release training code including supervised fine-tuning, reinforcement learning, and on-policy distillation, enabling reproducibility and further research.

---

## Conclusion

SenseNova-U1.5 represents a significant step toward **native multimodal systems** in which visual understanding and generation are learned within a common computational substrate rather than connected through separate perceptual and generative pathways. The key contributions are:

1. **Architectural innovation**: Spatially joint reconstruction replacing independent patch prediction, enabling 4K native generation with compact visual sequences.

2. **Training methodology**: A comprehensive pipeline spanning generation pre-training, unified mid-training, supervised fine-tuning, multi-expert reinforcement learning, and on-policy distillation.

3. **Empirical validation**: Demonstrated state-of-the-art performance across image fidelity, bilingual typography, complex composition, multi-reference editing, interleaved generation, instruction following, visual preservation, and fine-grained control.

The results position native unified modelling as a **promising path toward systems that perceive, reason, and create within a fully end-to-end framework** — multimodal models that do not merely combine modalities but develop a unified mechanism for interpreting, imagining, and acting upon visual worlds.

**Future directions** implied by this work include: further scaling of native unified architectures, deeper exploration of the transfer between understanding and generation capabilities, and extension of the multi-expert distillation framework to additional modalities and task families.

---

_Markdown view of https://picx.dev/p/P4eOe4, served by PicX — AI-generated visual whiteboard summaries of research papers._