# LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Model > LLaDA2.0-Uni is a unified discrete diffusion model that uses semantic tokens to perform both multimodal understanding and generation within a single framework. - **Source:** [arXiv](https://arxiv.org/abs/2604.20796) - **Published:** 2026-04-24 - **Permalink:** https://picx.dev/p/D5leQk - **Whiteboard:** https://picx.dev/p/D5leQk/image ## Summary # LLaDA2.0-Uni: A Comprehensive Summary ## Summary (Overview) * **Unified Architecture for Multimodal Tasks:** LLaDA2.0-Uni is a novel unified discrete diffusion large language model (dLLM) that natively supports both multimodal understanding (e.g., VQA, document reasoning) and generation (e.g., image generation, editing) within a single framework. * **Core Technical Innovations:** The architecture integrates three key components: 1) a **SigLIP-VQ semantic tokenizer** for discretizing images, 2) a **16B MoE-based dLLM backbone** trained with a block-level masked diffusion objective for unified sequence modeling, and 3) a **distillation-optimized diffusion decoder** for high-fidelity image reconstruction. * **Strong Benchmark Performance:** The model achieves performance on par with specialized vision-language models (VLMs) in understanding tasks and competitive results with state-of-the-art image generation models, while also excelling in editing and complex interleaved generation/reasoning tasks. * **Efficient Inference:** The model employs training-free inference acceleration via **SPRINT** (Sparse Prefix Retention with Inference-time Non-uniform Token Unmasking) and a distilled diffusion decoder, achieving up to 1.6× speedup and 8-step CFG-free image generation. * **Scalable Data and Training Pipeline:** The model is supported by a large-scale, carefully curated multimodal dataset and a tailored three-stage training pipeline (vision-language alignment, multi-task pre-training, supervised fine-tuning) that progressively builds its capabilities. ## Introduction and Theoretical Foundation Large language models (LLMs) have expanded beyond text to handle diverse multimodal tasks, primarily categorized into **understanding** (e.g., visual question answering) and **generation** (e.g., text-to-image). Traditionally, these are handled by separate specialized models. A unified model offers key benefits: mutual enhancement between understanding and generation, improved deployment efficiency, and unlocking advanced capabilities like **interleaved generation and reasoning**, moving closer to Artificial General Intelligence (AGI). Current unified models are predominantly based on **autoregressive (AR)** architectures, which tokenize images into discrete sequences for next-token prediction. An alternative paradigm is offered by **masked diffusion models**, which have inherent advantages in parallel decoding and bidirectional context modeling. However, existing unified masked diffusion models (e.g., MMaDA, Lumina-DiMOO) lag behind AR-based models due to architectural limitations: 1) reconstructive VQ tokenizers lack semantic information, harming understanding; 2) excessive image compression degrades generation quality; 3) fully bidirectional modeling is unreliable for text; 4) they assume fixed output lengths. **LLaDA2.0-Uni** addresses these limitations by proposing a unified dLLM-based MoE model. Its core innovation is using **fully discrete semantic tokens** for both understanding and generation. This is achieved via a **SigLIP-VQ tokenizer**, which converts images into semantic tokens, preserving crucial details for complex reasoning. This unified representation allows both text and images to be optimized under a shared **block-level masked diffusion objective** within the dLLM backbone, while a dedicated diffusion decoder reconstructs images from the generated tokens. ## Methodology ### 2.1 Architecture Overview LLaDA2.0-Uni consists of three core components: 1. **Semantic Discrete Tokenizer (SigLIP-VQ):** Converts continuous images into discrete semantic tokens. It uses a pre-trained SigLIP2-g ViT as a feature extractor and a vector quantizer with a codebook (vocab size 16,384, dimensionality 2,048). Unlike reconstruction-based VQ-VAEs, it is trained on understanding tasks, preserving rich semantics. 2. **Diffusion Large Language Model (16B MoE Backbone):** Built upon LLaDA-2.0-mini, it processes interleaved sequences of text and visual tokens. Key design choices: * **Block-wise Attention:** Adopted instead of full bidirectional attention to balance parallel decoding speed with training stability, especially for semantically aligned SigLIP-VQ tokens. * **1D RoPE with Size Tokens:** Uses standard 1D Rotary Position Embedding (RoPE). Special `` and `` tokens (e.g., ``) are prepended to the flattened visual sequence to represent 2D spatial information and enable arbitrary resolution handling. 3. **Diffusion Decoder:** A model built upon Z-Image-Base (6B) that reconstructs high-fidelity images from the semantic tokens generated by the dLLM backbone. It performs 2× super-resolution and is optimized via **model distillation** for efficient 8-step CFG-free inference. ### 2.2 Training-free Inference Acceleration: SPRINT To accelerate inference beyond parallel decoding, the paper proposes **SPRINT**, which reduces cost along two axes: 1. **Sparse Prefix Retention:** Prunes the prefix Key-Value (KV) cache in a modality-aware manner to lower per-step attention cost. A composite importance score $s_i$ for each prefix position $i$ is calculated: $$ s_i = \alpha \cdot \bar{I}_i + (1 - \alpha) \cdot c_i $$ where $\bar{I}_i = \| k_i \|_2 / \left( \frac{1}{L} \sum_{j=1}^{L} \| k_j \|_2 \right)$ is the mean-normalized key norm, $c_i = \max_v p_\theta(v | x_t)$ is the top-1 softmax confidence, and $\alpha=0.5$. Separate keep ratios are used for text ($r_{text}$) and image ($r_{img}$) tokens. 2. **Non-uniform Token Unmasking:** Replaces the fixed denoising schedule with a confidence-adaptive strategy. At each step, all masked positions with confidence exceeding a threshold $\tau$ are accepted: $$ A = \{ n \in [m] : c_n > \tau \}. $$ A minimum number of acceptances is enforced to guarantee termination. ### 3. Data Preparation A large-scale, meticulously curated dataset is constructed: * **Multimodal Understanding:** Includes image-caption data, OCR (via a coarse-to-fine pipeline), grounding/counting data, world knowledge/reasoning data, and high-quality text data. * **Image Generation:** Over 200M web images filtered for resolution, aesthetics (ArtiMuse score >60), and quality (DeQA-Score >4.0). Captions are generated/enhanced by Qwen3-VL. * **Image Editing:** Combines open-source datasets (X2Edit, OmniEdit, etc.) and synthesized pairs. Instructions are refined by Qwen3-VL for accuracy. * **Interleaved Data:** Constructed from the Koala36M video corpus with strict filtering for duration, quality, and motion. Frame sequences (2-6 frames) are captioned to provide instruction-following data. * **Reasoning-Augmented Data:** Sourced from Flux-6M, Zebra-CoT, and Weave (~8M samples) to enable chain-of-thought reasoning. ### 4. Model Training A three-stage training pipeline is employed: **Stage 0: Vision-Language Alignment:** Aligns visual and linguistic representations using image-caption pairs. A random masking strategy is applied (image tokens for generation, text tokens for understanding). Progressive resolution from 256×256 to 512×512. **Stage 1: Multi-task Pre-training:** Trains on diverse multimodal data (understanding: interleaved, OCR, grounding; generation: editing, controllable generation, style transfer) to develop comprehensive capabilities. **Stage 2: Supervised Fine-Tuning (SFT):** Conducted in two phases (8k then 16k context) on high-quality instruction-following data for complex reasoning and generation. #### 4.2 Pre-Training Optimization * **BDLM Loss:** The training uses the Block Diffusion Language Model (BDLM) objective, which operates on block-level masked regions: $$ \mathcal{L}_{\text{BDLM}}(\theta) = -\mathbb{E}_{t, x_0, x_t} \left[ \frac{\alpha'_t}{1 - \alpha_t} \sum_{k=1}^{K} \sum_{i=1}^{L_B} \mathbb{1}[x^i_{t,k} = \text{[MASK]}] \log p_\theta(x^i_{0,k} | x_{0,