# Xiaomi-OCR-0 Technical Report

> Xiaomi-OCR-0, a 0.8B VLM, achieves state-of-the-art document parsing and strong OCR understanding by showing understanding supervision shifts from harmful to beneficial as parsing matures.

- **Source:** [arXiv](https://arxiv.org/abs/2609.36136)
- **Published:** 2026-10-24
- **Permalink:** https://picx.dev/p/qXuXtp
- **Whiteboard:** https://picx.dev/p/qXuXtp/image

## Summary

## Summary (Overview)

- **Xiaomi-OCR-0** is a unified 0.8B-parameter vision-language model (VLM) that jointly handles document parsing and OCR-centric understanding (OCR-VQA and key information extraction), initialized from Qwen3.5-0.8B.
- The model is trained on an approximately **170M-sample OCR-centric corpus** built via an automated data engine combining expert consensus (Ensemble Triplet Consensus), render-based verification, multi-factor sample mining, and targeted synthetic data generation.
- A progressive training recipe is introduced: **Q-Mask-based text anchoring** → **continued pretraining (CPT)** → **mixed-task reinforcement learning (Mix-RL)**.
- Xiaomi-OCR-0 achieves state-of-the-art results: **95.24** on Real5-OmniDocBench (highest among compared methods), **96.83** on OmniDocBench v1.6, **87.94** on Wild-OmniDocBench, and an average of **83.2** across five OCR-VQA benchmarks.
- A key finding: **understanding-to-parsing transfer is capability-dependent** — understanding supervision *hurts* parsing at early training stages but *benefits* it once the parsing foundation matures.

## Introduction and Theoretical Foundation

Modern OCR systems must support two complementary capabilities:

1. **Document parsing** — faithful, exhaustive reconstruction of a document's content and structure (text, tables, formulas, reading order).
2. **OCR-centric understanding** — selective, task-conditioned interpretation of task-relevant visual text, encompassing OCR-oriented visual question answering (OCR-VQA) and key information extraction (KIE).

The paper identifies two central challenges in jointly developing these capabilities:

- **Scalable reliable supervision**: Text, tables, formulas, and reading order require different verification procedures; plausible model-generated annotations can conceal subtle errors.
- **Divergent objectives**: Parsing requires exhaustive reconstruction, while understanding requires selective, task-conditioned responses. Simply mixing training data does not guarantee positive transfer.

The authors' experiments reveal that the interaction between parsing and understanding **depends on the maturity of the parsing capability**:

> "When parsing capability is still immature, introducing understanding supervision too early can interfere with developing a strong parsing foundation, resulting in negative transfer. As parsing capability matures, however, the same understanding supervision becomes beneficial, shifting from negative to positive transfer."

The model builds on **Qwen3.5-0.8B** [55] and uses **PP-DocLayoutV3** [14] as an external layout detector for two-stage inference.

## Methodology

### 1. Data Engine (~170M samples)

**Automated Annotation Pipeline** (Figure 2 in paper):

- **Ensemble Triplet Consensus (ETC)**: Eight heterogeneous OCR experts ($\mathcal{M} = \{M_1, \ldots, M_N\}$) produce predictions for each region. Pairwise agreement is measured with region-specific similarity $S(y_i, y_j) \in [0,1]$: 1−NED for text, TEDS for tables, CDM for formulas.

The consensus score for each expert is:

$$
C_i = \frac{1}{N-1} \sum_{j \neq i} S(y_i, y_j). \tag{1}
$$

The top-3 experts form the consensus triplet $\mathcal{T}^\star$, and the region's agreement score is:

$$
a(x) = \frac{1}{3}\sum_{\substack{i,j\in \mathcal{T}^{\star}\\ i < j}}S(y_{i},y_{j}). \tag{2}
$$

The pseudo-label is the **triplet medoid**:

$$
y^{\star}(x) = y_{i^{\star}}, \qquad i^{\star} = \arg \max_{i \in \mathcal{T}^{\star}} \sum_{j \in \mathcal{T}^{\star} \setminus \{i\}} S(y_i, y_j). \tag{3}
$$

Regions are tiered by agreement:

$$
d_{\text{agree}}(x) = \left\{ \begin{array}{ll} \text{high}, & a(x) \geq 0.9, \\ \text{medium}, & 0.6 \leq a(x) < 0.9, \\ \text{low}, & a(x) < 0.6. \end{array} \right. \tag{4}
$$

- **Render-Guided Refine-and-Judge**: Medium/low-agreement regions are refined by Qwen3.5-122B-A10B via iterative render-based verification (up to 3 rounds), followed by history-aware decoupled refinement (Critique–Refine–Verify) and re-consensus.

**Multi-Factor Sample Mining**: Combines expert agreement $a_i$, model self-consistency $u_i$, and semantic clusters (via K-Means on Qwen3-VL embeddings) to construct task-specific mixtures. Cluster-based sampling uses quotas:

$$
q_k = \left\{ \begin{array}{ll} \max(1, \lfloor \rho \cdot |\mathcal{C}_k| \rfloor), & |\mathcal{C}_k| < n/K, \\ \min(|\mathcal{C}_k|, \max(1, \text{round}(n \cdot |\mathcal{C}_k| / N_{\text{data}}))), & |\mathcal{C}_k| \geq n/K, \end{array} \right. \tag{9}
$$

with within-cluster sampling probability:

$$
P(i \mid k) = \frac{(a_i + \varepsilon)^{f_a/\tau}(1 - u_i + \varepsilon)^{f_u/\tau}}{\sum_{j: x_j \in \mathcal{C}_k}(a_j + \varepsilon)^{f_a/\tau}(1 - u_j + \varepsilon)^{f_u/\tau}}. \tag{10}
$$

**Synthetic Data Generation**: Two modes — **coverage-driven** (fills distributional gaps in structures, styles, low-resource scripts) and **failure-driven** (generates targeted variants around diagnosed model errors).

### 2. Progressive Training Recipe

**Stage 1 — Text-Anchoring Pretraining (Q-Mask)** : Uses the Spatial Supervision Alignment (SSA) objective:

$$
\mathcal{L}_{\mathrm{SSA}} = \lambda_{\mathrm{txt}} \mathcal{L}_{\mathrm{NTP}} + \lambda_{\mathrm{seg}}(\mathcal{L}_{\mathrm{Dice}} + \mathcal{L}_{\mathrm{CE}}). \tag{12}
$$

**Stage 2 — Continued Pretraining (CPT)** : Standard next-token prediction plus retained MTP head:

$$
\mathcal{L}_{\mathrm{CPT}}(\theta) = -\mathbb{E}_{(I, c, y) \sim \mathcal{D}}\left[\sum_{t=1}^{|y|} \log \pi_{\theta}(y_t \mid y_{<t}, I, c)\right] + \lambda_{\mathrm{MTP}} \mathcal{L}_{\mathrm{MTP}}(\theta). \tag{13}
$$

**Stage 3 — Mixed-Task Reinforcement Learning (Mix-RL)** : DAPO-style GRPO objective with token-level aggregation, clip-higher, and dynamic sampling:

$$
\mathcal{J}_{\mathrm{GRPO}}(\theta) = \mathbb{E}_{x \sim \mathcal{D}, \{o_i\}_{i=1}^G \sim \pi_{\mathrm{old}}}\left[ \frac{1}{\sum_{i=1}^G |o_i|} \sum_{i=1}^G \sum_{t=1}^{|o_i|} \min\left(r_{i,t}(\theta)\hat{A}_i, \operatorname{clip}(r_{i,t}(\theta), 1-\varepsilon_{\text{low}}, 1+\varepsilon_{\text{high}})\hat{A}_i\right) \right]. \tag{14}
$$

The document-parsing reward is a weighted combination:

$$
R_{\mathrm{doc}} = w_{\mathrm{text}} s_{\mathrm{text}} + w_{\mathrm{table}} s_{\mathrm{table}} + w_{\mathrm{formula}} s_{\mathrm{formula}}, \tag{15}
$$

where weights reflect ground-truth modality character coverage. Rewards use text edit similarity, TEDS, and CDM; KIE uses JSON validity + field matching; OCR-VQA uses ANLS.

## Empirical Validation / Results

### Document Parsing

**OmniDocBench v1.6** (Table 1): Xiaomi-OCR-0 achieves **96.83 Overall**, close to TeleOCR's 96.87 (1.2B) and above same-size OvisOCR2 (96.58). Formula CDM of **98.50** is the highest in the comparison.

**Real5-OmniDocBench** (Table 2): **95.24 Overall**, leading all compared methods across all five acquisition conditions (scanning, warping, screen-photo, illumination, skew). The gain over PaddleOCR-VL-1.6 is 2.05 points overall, rising to 2.98 on warped pages.

**Wild-OmniDocBench** (Table 3): **87.94 Overall**, trailing TeleOCR's 88.53 but with the lowest reading-order edit distance (0.1931).

**Element-Level**: PubTabNet TEDS of **92.27** (highest in Table 4); UniMER-Test Overall of **97.84** with SPE of 99.460 (highest in Table 5).

**Long-Tail (Chronicles-OCR)**: Mature Average of **0.73**, exceeding Seed2.0 Pro's 0.72 (Table 7).

### OCR-Centric Understanding

**OCR-VQA** (Table 6): Mean of **83.2** across DocVQA, InfoVQA, ChartQA, OCRBench, and TextVQA — a gain of 10.3 points over Qwen3.5-0.8B and 6.7 over HunyuanOCR-1.5. Largest gains are on InfoVQA (+14.8) and ChartQA (+15.1). The 0.8B model exceeds Qwen3.5-2B by 2.3 points and trails Qwen3.5-4B by only 1.7 points.

**KIE** (Tables 8–9): **59.09%** overall accuracy, exceeding Qwen3.5-2B by 7.49 points. Gains are concentrated in flight-information (73.00% vs. 43.00%) and document-field extraction (77.89% vs. 63.15%). Address extraction remains difficult (45.93%); invoices trail Qwen3.5-2B (65.00% vs. 77.00%).

### Key Ablations

**Transfer from Understanding to Parsing** (Table 10): Understanding supervision *hurts* at the early stage (−0.330 Overall) but *benefits* at the middle (+0.639) and late stages. Page-level supervision also improves the parsing foundation.

**Q-Mask and Mix-RL** (Table 11): Q-Mask adds +0.2957 Overall (largest gain in Formula CDM, +0.4931); Mix-RL adds +0.3538 Overall (largest gains in Table TEDS, +0.7114).

**Model Capacity** (Table 12): Scaling 0.8B → 4B yields ~0.15 point gain on parsing but ~4.9 points on understanding, indicating understanding places greater demands on model capacity.

**MOPD vs. Mix-RL**: MOPD reaches competitive parsing scores (96.7417) at lower compute; longer Mix-RL achieves higher peaks on both parsing (96.8277) and understanding (83.2 vs. 82.7).

## Theoretical and Practical Implications

- **Capability-Dependent Transfer**: The paper provides systematic evidence that the interaction between parsing and understanding supervision is not static but depends on the maturity of the parsing capability. This suggests a **curriculum principle**: establish parsing before emphasizing understanding.
- **Scalable Data Production**: The automated data engine (ETC + render-based verification + multi-factor mining + targeted synthesis) demonstrates that high-quality OCR supervision can be produced at scale (~170M samples) with minimal human intervention, addressing a major bottleneck in OCR model development.
- **Compact Model Efficiency**: A 0.8B model can rival or exceed much larger general VLMs (e.g., Qwen3-VL-235B, GPT-5.2) on document parsing benchmarks while supporting OCR-VQA and KIE, suggesting that task-specialized compact models remain a highly efficient design choice.
- **Training Objective Design**: The comparison of Mix-RL with MOPD offers practical guidance: distillation provides fast convergence, while longer RL runs achieve higher final performance — a compute/performance trade-off relevant to practitioners.

## Conclusion

Xiaomi-OCR-0 demonstrates that a compact 0.8B VLM can achieve state-of-the-art document parsing and strong OCR-centric understanding through a combination of: (1) a scalable automated data engine, (2) a progressive training recipe (Q-Mask → CPT → Mix-RL), and (3) a principled understanding of when and how understanding supervision transfers to parsing.

Key takeaways:
- Understanding supervision shifts from **negative to positive transfer** as the parsing foundation matures.
- The 4B teacher comparison reveals that **understanding is more capacity-demanding** than parsing, supporting a parsing-first curriculum.
- **MOPD converges faster** on parsing, while **extended Mix-RL** reaches higher final performance on both capabilities.

**Future work** will focus on narrowing the understanding gap relative to larger models, improving robustness under physical-world recapture and long-tail script shifts, and extending coverage across document types and languages while maintaining a compact inference model.

---

_Markdown view of https://picx.dev/p/qXuXtp, served by PicX — AI-generated visual whiteboard summaries of research papers._