Summary (Overview)

• Xiaomi-OCR-0 is a unified 0.8B-parameter vision-language model (VLM) that jointly handles document parsing and OCR-centric understanding (OCR-VQA and key information extraction), initialized from Qwen3.5-0.8B.
• The model is trained on an approximately 170M-sample OCR-centric corpus built via an automated data engine combining expert consensus (Ensemble Triplet Consensus), render-based verification, multi-factor sample mining, and targeted synthetic data generation.
• A progressive training recipe is introduced: Q-Mask-based text anchoring → continued pretraining (CPT) → mixed-task reinforcement learning (Mix-RL).
• Xiaomi-OCR-0 achieves state-of-the-art results: 95.24 on Real5-OmniDocBench (highest among compared methods), 96.83 on OmniDocBench v1.6, 87.94 on Wild-OmniDocBench, and an average of 83.2 across five OCR-VQA benchmarks.
• A key finding: understanding-to-parsing transfer is capability-dependent — understanding supervision hurts parsing at early training stages but benefits it once the parsing foundation matures.

Introduction and Theoretical Foundation

Modern OCR systems must support two complementary capabilities:

1. Document parsing — faithful, exhaustive reconstruction of a document's content and structure (text, tables, formulas, reading order).
2. OCR-centric understanding — selective, task-conditioned interpretation of task-relevant visual text, encompassing OCR-oriented visual question answering (OCR-VQA) and key information extraction (KIE).

The paper identifies two central challenges in jointly developing these capabilities:

• Scalable reliable supervision: Text, tables, formulas, and reading order require different verification procedures; plausible model-generated annotations can conceal subtle errors.
• Divergent objectives: Parsing requires exhaustive reconstruction, while understanding requires selective, task-conditioned responses. Simply mixing training data does not guarantee positive transfer.

The authors' experiments reveal that the interaction between parsing and understanding depends on the maturity of the parsing capability:

"When parsing capability is still immature, introducing understanding supervision too early can interfere with developing a strong parsing foundation, resulting in negative transfer. As parsing capability matures, however, the same understanding supervision becomes beneficial, shifting from negative to positive transfer."

The model builds on Qwen3.5-0.8B [55] and uses PP-DocLayoutV3 [14] as an external layout detector for two-stage inference.

Methodology

1. Data Engine (~170M samples)

Automated Annotation Pipeline (Figure 2 in paper):

• Ensemble Triplet Consensus (ETC): Eight heterogeneous OCR experts ($\mathcal{M} = \{M_1, \ldots, M_N\}$) produce predictions for each region. Pairwise agreement is measured with region-specific similarity $S(y_i, y_j) \in [0,1]$: 1−NED for text, TEDS for tables, CDM for formulas.

The consensus score for each expert is:

$$C_i = \frac{1}{N-1} \sum_{j \neq i} S(y_i, y_j). \tag{1}$$

The top-3 experts form the consensus triplet $\mathcal{T}^\star$, and the region's agreement score is:

$$a(x) = \frac{1}{3}\sum_{\substack{i,j\in \mathcal{T}^{\star}\\ i < j}}S(y_{i},y_{j}). \tag{2}$$

The pseudo-label is the triplet medoid:

$$y^{\star}(x) = y_{i^{\star}}, \qquad i^{\star} = \arg \max_{i \in \mathcal{T}^{\star}} \sum_{j \in \mathcal{T}^{\star} \setminus \{i\}} S(y_i, y_j). \tag{3}$$

Regions are tiered by agreement:

$$d_{\text{agree}}(x) = \left\{ \begin{array}{ll} \text{high}, & a(x) \geq 0.9, \\ \text{medium}, & 0.6 \leq a(x) < 0.9, \\ \text{low}, & a(x) < 0.6. \end{array} \right. \tag{4}$$

• Render-Guided Refine-and-Judge: Medium/low-agreement regions are refined by Qwen3.5-122B-A10B via iterative render-based verification (up to 3 rounds), followed by history-aware decoupled refinement (Critique–Refine–Verify) and re-consensus.

Multi-Factor Sample Mining: Combines expert agreement $a_i$, model self-consistency $u_i$, and semantic clusters (via K-Means on Qwen3-VL embeddings) to construct task-specific mixtures. Cluster-based sampling uses quotas:

$$q_k = \left\{ \begin{array}{ll} \max(1, \lfloor \rho \cdot |\mathcal{C}_k| \rfloor), & |\mathcal{C}_k| < n/K, \\ \min(|\mathcal{C}_k|, \max(1, \text{round}(n \cdot |\mathcal{C}_k| / N_{\text{data}}))), & |\mathcal{C}_k| \geq n/K, \end{array} \right. \tag{9}$$

with within-cluster sampling probability:

$$P(i \mid k) = \frac{(a_i + \varepsilon)^{f_a/\tau}(1 - u_i + \varepsilon)^{f_u/\tau}}{\sum_{j: x_j \in \mathcal{C}_k}(a_j + \varepsilon)^{f_a/\tau}(1 - u_j + \varepsilon)^{f_u/\tau}}. \tag{10}$$

Synthetic Data Generation: Two modes — coverage-driven (fills distributional gaps in structures, styles, low-resource scripts) and failure-driven (generates targeted variants around diagnosed model errors).

2. Progressive Training Recipe

Stage 1 — Text-Anchoring Pretraining (Q-Mask) : Uses the Spatial Supervision Alignment (SSA) objective:

$$\mathcal{L}_{\mathrm{SSA}} = \lambda_{\mathrm{txt}} \mathcal{L}_{\mathrm{NTP}} + \lambda_{\mathrm{seg}}(\mathcal{L}_{\mathrm{Dice}} + \mathcal{L}_{\mathrm{CE}}). \tag{12}$$

Stage 2 — Continued Pretraining (CPT) : Standard next-token prediction plus retained MTP head:

$$\mathcal{L}_{\mathrm{CPT}}(\theta) = -\mathbb{E}_{(I, c, y) \sim \mathcal{D}}\left[\sum_{t=1}^{|y|} \log \pi_{\theta}(y_t \mid y_{<t}, I, c)\right] + \lambda_{\mathrm{MTP}} \mathcal{L}_{\mathrm{MTP}}(\theta). \tag{13}$$

Stage 3 — Mixed-Task Reinforcement Learning (Mix-RL) : DAPO-style GRPO objective with token-level aggregation, clip-higher, and dynamic sampling:

$$\mathcal{J}_{\mathrm{GRPO}}(\theta) = \mathbb{E}_{x \sim \mathcal{D}, \{o_i\}_{i=1}^G \sim \pi_{\mathrm{old}}}\left[ \frac{1}{\sum_{i=1}^G |o_i|} \sum_{i=1}^G \sum_{t=1}^{|o_i|} \min\left(r_{i,t}(\theta)\hat{A}_i, \operatorname{clip}(r_{i,t}(\theta), 1-\varepsilon_{\text{low}}, 1+\varepsilon_{\text{high}})\hat{A}_i\right) \right]. \tag{14}$$

The document-parsing reward is a weighted combination:

$$R_{\mathrm{doc}} = w_{\mathrm{text}} s_{\mathrm{text}} + w_{\mathrm{table}} s_{\mathrm{table}} + w_{\mathrm{formula}} s_{\mathrm{formula}}, \tag{15}$$

where weights reflect ground-truth modality character coverage. Rewards use text edit similarity, TEDS, and CDM; KIE uses JSON validity + field matching; OCR-VQA uses ANLS.

Empirical Validation / Results

Document Parsing

OmniDocBench v1.6 (Table 1): Xiaomi-OCR-0 achieves 96.83 Overall, close to TeleOCR's 96.87 (1.2B) and above same-size OvisOCR2 (96.58). Formula CDM of 98.50 is the highest in the comparison.

Real5-OmniDocBench (Table 2): 95.24 Overall, leading all compared methods across all five acquisition conditions (scanning, warping, screen-photo, illumination, skew). The gain over PaddleOCR-VL-1.6 is 2.05 points overall, rising to 2.98 on warped pages.

Wild-OmniDocBench (Table 3): 87.94 Overall, trailing TeleOCR's 88.53 but with the lowest reading-order edit distance (0.1931).

Element-Level: PubTabNet TEDS of 92.27 (highest in Table 4); UniMER-Test Overall of 97.84 with SPE of 99.460 (highest in Table 5).

Long-Tail (Chronicles-OCR): Mature Average of 0.73, exceeding Seed2.0 Pro's 0.72 (Table 7).

OCR-Centric Understanding

OCR-VQA (Table 6): Mean of 83.2 across DocVQA, InfoVQA, ChartQA, OCRBench, and TextVQA — a gain of 10.3 points over Qwen3.5-0.8B and 6.7 over HunyuanOCR-1.5. Largest gains are on InfoVQA (+14.8) and ChartQA (+15.1). The 0.8B model exceeds Qwen3.5-2B by 2.3 points and trails Qwen3.5-4B by only 1.7 points.

KIE (Tables 8–9): 59.09% overall accuracy, exceeding Qwen3.5-2B by 7.49 points. Gains are concentrated in flight-information (73.00% vs. 43.00%) and document-field extraction (77.89% vs. 63.15%). Address extraction remains difficult (45.93%); invoices trail Qwen3.5-2B (65.00% vs. 77.00%).

Key Ablations

Transfer from Understanding to Parsing (Table 10): Understanding supervision hurts at the early stage (−0.330 Overall) but benefits at the middle (+0.639) and late stages. Page-level supervision also improves the parsing foundation.

Q-Mask and Mix-RL (Table 11): Q-Mask adds +0.2957 Overall (largest gain in Formula CDM, +0.4931); Mix-RL adds +0.3538 Overall (largest gains in Table TEDS, +0.7114).

Model Capacity (Table 12): Scaling 0.8B → 4B yields ~0.15 point gain on parsing but ~4.9 points on understanding, indicating understanding places greater demands on model capacity.

MOPD vs. Mix-RL: MOPD reaches competitive parsing scores (96.7417) at lower compute; longer Mix-RL achieves higher peaks on both parsing (96.8277) and understanding (83.2 vs. 82.7).

Theoretical and Practical Implications

• Capability-Dependent Transfer: The paper provides systematic evidence that the interaction between parsing and understanding supervision is not static but depends on the maturity of the parsing capability. This suggests a curriculum principle: establish parsing before emphasizing understanding.
• Scalable Data Production: The automated data engine (ETC + render-based verification + multi-factor mining + targeted synthesis) demonstrates that high-quality OCR supervision can be produced at scale (~170M samples) with minimal human intervention, addressing a major bottleneck in OCR model development.
• Compact Model Efficiency: A 0.8B model can rival or exceed much larger general VLMs (e.g., Qwen3-VL-235B, GPT-5.2) on document parsing benchmarks while supporting OCR-VQA and KIE, suggesting that task-specialized compact models remain a highly efficient design choice.
• Training Objective Design: The comparison of Mix-RL with MOPD offers practical guidance: distillation provides fast convergence, while longer RL runs achieve higher final performance — a compute/performance trade-off relevant to practitioners.

Conclusion

Xiaomi-OCR-0 demonstrates that a compact 0.8B VLM can achieve state-of-the-art document parsing and strong OCR-centric understanding through a combination of: (1) a scalable automated data engine, (2) a progressive training recipe (Q-Mask → CPT → Mix-RL), and (3) a principled understanding of when and how understanding supervision transfers to parsing.

Key takeaways:

• Understanding supervision shifts from negative to positive transfer as the parsing foundation matures.
• The 4B teacher comparison reveals that understanding is more capacity-demanding than parsing, supporting a parsing-first curriculum.
• MOPD converges faster on parsing, while extended Mix-RL reaches higher final performance on both capabilities.

Future work will focus on narrowing the understanding gap relative to larger models, improving robustness under physical-world recapture and long-tail script shifts, and extending coverage across document types and languages while maintaining a compact inference model.