# OvisOCR2 Technical Report

> OvisOCR2, a 0.8B end-to-end model, achieves state-of-the-art document parsing via a dual-pipeline data engine and multi-stage training.

- **Source:** [arXiv](https://arxiv.org/abs/2607.13639)
- **Published:** 2026-07-17
- **Permalink:** https://picx.dev/p/troaBQ
- **Whiteboard:** https://picx.dev/p/troaBQ/image

## Summary

# Summary of OvisOCR2 Technical Report

## Summary (Overview)

- **OvisOCR2** is a 0.8B parameter end-to-end document parsing model that converts a document page image directly into a Markdown representation in natural reading order, covering text, formulas, tables, and visual regions.
- A **dual-pipeline data engine** combines filtered real-document annotations (from OCR parsers with rule-based normalization and manual spot-checking) with synthetic pages generated from HTML templates, ensuring precise ground-truth alignment.
- The **training recipe** includes supervised fine-tuning (SFT), reinforcement learning (GRPO) on a 4B teacher branch, on-policy distillation (OPD) into the 0.8B student, and model fusion via weighted parameter averaging.
- On **OmniDocBench v1.6**, OvisOCR2 achieves a state-of-the-art overall score of **96.58**, surpassing all pipeline methods (e.g., PaddleOCR-VL-1.6, MinerU2.5-Pro) and previous end-to-end models.
- On **PureDocBench**, it achieves the highest **Avg3 score of 75.06**, and on an in-house benchmark covering long-tail and challenging scenarios, it obtains the best overall performance, especially on handwriting and complex-table subsets.

## Introduction and Theoretical Foundation

Document parsing extends plain OCR by preserving page organization, reading order, tables, formulas, figures, and other layout-dependent elements, typically outputting Markdown. Approaches fall into two families:

- **Pipeline methods**: Decompose parsing into layout analysis, region-level content recognition, and merging. They dominate public leaderboards but suffer from deployment complexity (multiple models with different runtime loads) and error accumulation across stages.
- **End-to-end methods**: Use a single model to generate the Markdown representation in one pass, simplifying deployment and allowing the model to condition on page-level context. However, they have historically lagged behind pipelines in performance.

The authors argue that the end-to-end approach is more elegant and develop OvisOCR2 by post-training **Qwen3.5-0.8B**, the smallest model in the Qwen3.5 family. The goal is to achieve state-of-the-art performance with a compact model, requiring both a carefully designed data engine and a training recipe suited to long, structured outputs.

## Methodology

### Data Engine

The data engine consists of two complementary pipelines (see Figure 2 in the paper):

**Real-world data pipeline**:
- Processes document images using specialized OCR parsers (PaddleOCR-VL-1.5 or MinerU2.5-Pro) to obtain structured JSON outputs.
- Converts these outputs into a unified Markdown format via rule-based normalization (category validation, text-block merging, LaTeX-style formula normalization, HTML table normalization, visual-region bounding box serialization).
- Applies manual spot-checking on subset samples to verify text correspondence, formula accuracy, table alignment, reading-order consistency, and visual-region placement. Low-quality subsets are removed while acceptable ones are retained.

**Synthetic data pipeline**:
- Generates both document images and their Markdown targets from the same HTML source, ensuring deterministic, noise-free labels.
- Starts with **hard sample mining** from failure cases (table-heavy layouts, irregular structures, handwritten regions, etc.) to create reusable HTML templates.
- Uses an **agent-based diversification** procedure to expand templates into diverse HTML pages with content-level and structure-level variations.
- **Markdown ground truth** is generated directly from the HTML source via a serializer with document-type-aware reading order (single-column: top-to-bottom, left-to-right; multi-column: column partitioning).
- Pages are rendered using **Playwright** for browser-based, realistic rendering, with element-level geometry recorded.
- **Iterative quality control** ensures renderable, category-compliant, and valid Markdown before large-scale generation.

### Training Pipeline

The training pipeline is organized in stages (see Figure 3 in the paper):

**Supervised Fine-Tuning (SFT)**:
- Establishes the base policy using the full data mixture (real + synthetic).
- Both Qwen3.5-0.8B and Qwen3.5-4B are trained with full-parameter SFT (0.8B for 2 epochs, 4B for 20% of an epoch). A 16K maximum sequence length and dynamic image-resolution budget are used.

**Reinforcement Learning (RL)**:
- Uses **Group Relative Policy Optimization (GRPO)** to improve parsing on hard cases using multi-component rewards.
- RL data primarily from synthetic pipeline (where ground truth is accurate) plus a smaller set of high-quality real documents.
- **On-policy filtering** focuses training on pages where the model can occasionally produce clearly better responses.
- **Multi-component reward design** (Table 1):

| Component | Score | Measured aspect |
|-----------|-------|-----------------|
| Text | 1 - normalized edit distance | Text fidelity |
| Formula | CDM (character detection matching) | Visual formula matching |
| Table | TEDS (tree-edit distance similarity) | Table content and topology |

The page-level reward is computed as:

$$R(y, y^*) = \frac{\sum_{c \in \mathcal{C}} a_c(y^*) \, s_c(y, y^*)}{\sum_{c \in \mathcal{C}} a_c(y^*)}$$

where $\mathcal{C} = \{\text{text}, \text{table}, \text{formula}\}$, $a_c(y^*)$ is the availability indicator (1 only if the reference contains evaluable units of that type), and $s_c(y, y^*) \in [0,1]$ is the component score.

- Scalable optimizations: hierarchical parallelism for reward computation (exact matches bypass expensive rendering), object-store-backed reference passing for large visual tensors, and common-prefix masks for long responses.

**On-Policy Distillation (OPD)**:
- Direct RL on 0.8B shows higher KL divergence and unstable table quality (Figure 4). Therefore, the RL-aligned 4B model is used as a teacher.
- The student generates responses; the teacher evaluates on-policy trajectories with token-level distribution supervision.
- The loss uses **student top-k reverse KL**:

$$\bar{p}_{t,v} = \frac{\pi_\theta(v | c_t)}{\sum_{u \in S_t} \pi_\theta(u | c_t)}, \quad \bar{q}_{t,v} = \frac{\pi_\phi(v | c_t)}{\sum_{u \in S_t} \pi_\phi(u | c_t)}$$

$$\mathcal{L}_{\text{OPD}}(\theta) = \frac{1}{|I|} \sum_{t \in I} D_{\text{KL}}(\bar{p}_t \parallel \bar{q}_t)$$

where $S_t = \text{TopK}_k(\pi_\theta(\cdot | c_t))$ is the student's top-k support. This reduces tensor size from $O(TV)$ to $O(Tk)$ and is mode-seeking, discouraging student probability on tokens the teacher assigns low probability.

**Model Fusion**:
- Weighted parameter averaging of several candidate OvisOCR2 variants trained with different data mixtures and configurations.

## Empirical Validation / Results

### OmniDocBench v1.6

Evaluates 1,651 pages across 10 document types, 5 layout types, and 5 language types. Metrics: text edit distance, formula CDM, table TEDS/TEDS-S, reading-order edit distance. **Overall score** = average of text score (1 - edit distance), formula CDM, and table TEDS.

| Model type | Method | Parameters | Overall ↑ | Text Edit ↓ | Formula CDM ↑ | Table TEDS ↑ | Table TEDS-S ↑ | RO Edit ↓ |
|------------|--------|------------|-----------|-------------|---------------|--------------|----------------|-----------|
| Specialized VLMs (pipeline) | PaddleOCR-VL-1.6 | 0.9B | 96.33 | 0.033 | 97.49 | 94.76 | 97.11 | 0.127 |
| Specialized VLMs (end-to-end) | HunyuanOCR-1.5 | 1B | 94.74 | 0.039 | 94.50 | 93.67 | 94.71 | 0.129 |
| **OvisOCR2** (end-to-end) | **0.8B** | **96.58** | **0.025** | **97.53** | **94.76** | **97.16** | **0.111** |

OvisOCR2 achieves the highest overall score, lowest text edit distance, highest formula CDM, tied highest TEDS, highest TEDS-S, and lowest reading-order edit distance, surpassing all pipeline and end-to-end methods.

### PureDocBench

1,475 pages across Clean, Digital, and Real tracks (4,425 images total). Avg3 = mean of three track scores.

| Model | Parameters | Clean ↑ | Digital ↑ | Real ↑ | Avg3 ↑ |
|-------|------------|---------|-----------|--------|--------|
| FD-RL (end-to-end) | 4B | 78.38 | 76.33 | 67.04 | 73.92 |
| **OvisOCR2** | **0.8B** | **81.55** | **77.09** | 66.56 | **75.06** |

OvisOCR2 ranks first on Clean and Digital tracks, and first overall in Avg3. On the Real track, it remains below strong general VLMs (Gemini-3.1-Pro, Qwen3.5-122B-A10B), indicating a direction for future robustness improvement.

### In-house Benchmark

>1,000 pages covering long-tail scenarios (forms, scanned reports, handwritten annotations, complex tables). Evaluated using the OmniDocBench protocol.

| Model | Text Edit ↓ | Formula CDM ↑ | Table TEDS ↑ | Table TEDS-S ↑ | RO Edit ↓ | Overall ↑ |
|-------|-------------|---------------|--------------|----------------|-----------|-----------|
| PaddleOCR-VL-1.6 | 0.1292 | 85.13 | 76.42 | 80.74 | 0.2358 | 82.88 |
| **OvisOCR2** | **0.0850** | **86.32** | **78.80** | **82.87** | **0.1885** | **85.54** |

Performance by difficulty level:

| Model | Easy ↑ | Medium ↑ | Hard ↑ |
|-------|--------|----------|--------|
| PaddleOCR-VL-1.6 | 84.98 | 83.16 | 75.06 |
| **OvisOCR2** | **87.95** | **85.82** | **78.99** |

**Handwriting subset**:

| Model | Overall ↑ | Text Edit ↓ | Formula CDM ↑ | Table TEDS ↑ | Table TEDS-S ↑ | RO Edit ↓ |
|-------|-----------|-------------|---------------|--------------|----------------|-----------|
| GLM-OCR | 69.58 | 0.2421 | 75.63 | 57.31 | 65.76 | 0.2340 |
| **OvisOCR2** | **72.28** | **0.1561** | **81.51** | 50.95 | 59.84 | **0.1733** |

**Complex-table subset**:

| Model | Overall ↑ | Text Edit ↓ | Table TEDS ↑ | Table TEDS-S ↑ | Missing Rate ↓ |
|-------|-----------|-------------|--------------|----------------|----------------|
| MinerU2.5-Pro | 71.43 | 0.3407 | 76.92 | 79.71 | 0.1327 |
| **OvisOCR2** | **83.97** | **0.1040** | **78.34** | **81.33** | **0.0796** |

OvisOCR2 achieves the highest overall score and lowest missing rate on complex tables, while pipeline methods miss 13–17% of tables during layout parsing (an irrecoverable error).

## Theoretical and Practical Implications

- **End-to-end superiority**: OvisOCR2 demonstrates that a compact end-to-end model can surpass pipeline methods on public benchmarks, contradicting the previous trend where pipelines dominated. The single-pass design simplifies deployment, eliminates error accumulation, and allows the model to leverage page-level context.
- **Data engine design**: The combination of filtered real-world data and source-aligned synthetic data is crucial. Synthetic data provide clean, controllable, and long-tail coverage, while real data introduce natural visual variation. The manual spot-checking stage ensures quality without relying on costly human annotation.
- **Training recipe**: RL with multi-component rewards (text, formula, table) addresses structural errors that next-token prediction alone cannot capture. On-policy distillation from a larger teacher (4B) to a smaller student (0.8B) avoids instability of direct RL on compact models. Model fusion further improves performance.
- **Practical impact**: The 0.8B model is deployable on edge devices with low latency, yet achieves state-of-the-art results. This opens possibilities for real-time document parsing in OCR-heavy workflows like digitization, information retrieval, and document understanding.
- **Remaining challenges**: Robustness to degraded real-world images (phone captures, photocopies, compressed screenshots) and performance on handwriting remain areas for improvement. The paper suggests future work on these fronts.

## Conclusion

OvisOCR2 is a 0.8B end-to-end document parsing model that achieves state-of-the-art results on OmniDocBench v1.6 (96.58 overall) and PureDocBench (Avg3 75.06), outperforming both pipeline and end-to-end methods. Its success is enabled by a dual-pipeline data engine (filtered real annotations + source-aligned synthetic pages) and a multi-stage training recipe (SFT, RL with multi-component rewards, on-policy distillation, model fusion). On an in-house benchmark, it shows consistent leadership across difficulty levels, especially on handwriting and complex-table subsets. Future work will focus on improving robustness to degraded real-world images and further strengthening performance on handwriting-heavy and complex-table documents.

---

_Markdown view of https://picx.dev/p/troaBQ, served by PicX — AI-generated visual whiteboard summaries of research papers._