OCR-EDR: Rendering-Aware Diagnosis and Repair for Closed-Loop OCR Improvement

Summary (Overview)

• Unified framework: OCR-EDR (OCR Error Diagnosis and Repair) integrates fine-grained error diagnosis, executable correction, and rendering-based visual reassessment into a single closed-loop paradigm that preserves valid predictions while repairing genuine errors.

• Key innovation: The framework jointly analyzes three inputs—source image, editable OCR prediction, and its rendered image—allowing it to distinguish genuine errors from rendering-equivalent but non-canonical encodings (e.g., \to vs \rightarrow) that should be preserved.

• OCRErrBench: A new 900-case benchmark balanced across text (450) and formulas (450), with 457 valid (Good) and 443 erroneous (Bad) cases drawn from diverse real OCR predictions, supporting unified evaluation of preservation, diagnosis, localization, and repair.

• DocEDR model: Built on Qwen3.5-9B, achieves 94.78% diagnostic accuracy on OCRErrBench, repairs 86.23% of erroneous inputs to visual consistency, and improves formula Case-F1 by 30.99 percentage points over DOCR-Inspector-7B.

• Cross-benchmark gains: Repairs improve formula CDM by 2.24–4.62 points across Bad subsets of four different OCR systems on UniMER-Test, demonstrating generalizable inference-time correction.

Introduction and Theoretical Foundation

Background and Motivation

Document OCR systems perform well on routine documents but remain error-prone on complex formulas, structured text, and long-tail formats. Two failure modes dominate:

1. Specialized recognizers in traditional OCR pipelines are limited by recognition capability and training coverage, potentially missing fine-grained content or misrecognizing characters, symbols, and structures.

2. General vision-language models may hallucinate outputs unsupported by the source image.

A critical complication: structured OCR may represent the same visible content using different but equivalent encodings, which should not be treated as recognition errors. Examples include \to versus \rightarrow, and {\bf x} versus \textbf{x}—these differ as strings but render identically.

Limitations of Existing Approaches

• Text-based post-correction (Dong and Smith 2018; Lyu et al. 2021; Thomas et al. 2024) maps noisy OCR sequences to cleaner text but cannot verify whether an edit restores the source document's visual content.

• DOCR-Inspector (Zhang et al. 2025) defines 28 real-world error types with fine-grained feedback but cannot autonomously repair predictions, inspect new renderings, or decide when to stop. It is also sensitive to representational conventions, rejecting valid outputs that differ only in string form.

• LATTE (Jiang et al. 2025) introduces visual comparison via rendered delta-view feedback for LaTeX formulas and tables but operates only on identified errors in task-specific formats.

The OCR-EDR Formulation

OCR-EDR defines a stateful diagnosis-and-repair task: Given a source region image $I$, an initial OCR result $p_0$, and its rendered image $R_0 = \mathcal{R}(p_0)$, the system returns a final OCR result $\hat{p}$ that recovers the visible content. At each turn $t$, the policy:

• Compares source image, current OCR result, and its rendering
• Emits an agent action $a_t$ with structured payload $z_t$
• May apply local/global repair, request updated rendering, or stop

Methodology

Task Formulation

Input state at turn $t$:

$$h_t = (I, p_t, R_t, \tau_{<t}) \tag{1}$$

where $p_t$ is the current OCR candidate, $R_t$ is its rendering observation, and $\tau_{<t}$ records preceding state transitions.

Rendering states: Two non-image states are distinguished:

• $\perp_{\mathrm{stale}}$: rendering becomes outdated after an edit
• $\perp_{\mathrm{fail}}$: explicit rendering failure

Error taxonomy (five atomic classes):

Policy-level action space: inspect, diagnose_scope, localize, patch, global_patch, request_render, stop

DocEDR Model Architecture

Built on Qwen3.5-9B, DocEDR evolves a single editable OCR hypothesis through a trajectory rather than mapping a fixed input directly to output. Key design features:

• Diagnostic actions (inspect, diagnose_scope, localize) append structured evidence without modifying $p_t$
• Patch actions apply localized insert/delete/replace operations; global_patch reconstructs when localized editing is inappropriate
• request_render keeps $p_t$ unchanged and updates observation with $\mathcal{R}(p_t)$
• stop returns the current candidate

Training Strategy: Three Stages

1. Verifier SFT

Train a vision-text consistency verifier $V_\phi$ that predicts validity and atomic errors:

$$\mathcal{L}_{\mathrm{ver}} = -\sum_i \log P_\phi(d_i^\star \mid I_i, p_i, R_i) \tag{2}$$

This checkpoint provides a stable visual-consistency signal and initialization encoding validity, error type, location, and correction cues.

2. Curriculum Repair SFT

Three supervised curricula before reward optimization:

• Stage 1: Single-turn direct repair without rendering
• Stage 2: Decision between requesting rendering and stopping
• Stage 3: Real rendering feedback with multi-turn residual repair and recovery

Each supervised trajectory is optimized via:

$$\mathcal{L}_{\mathrm{CSFT}}^{(k)} = -\mathbb{E}_{\tau^\star \sim \mathcal{D}_k}\left[\sum_{t=0}^{T^\star-1} \log \pi_\theta(a_t^\star, z_t^\star \mid h_t^\star)\right] \tag{3}$$

3. Reward Optimization (GRPO)

Label-conditioned reward with branch-specific objectives for Good ($y_0 = g$) and Bad ($y_0 = b$) inputs:

$$R_{y_0}(\tau) = \begin{cases} S_k(p_T, p_0) - C_g(\tau), & y_0 = g, \\ S_f^V(e_T) + S_p(\tau) - C_b(\tau), & y_0 = b. \end{cases} \tag{4}$$

• For good inputs: $S_k$ rewards preservation; $C_g$ penalizes unnecessary editing/rendering/turns
• For bad inputs: $S_f^V(e_T)$ combines reference agreement with frozen verifier assessment; $S_p$ rewards progress

Group-relative advantage with clipping:

$$\hat{A}_i = \frac{R_i - \bar{R}}{\mathrm{std}(R) + \epsilon} \tag{5}$$

Empirical Validation / Results

OCRErrBench Construction

• 900 cases balanced: 450 text, 450 formulas; 457 Good, 443 Bad
• Collected from DeepSeek-OCR, PaddleOCR-VL-1.5, and Qwen3.5-27B on OmniDocBench and PureDocBench
• Strict source-level isolation: training data excluded from benchmarks used for evaluation
• Triple-model annotation: GPT-5.6 Sol, Claude Opus 4.8, and Gemini 3.1 Pro must unanimously agree; rendering-equivalent positives receive separate manual audit

Diagnostic Results on OCRErrBench

DocEDR outperforms the strongest baseline (Qwen3.5-397B) by 7.00 accuracy points ($p = 2.95 \times 10^{-9}$) and DOCR-Inspector-7B by substantial margins on Type-F1 and localization.

Boundary Diagnosis: Rendering-Equivalent vs. Genuine Errors

Edit distance falsely rejects 57.14% of equivalent text; DocEDR reduces this to 25.81% while achieving near-zero false negatives on genuine local errors.

Cross-Benchmark Zero-Shot Diagnosis (DOCRcaseBench, N=637)

Formula Case-F1 improves from 56.69% to 87.68%—61 additional correct cases (31.77% of the formula subset).

Repair Results on OCRErrBench

DocEDR exceeds the strongest direct baseline by 55.98 points on ExactFix and 14.00 points on VisFix (95% CI: 9.48–18.74, $p = 6.44 \times 10^{-9}$).

Formula Recognition Refinement on UniMER-Test (N=12,683 per parser)

Repair improves every fixed Bad partition by 2.24–4.62 CDM points, with the largest gain on GLM-4.6V-Flash, demonstrating consistent gains across parsers with substantially different baselines.

Ablation Findings

• Input modality: Full (I + p + R) input performs best. Removing the OCR sequence drops localization to 80.54/55.25% (text/formula); removing rendering doubles equivalent-positive FP from 4.75% to 8.86%.
• Interaction mechanism: Rendering adds 3.61 ExactFix and 3.39 VisFix points in one step; iteration without rendering lowers ExactFix to 66.82%, versus 82.17% with the complete loop.

Theoretical and Practical Implications

Theoretical Contributions

1. Rendering-aware evaluation: The work demonstrates that visual consistency—not string identity—is the correct criterion for OCR validation. Rendering-equivalent encodings must be preserved, which challenges edit-distance-based evaluation paradigms.

2. Stateful correction as unified paradigm: Treating OCR output as an editable state within a closed-loop process (edit → render → reassess → decide) unifies previously fragmented capabilities: preservation, diagnosis, localization, and repair.

3. Label-conditioned reward design: The asymmetric treatment of Good (preservation) vs. Bad (repair) inputs addresses the fundamental tension between not corrupting valid outputs and maximizing correction efficacy.

Practical Implications

1. Actionable feedback: OCR-EDR transforms aggregate error statistics into specific, executable corrections—turning "how many errors" into "what's wrong, where, and how to fix it."

2. Cross-system generalizability: The consistent CDM gains across four different OCR parsers demonstrate that diagnosis and repair capabilities transfer across model families without retraining.

3. Human-AI collaboration: The triple-model annotation pipeline with expert review provides a scalable template for constructing high-quality error diagnosis benchmarks.

4. Data reuse: Verified corrections can serve as supervision for subsequent OCR optimization, closing the loop from inference-time correction to training-time improvement.

Conclusion

OCR-EDR establishes a rendering-aware closed-loop paradigm for document OCR error diagnosis and repair. The key contributions are:

1. Unified task formulation connecting fine-grained diagnosis, executable correction, and rendering-based reassessment while preserving correct and rendering-equivalent predictions.

2. OCRErrBench: A 900-case benchmark with unified evaluation of preservation, diagnosis, localization, and repair across text and formulas.

3. DocEDR model with staged training (verifier SFT → curriculum repair SFT → GRPO) that instantiates OCR-EDR as a unified policy.

4. Verified cross-benchmark corrections: Diagnosis translates into consistent performance gains on error subsets across diverse OCR systems.

The results—94.78% diagnostic accuracy, 86.23% repair-to-visual-consistency, and up to 30.99 percentage-point Case-F1 improvements—demonstrate that difficult OCR cases can serve as actionable feedback rather than terminal failures)Skip to content

Future Directions

The authors suggest that OCR-EDR advances toward OCR systems that continuously improve their predictions and data, implying future work on:

• Extending the framework to additional document types and modalities
• Using verified repairs as training supervision for upstream OCR models
• Scaling the approach to richer interaction loops and larger action spaces

---

Key formulas and definitions:

• $I$: source region image
• $p$: editable OCR prediction (state)
• $\mathcal{R}(p)$: rendering function producing image of prediction
• $R_t$: rendering observation at turn $t$
• $\tau$: trajectory of state transitions
• $y_0 \in \{g, b\}$: ground-truth validity (good/bad)
• $p^\star$: training reference
• $e_T$: terminal evidence $(I, p_T, \mathcal{R}(p_T), p^\star)$
• $S_k$: preservation score
• $S_f^V$: verifier-aware terminal repair score
• $S_p$: trajectory progress score
• $C_g, C_b$: interaction costs for good/bad branches
• CDM: Character Detection Match (formula evaluation metric)