Summary (Overview)
- PAPER2LEAN is an end-to-end agentic framework that converts mathematical research papers into integrated Lean projects, addressing the challenge of paper-level formalization beyond isolated theorem translation.
- The framework extracts mathematical content into a dependency DAG (Directed Acyclic Graph) and coordinates specialized agents for statement formalization, semantic checking, proof construction, and library integration.
- Evaluated on 20 mathematical optimization papers, all automatically generated projects compile successfully without proof placeholders (no
sorry), with 473 of 494 items (95.7%) judged source-faithful by expert review. - The authors release OPTPAPER-20, a corpus of 20 corrected paper-level formalizations comprising 494 Lean files and 110,910 lines of Lean code, all passing clean builds.
- A key finding: Lean compilation alone does not establish semantic fidelity to the source paper—21 discrepancies were found in Lean-valid developments with no proof placeholders.
Introduction and Theoretical Foundation
Background
Interactive theorem provers (ITPs) such as Rocq, Isabelle, and Lean provide formal languages and trusted kernels for constructing machine-checkable mathematics. Lean, based on dependent type theory, supports executable, machine-checkable representations of mathematical statements and proofs. Its mathematical library mathlib offers extensive reusable definitions, theorems, and proof automation.
The Challenge of Paper-Level Formalization
Formalizing an entire research paper differs fundamentally from translating an isolated theorem:
- Content recovery: Mathematical content must be extracted from PDFs, including notation, cross-references, and the structure of definitions and results.
- Semantic alignment: Formal statements must remain faithful to the source despite omitted/implicit assumptions and notation variations.
- Global consistency: Proofs and declarations must integrate into one project with consistent imports, interfaces, and downstream dependencies as the formalization evolves.
Key Insight
"Successful proof construction establishes correctness relative to the generated statement, but does not by itself establish source fidelity."
This motivates the need for semantic checking as a distinct verification layer beyond Lean's type checking.
Methodology
PAPER2LEAN operates in four main stages:
1. OCR and Extraction
- Uses MinerU to convert PDFs to Markdown/LaTeX.
- An extraction agent identifies mathematical units: standing assumptions, definitions, algorithms, lemmas, propositions, theorems, corollaries, and formally relevant remarks.
- External citations are represented explicitly; definition-gap items are introduced when constructions are used but not defined.
- Content is stored in a canonical JSON representation, with each item recording its source statement, proof text, type, and dependencies.
2. Dependency DAG Construction
Following Ju et al. (2026a), each paper is represented as a directed acyclic graph (DAG) where:
- Nodes = mathematical units
- Directed edges = dependencies (edge means depends on )
Upper layers contain assumptions and definitions; lower layers contain lemmas and theorems. Cycles are rejected, and each item is assigned a dependency depth.
3. Statement Formalization
The orchestrator coordinates three workers per item:
- Formalizer: Reads source statement and dependencies, imports dependency modules, searches mathlib and the shared Foundation library, generates initial Lean formalization.
- Semantic Checker: Reviews the generated statement for semantic misalignment; if detected, marks item as
inaccuratewith detailed feedback. - Semantic Repairer: Fixes issues; the repair-and-check loop continues until alignment is confirmed.
Statements are frozen before proving unless the proving agent identifies a gap.
4. Theorem Proving
The proving orchestrator coordinates four roles:
| Role | Function |
|---|---|
| Planner | Analyzes target sorry placeholders, reads natural language proofs, decomposes into subgoals, identifies lemmas/tactics |
| Formalizer | States auxiliary lemmas (kept in same file, not added to paper-level DAG) |
| Prover | Constructs Lean proofs, fills in sorry; returns diagnostics on failure |
| Golfer | Simplifies proofs post-success; runs mathlib linters, fixes warnings |
Key mechanism: Items in one depth layer are proved concurrently; dependent layers wait until all items in the current layer have compiling, frozen interfaces (a barrier). The plan–prove–feedback loop continues until no sorry, admit, axiom, or Lean errors remain.
5. Library Integration
- After a batch of papers is formalized, the integration agent groups candidate definitions/theorems expressing the same mathematical content.
- Frequently recurring candidates are consolidated into shared Foundation interfaces.
- Affected paper projects are updated and rebuilt against shared interfaces.
- Example: Two paper-local definitions of ε-enlargement (from Moudafi & Oliny 2003, Burachik et al. 2001) consolidated into one Foundation interface (Figure 3).
Empirical Validation / Results
Experimental Setup
- Base model: GPT-5.6-Terra (reasoning effort: high) via Codex CLI coding agents
- Lean toolchain:
leanprover/lean4:v4.30.0-rc2 - Supporting library: Foundation, built using M2F from Bauschke & Combettes (2011), plus theorems from Optlib and Bai et al. (2026)
- All agents interact with Lean via Lean LSP MCP (compiler feedback, goalstate inspection, semantic search)
Paper-Level Results (Table 1)
All 20 projects compile successfully without sorry placeholders. Key metrics:
| Metric | Value |
|---|---|
| Total papers | 20 |
| Total Lean files | 494 |
| Total lines of code | 110,910 |
| Total token usage | ~2.17 billion |
| Faithful items | 473/494 (95.7%) |
| Papers with no discrepancies | 13 |
| Papers with discrepancies | 7 (21 unfaithful statements total) |
Notable per-paper results:
- Best fidelity: 15 papers at 100% faithful
- Lowest fidelity: Lorenz & Pock (2015) at 75% (18/24), Adly & Attouch (2023) at 80% (20/25), Alvarez et al. (2002) at 82% (27/33)
- Largest project: Baji & Cabot (2006) with 15,639 lines of Lean code
- Smallest project: Alvarez & Attouch (2001) with 1,428 lines
Model Comparison
Statement Generation (Table 2): Both GPT-5.6-Terra and DeepSeek-V4.1-Flash achieve near-identical alignment rates. DeepSeek uses ~17.8% fewer tokens on average.
Proof Construction (Table 3): Both models complete all 81 proof-construction tasks. However, GPT-5.6-Terra uses substantially fewer tokens—DeepSeek's total usage is approximately 3.41× that of GPT-5.6-Terra, with per-paper ratios ranging from 1.29 to 4.31.
Semantic Evaluation Protocol
- Four experts in mathematical optimization with Lean experience
- Each paper reviewed by two reviewers (cross-audit design)
- Review occurs after all items are proved and projects build without
sorry - Acceptable revision: Restoring an assumption supported by the whole source context
- Unacceptable revision: Introducing an unsupported, non-redundant assumption that restricts the original claim
Theoretical and Practical Implications
Theoretical Implications
-
Semantic fidelity ≠ proof verification: The 21 discrepancies found in Lean-valid developments demonstrate that type-checking correctness is necessary but not sufficient for faithful formalization. This is a fundamental insight for the autoformalization community.
-
DAG-based dependency management: Representing papers as dependency DAGs with layered barriers enables parallel processing while maintaining global consistency—a scalable architecture for large-scale formalization.
-
The "similar argument" problem: Case Study 2 (Remark 2.19 of Abbas & Attouch 2015) shows that agents can miss contextual dependencies behind phrases like "by a similar argument," even when semantic checking and proof construction both succeed.
Practical Implications
-
OPTPAPER-20 as a resource: The released corpus (494 Lean files, 110,910 lines) provides a substantial benchmark for future research in paper-level formalization.
-
Foundation library: The shared supporting library built from Bauschke & Combettes (2011) plus Optlib theorems enables formalizing optimization papers that rely on background theory unavailable in mathlib.
-
Token efficiency matters: The model comparison shows that while multiple models can complete proof tasks, token consumption varies dramatically (3.41× difference), impacting scalability for large-scale formalization efforts.
-
Human review remains essential: Despite high automation, expert review caught 21 unfaithful statements, motivating future work on automated semantic checking.
Conclusion
PAPER2LEAN demonstrates the feasibility of agentic paper-level formalization: all 20 optimization papers were converted into compiling Lean projects without proof placeholders. The central design insight is the dependency DAG supporting staged processing, compilation-gated validation, and reuse of verified interfaces.
The audit's key finding—that 473/494 declarations were faithful before correction, with all discrepancies repaired in the released corpus—supports paper-level formalization as a practical direction while showing that semantic review remains necessary for detecting:
- Omitted assumptions
- Altered claims
- Lost contextual dependencies
Future Work
The authors identify several directions:
- Automated semantic checking (citing Liu et al., 2026d; 2025)
- Better recovery of implicit assumptions
- Lower token costs
- Broader mathematical domains
- More systematic comparisons with alternative agent architectures and formalization pipelines
Related papers
- Pro$^2$ Agent: Scientific Problem Probing for Autonomous AI R&D
Pro²Agent's explicit Problem State—tracking diagnosed bottlenecks, candidate solutions, and evidence—achieves a record 76% medal rate on MLE-Bench, outperforming code-centric agents.
- The Critic Can Hurt: On the Safety of Critics in LLM-Guided AutoML Search
A naive LLM critic can drive AutoML search below the no-critic random-search floor on 7 of 8 tabular datasets, with losses up to 0.245 AUC.
- Evaluating Governed LLM-Driven ML Experimentation: Lifecycle, Provenance, and Failure Metrics
Governed harnesses with append-only ledgers expose process failures invisible to task scores, achieving 100% provenance across 1,445 trials while ungoverned loops silently lose 86% of trials.