Compiler-Guided Proof Search with Cross-Model Synergy for Context-Dependent Theorem Proving

Summary (Overview)

  • Problem addressed: Theorem proving in real-world Lean 4 projects is difficult because proofs depend on project-specific context (local definitions, project-specific lemmas, naming conventions). Standard independent sampling (pass@k) treats the compiler as a final filter rather than a source of guidance.

  • Key contribution: The authors propose a compiler-guided search framework that balances exploration and exploitation. It combines two complementary LLMs (a Lean-specialized model and a general reasoning model), uses compiler feedback for structured error reporting, and employs a pairwise comparison mechanism to preserve promising proof states during refinement.

  • Core mechanism: The system alternates between exploration (drawing new candidates from both models) and exploitation (repairing the current-best proof). A stagnation counter triggers a return to exploration when refinement stops making progress.

  • Main findings: On multiple Lean 4 projects, the framework outperforms both base models and agentic baselines, achieving e.g. 12.66% absolute pass-rate improvement over the best single model at lower cost on miniCTX-v2, and 3.8 points over the best single-model pass@32 on RLMEval-FLT3. The gains are attributed to explicit exploration–exploitation balancing, not to additional model calls alone.

  • Analysis: Ablation experiments confirm that all components contribute; pairwise comparison especially keeps the system from accumulating errors and wasting model calls on unpromising refinements.

Introduction and Theoretical Foundation

Background and Motivation

Theorem proving in formal systems such as Lean has seen major advances recently. Systems like DeepSeek-Prover-V2 and AlphaProof have excelled on Olympiad-style problems, but real-world, project-level theorem proving remains challenging because:

  • The proof context includes project-specific definitions, lemmas, notations, and dependencies.
  • Theorems often rely on highly specific naming conventions and proof patterns not present in pretraining data.
  • The theorem and its context jointly determine the correct proof strategy; hence the problem is context-dependent.

Problem with Existing Approaches

  • Pass@k sampling: Independent samples rarely benefit from each other; every failed attempt is discarded inaccessible feedback. This is wasteful for hard theorems that require iterative repair.
  • Agentic approaches (ReAct-style): Use an LLM controller to decide what to do next.
  • Tactic-level search: Expensive and brittle for context-dependent problems.

Observation motivating the design: failed proofs are not useless: they contain a nearly correct structure or a promising partial sequence of tactics nasally. But whether the reuse is helpful depends on both (a) the starting point and (b) the sequence of refinements, which can degrade a promising proof. This motivates the pairwaise comparison for selecting and preserving the best current proof state.

Summary and Conclusions

The paper introduces CARP (Compiler-Guided Adaptive Refinement Protocol), a method for context-dependent theorem proving in Lean 4. Key elements:

  • Two complementary models: a generalist (strong context usage) and a specialist (strong syntax and formal tactics).
  • Compiler feedback: structured errors from Lean are used for both repair and comparison.
  • Pairwise comparison: an LLM judge decides whether a new refinement replaces the current best proof, or if the old one should be kept.
  • Stagnation counter: triggers resampling when refinement no longer improves the proof.
  • AutoSolve: a fast deterministic tactic that closes goals without LLM calls.

Experiments on 7 real-world Lean 4 projects (miniCTX-v2) show significant improvements over both base models and a ReAct-style agent baseline<｜begin▁of▁file｜>

Reference

</table>

[1] Aditya Kurniawan et al. Minimax-01: Scaling Mixture-of-Experts with Sub-1-Bit Compression. [2] Anand Rao et al. RL, but don't do RL! Better alignment via automated evaluation and LLM feedback. [3] Clara Loh and Richard Pither. Grothendieck: A Lean 4 Framework for Integrals and Expectation Theory.

[4] Leonardo de Moura, Soonho Kong, Jeremy Avigad, Floris van Doorn, and Jakob von Raumer. The Lean theorem prover (system description). In Automated Deduction (CADE), 2015.

[5] Shanghua Gao, Qi Tan, et al. IMO金牌s. 2024.

[6] Thibault Gaillard, et al. StableCode technical report. 2024.

[7] Google. Gemini 2.5: Our most intelligent AI model. 2025. [Online].

[8] Jesse Michael Han, wj45, Michael Jamison, Tsvi Benson-Tilsen, Siddhartha Gadgil, Mario Carneiro, Joe Palermo, and Navit. lean-agent: Creating adaptive, scalable, automated agents for lean. https://github.com/lean-dojo/LeanAgent, 2024.

[9] Jesse Michael Han et al. Lean-agent: Creating adaptive, scalable, automated agents for lean. GitHub repository. https://github.com/lean-dojo/LeanAgent.

[10] Jiarui Hu, Thomas Zhu, and Sean Welleck. miniCTX: Neural Theorem Proving with Long Context, 2024.

[11] Angeliki Koutsoukou-Argyraki et al. The lean 4 theorem prover and its formal mathematics projects. 2024.

[12] Alex Lin, Zhihan Yang, et al. Llemma: An open language model for mathematics. arXiv, 2023. [13] Zihan Liu et al. CRITIC: LLaMA learns to self-correct, 2023.

[14] Gabriel Poiroux et al. Project-level code generation for Lean 4. 2025a.

[15] Gabriel Poiroux et al. Interleaving generation and evolution for project-level theorem proving. 2025b.

[16] Maciej Mikula et al.

...

[38] ...

Please summarize the paper.

Related papers