# Process-Verified Reinforcement Learning for Theorem Proving via Lean

> Process-verified reinforcement learning using Lean's tactic-level feedback with first-error credit assignment improves theorem proving accuracy over outcome-only RL baselines.

- **Source:** [arXiv](https://arxiv.org/abs/2606.20068)
- **Published:** 2026-09-05
- **Permalink:** https://picx.dev/p/2a3nug
- **Whiteboard:** https://picx.dev/p/2a3nug/image

## Summary

# Process-Verified Reinforcement Learning for Theorem Proving via Lean

## Summary (Overview)

- **Core Contribution**: This paper introduces a novel reinforcement learning (RL) framework that leverages the Lean proof assistant as a *process-level reward oracle*, providing both global outcome signals and fine-grained tactic-level rewards during training—moving beyond the conventional use of Lean solely as a binary verifier.

- **Key Innovation**: The authors formalize Lean's symbolic feedback into a tactic-level MDP, using a *first-error propagation rule* and a *first-token credit assignment strategy* to transform structured proof feedback into dense, sound training signals within a GRPO-style objective.

- **Empirical Results**: The method consistently outperforms outcome-only GRPO baselines across MiniF2F and ProofNet benchmarks. For STP-Lean, it achieves up to +2.5%p improvement on MiniF2F (pass@64) and +1.4%p on ProofNet (pass@32), while demonstrating more stable training dynamics.

- **Key Insight**: The paper demonstrates that proof assistants can serve not only as verifiers at inference
inference time but also as structured## Introduction and Theoretical Foundation

### Background## Background and Motivation

The paper addresses a fundamental challenge in reinforcement learning for formal theorem proving: traditional RLVR (Reinforcement Learning from Verifiable Rewards) relies on sparse binary outcome signals, while formal proof assistants like Lean offer rich, structured feedback at each step of proof## 3.1 Lean4 Formalization

The paper formalizes Lean's feedback mechanism through three key functions:

- **Parsing function** $f: \mathcal{Y} \to \mathcal{T}^*$: extracts the sequence of tactics from a proof, sorted by starting position
- **Global scoring function** $### $g: \mathcal{Y}# 3.2 Tactic-Level MDP

The framework models the proof generation process as a tactic-level MDP $\mathcal{M} = (\mathcal{S# 3.3 Credit Assignment in RL

The paper contrasts two RL approaches:
- **PPO**: Uses Generalized AdvantageSkip to content
Advantage Estimation with# Methodology

### 4.1import numpy as np 1 Defining Tconst T# Tactic-Level Rewards

The## Empirical Validation / Results

The paper# Methodology

The paper defines two complementary reward signals:

- **Outcome reward**: $r_{\text{outcome}}(Y_i) = g(Y_i)$ where $g(Y_i) = 1$ if the proof passes Lean
  verification, 0 otherwise
- **Process reward**: $\varphi(Y, T)$ assigning values:
  - $1$ for successful tactics in complete# Empirical Validation / Results

The paper provides a detailed analysis of experimental results across multiple dimensions:

1. **Main Results** (Table 1): STP-Lean + Ours achieves 59.2% on MiniF2F## Theoretical and Practical Implications

The paper# Conclusion

The paper concludes with several key takeaways:

---

_Markdown view of https://picx.dev/p/2a3nug, served by PicX — AI-generated visual whiteboard summaries of research papers._