Summary (Overview)

• CodeLift is a reinforcement learning method for software engineering (SWE) agents that dynamically calibrates pass-rate rewards by treating the test suite as an adaptive curriculum, addressing three identified imbalances in raw pass-rate rewards: multiplicity, progress-regression, and difficulty imbalances.
• The method introduces behavior balancing, which groups tests and patches by their outcome patterns and weights test groups by the fraction of patch groups they "kill" (fail), naturally reducing the influence of trivial all-passing tests while emphasizing challenging ones.
• CodeLift also introduces snapshot slicing, which applies the same scoring rule to intermediate repository states at self-test checkpoints, enabling credit assignment for useful intermediate actions even when later edits introduce regressions.
• Experiments on SWE-bench Verified and SWE-bench Pro (Python) show consistent improvements over binary and enhanced pass-rate baselines across pass@1/2/4/8, with gains ranging from +2.75 to +7.40 percentage points on Verified.
• Analyses demonstrate that CodeLift provides denser learning signals (75.1% active rollout groups vs. Binary's 47.9%), more balanced progress-regression rewards (1.08 ratio vs. PassRate's 3.71), and an adaptive curriculum where test weights dynamically shift as the policy improves.

Introduction and Theoretical Foundation

Background

Large language model (LLM) agents have made significant progress in software engineering (SWE), with reinforcement learning with verifiable rewards (RLVR) serving as a key contributor. In typical SWE agentic tasks, agents navigate repositories, edit code, and run tests, with final patches verified against hidden test suites.

The Binary Reward Problem

The standard recipe uses a pass-all-test binary verifier:

$$J_{\mathrm{success}} (\theta) = \mathbb{E}_{x, \tau} [ R_{\mathrm{Binary}} (s_{\mathrm{final}} (\tau)) ], \qquad R_{\mathrm{Binary}} (s) = \prod_{t = 1}^{m_{x}} o_{t} (s),\tag{1}$$

where $o_t(s) \in \{0, 1\}$ is the result of test $t$. This binary reward is too sparse to distinguish patches that partially resolve issues from those that break existing functionality. When all patches in a rollout group fail, there is no learning signal from relative advantage.

Pass-Rate as a Dense Alternative

The natural dense alternative is test pass-rate:

$$R_{\text{PassRate}} (s) = \frac{1}{m_{x}} \sum_{t = 1}^{m_{x}} o_{t} (s).\tag{2}$$

However, the paper identifies three test-weighting imbalances that miscalibrate pass-rate rewards:

1. Multiplicity imbalance: Multiple tests may check the same functionality and fail together. A patch with higher pass rate might fix only a few heavily-tested functionalities while breaking many lightly-tested ones, creating reward bias.

2. Progress-regression imbalance: Most repository tests are pass-to-pass (P2P) regression tests protecting existing functionality, while only a few fail-to-pass (F2P) tests check requested progress. Under pass-rate rewards, penalties for P2P regressions can outweigh rewards for F2P progress, discouraging agents from making changes.

3. Difficulty imbalance: Pass rate assigns equal weight to tests of different difficulty. When easy tests dominate, a patch can get high reward while still failing challenging tests.

Methodology

Behavior Balancing

For each issue $x$, CodeLift evaluates $n \geq 2$ final patches on its $m_x$ golden tests, forming a matrix $M \in \{0,1\}^{n \times m_x}$. It then:

1. Groups identical rows into patch groups $\mathcal{G}_{\text{patch}}$ and identical columns into test groups $\mathcal{G}_{\text{test}}$
2. Computes each test group's failure frequency and normalized weight:

$$d_{g} = \frac{1}{|\mathcal{G}_{\text{patch}}|} \sum_{p \in \mathcal{G}_{\text{patch}}} (1 - Y_{gp}), \quad w_{g} = \frac{d_{g}}{\sum_{h \in \mathcal{G}_{\text{test}}} d_{h}}.\tag{3}$$

where $\sum_g w_g = 1$, or $w_g = 0$ if all $d_g = 0$. This ensures:

• All-passing tests receive zero weight (they cannot distinguish patch quality)
• Tests passed by fewer patch groups receive more weight
• The weights adapt dynamically as the policy improves, creating an adaptive curriculum

Snapshot Slicing

CodeLift uses self-test actions as checkpoints to cut rollouts into token intervals. For each slice $i$ with cumulative endpoint state $s_i^j$, it computes the state score:

$$S(s) = \sum_{g \in \mathcal{G}_{\mathrm{test}}} \frac{w_{g}}{|g|} \sum_{t \in g} o_{t}(s).\tag{4}$$

The snapshot correction and slice advantage are:

$$\Delta_{i}^{j} = S(s_{i}^{j}) - r_{\mathrm{final}}^{j}, \qquad A_{i}^{j} = \Delta_{i}^{j} + A_{\mathrm{final}}^{j}.\tag{5}$$

Credit Assignment

Using a leave-one-out (LOO) estimator:

$$b_{j} = \frac{1}{n - 1} \sum_{j' \neq j} r_{\mathrm{final}}^{j'}, \qquad A_{\mathrm{final}}^{j} = r_{\mathrm{final}}^{j} - b_{j}, \qquad A_{i}^{j} = S(s_{i}^{j}) - b_{j}.\tag{6}$$

This compares each endpoint against the mean of other $n-1$ rollouts' final scores, giving positive credit when it exceeds that reference. The full algorithm is summarized in Algorithm 1.

Empirical Validation / Results

Main Results

Table 1: Main results on SWE benchmarks (pass@k %)

CodeLift consistently outperforms all baselines across all benchmarks and sampling budgets.

Ablation Study

Table 2: Ablation for CodeLift's components

Both components contribute meaningfully: behavior-balanced scoring has larger impact on Pro, while snapshot slicing has larger impact on Verified.

Robustness Analysis

Table 3: Robustness across training settings (Qwen3-Coder on SWE-rebench)

CodeLift outperforms Binary under both RLOO and GRPO advantage estimators.

Theoretical and Practical Implications

Dense and Stable Learning Signals

• CodeLift maintains a mean of 75.1% active rollout groups (with non-zero advantages) throughout training, compared to Binary's 47.9%.
• Raw PassRate collapses quickly during training, while CodeLift maintains stable performance (35.8–36.2% after step 180 vs. Binary's 30.4–35.0%).

Balanced Progress-Regression Preferences

• Under raw PassRate, the mean penalty for introducing regressions is 3.71× the mean reward for making progress, encouraging conservative behavior (58.3% of positive advantages come from patches without pass-rate changes).
• CodeLift achieves a 1.08 ratio (near parity), with 81.6% of positive advantages coming from patches that make real progress.

Adaptive Curriculum

• CodeLift assigns zero weight to all-passing test groups, dynamically redistributing weight to tests that fewer patch groups pass.
• As the policy improves, easy test groups join the all-pass group without occupying weight space, while hard groups' average weights keep rising.
• This curriculum emerges naturally from existing final-patch test outcomes, requiring no extra rollout generation (only ~2.35 additional CPU-based test executions per rollout for snapshot slicing).

Conclusion

CodeLift combines behavior balancing and snapshot slicing to provide dense, well-calibrated rewards for SWE agents. The method:

1. Groups tests and patches by outcome behavior to reduce pass-rate imbalances
2. Naturally introduces an adaptive curriculum from existing final-patch outcomes
3. Applies the same scoring rule to intermediate patches, assigning credit for progress even when later edits introduce regressions

Experimental results on SWE-bench Verified and SWE-bench Pro (Python) demonstrate consistent advantages over binary and enhanced pass-rate baselines. The analysis confirms denser learning signals, more balanced rewards, and test weights that adapt to the changing policy.

Limitations: The method is validated on Python repositories only; performance in other languages remains to be evaluated. Snapshot slicing requires additional test executions (~2.35 non-final patches per rollout), though these run on CPUs rather than GPUs.

Future directions could include extending CodeLift to other programming languages and repository environments, and further refining the credit assignment mechanism for even finer-grained learning signals.