Full text not available for this paper

SWE-Growth: Teacher-Guided Adaptive Self-Play for Software Engineering Agents

Summary (Overview)

  • Novel framework: SWE-Growth introduces a teacher-guided adaptive self-play framework for training software engineering agents, using task-level gaps between a frozen Teacher and the current Student to coordinate supervised fine-tuning (SFT), reinforcement learning (GRPO), and task generation.
  • Key insight: Most self-play generated tasks (79.4%) provide zero group-relative learning signal under binary repair rewards; however, a frozen Teacher can identify learning opportunities within "silent" all-fail groups by providing verified repair demonstrations.
  • Curriculum design: Tasks with all-fail Student outcomes but reliable Teacher success (≥2/4 attempts) provide SFT demonstrations; after SFT, refreshed Student rollouts identify mixed-outcome tasks with positive Teacher–Student gaps for GRPO.
  • Results: Starting from 101 seed issues, SWE-Growth improves Qwen3-Coder-30B-A3B-Instruct from 27.7% to 31.7% on SWE-bench Pro and from 11.0% to 17.2% on held-out TerminalBench 2.1 by Round 2, exceeding Student-only self-play by 2.6 and 4.1 percentage points respectively.
  • Key finding: A weak Teacher (with lower aggregate performance than the Student) can still provide valuable task-level guidance, and the trained Student can solve tasks that neither the initial Student nor the Teacher solves.

Introduction and Theoretical Foundation

Background

Large language model (LLM)-based software engineering agents solve repository-level issues by exploring codebases, editing source code, and checking changes through execution. Training typically relies on prepared tasks, executable repository environments, and successful repair examples. As agents improve, some tasks become too easy while others remain beyond current capability, motivating adaptive task selection.

Motivation: From Task Generation to Learning Signals

The paper identifies a critical bottleneck in self-play for software engineering: generating tasks does not ensure useful learning signals. Under binary repair rewards, tasks where all sampled attempts succeed or fail provide zero group-relative advantages (a concept from Shao et al., 2024).

Diagnostic analysis (with G=4 repair attempts per task, 63 complete Student groups across Go, Python, TypeScript, and JavaScript):

Student-only GRPO (N=63)Teacher on all-fail (N=47)Selected subset
All-fail: 47 (74.6%)All-pass: 3 (4.8%)Mixed: 13 (20.6%)
Silent: 50 (79.4%)T≥1/4: 17 (36.2%)T≥2/4: 9 (19.1%)
——0/4→mixed: 3/5 (60.0%)

A group is "silent" when its four Student rollouts receive identical rewards (zero group-relative advantage). T≥k/4 means the Teacher succeeds in at least k of four attempts.

Key observations:

  • Only 20.6% of Student groups contain both successes and failures (usable for GRPO)
  • The frozen Teacher succeeds on 36.2% of all-fail tasks, providing verified repair demonstrations
  • After SFT on Teacher demonstrations, 60% of previously all-fail tasks yield mixed Student outcomes

This motivates using task-level Teacher–current Student gaps to guide curriculum selection.

Methodology

Problem Formulation

Let t∈Tt \in T denote a clean repository task (repository revision, task context, and designated tests). An Injector policy πinj\pi_{inj} proposes a bug patch bb, producing a synthetic repair task tbt_b. A Student Fixer πstu\pi_{stu} interacts with the modified repository and submits a repair patch pp. A frozen Teacher Fixer πtea\pi_{tea} attempts the same task under a matched interaction budget.

Learning to Inject Repository Bugs

The Injector generates injection patches:

b∼πinj(⋅∣t)b \sim \pi_{inj}(\cdot | t)

Only injections passing execution validation enter the repair curriculum.

EVT Validation of Injected Bugs

Three sequential checks validate injections:

  1. Executability E(t)E(t): clean repository and test harness run successfully
  2. Validity V(t,b)V(t,b): injection patch applies and satisfies edit constraints
  3. Target regression T(t,b)T(t,b): target tests change from passing to failing
Q(t,b)=E(t) V(t,b) T(t,b)Q(t,b) = E(t) \, V(t,b) \, T(t,b)

A task is retained only when Q(t,b)=1Q(t,b) = 1, non-target tests remain passing, and the patch satisfies non-catastrophic-edit constraints.

Repair Evaluation and Curriculum Routing

For each validated task tbt_b, the Student performs GG independent repair rollouts (G=4G=4 by default). A candidate patch receives reward:

r(tb,p)=A(tb,p) U(tb,p) P(tb,p)r(t_b, p) = A(t_b, p) \, U(t_b, p) \, P(t_b, p)

where AA, UU, PP are binary indicators for successful patch application, target-test restoration, and non-target-test preservation.

For agent aa at round kk, the estimated success rate is:

s^a,k(tb)=1G∑g=1Gr(tb,pa,g)\hat{s}_{a,k}(t_b) = \frac{1}{G} \sum_{g=1}^{G} r(t_b, p_{a,g})

The Teacher–Student capability gap is defined as:

s^ref,k(tb)=max⁡1≤m≤Ms^team,k(tb),Δ^k(tb)=s^ref,k(tb)−s^stu,k(tb)\hat{s}_{ref,k}(t_b) = \max_{1 \leq m \leq M} \hat{s}_{tea_m,k}(t_b), \quad \hat{\Delta}_k(t_b) = \hat{s}_{ref,k}(t_b) - \hat{s}_{stu,k}(t_b)

Curriculum Pools

SFT pool (Student-zero tasks with reliable Teacher successes):

DSFT,k={tb:s^stu,kpre(tb)=0,  s^ref,k(tb)≥12}\mathcal{D}_{SFT,k} = \left\{ t_b : \hat{s}_{stu,k}^{pre}(t_b) = 0, \; \hat{s}_{ref,k}(t_b) \geq \frac{1}{2} \right\}

With G=4G=4, this requires at least one Teacher success in two of four attempts.

GRPO pool (mixed Student outcomes with positive gap):

DGRPO,k={tb:0<s^stu,kpost(tb)<1,  Δ^kpost(tb)>0}\mathcal{D}_{GRPO,k} = \left\{ t_b : 0 < \hat{s}_{stu,k}^{post}(t_b) < 1, \; \hat{\Delta}_k^{post}(t_b) > 0 \right\}

Training Objectives

SFT objective with mixed demonstrations and replay examples:

Dmix,k=(1−λrep)Ddemo,k+λrepDreplay,k\mathcal{D}_{mix,k} = (1 - \lambda_{rep}) \mathcal{D}_{demo,k} + \lambda_{rep} \mathcal{D}_{replay,k} LSFT(θ)=−E(h,a)∼Dmix,klog⁡πθ(a∣h)\mathcal{L}_{SFT}(\theta) = -\mathbb{E}_{(h,a) \sim \mathcal{D}_{mix,k}} \log \pi_\theta(a|h)

GRPO objective with clipping and KL penalty:

ϕ(ρ,A)=min⁡{ρA,clip(ρ,1−ϵ,1+ϵ)A}\phi(\rho, A) = \min\{\rho A, \text{clip}(\rho, 1-\epsilon, 1+\epsilon) A\} ℓg,d,j(θ)=ϕ(ρg,d,j(θ),A^g)−βKg,d,j(θ)\ell_{g,d,j}(\theta) = \phi\left(\rho_{g,d,j}(\theta), \hat{A}_g\right) - \beta K_{g,d,j}(\theta) LGRPO(θ)=−EG[1G∑g=1G1∣Vg∣∑(d,j)∈Vgℓg,d,j(θ)]\mathcal{L}_{GRPO}(\theta) = -\mathbb{E}_G\left[\frac{1}{G} \sum_{g=1}^{G} \frac{1}{|\mathcal{V}_g|} \sum_{(d,j) \in \mathcal{V}_g} \ell_{g,d,j}(\theta)\right]

where Vg\mathcal{V}_g denotes valid action-token positions across retained decision records of rollout gg.

Injector Feedback

A valid injection earns reward if its task enters the SFT or GRPO pool:

Rinj,k(t,b)=Q(t,b)Fk(tb)−(1−Q(t,b))cEVT(t,b)R_{inj,k}(t,b) = Q(t,b) F_k(t_b) - (1 - Q(t,b)) c_{EVT}(t,b)

where FkF_k indicates SFT or GRPO eligibility, and cEVTc_{EVT} is 1.0 for non-executable/catastrophic injections, 0.4 for evaluable but invalid patches, and zero for valid edits without target regression.

Two-Stage Asynchronous Execution

The Injector generates bugs while the Student and Teacher evaluate previously generated tasks asynchronously. Complete Student-zero and mixed-outcome groups enter the Teacher queue for evaluation, and Teacher results determine task eligibility for SFT and GRPO.

Empirical Validation / Results

Main Results (SWE-bench Pro and TerminalBench 2.1)

Model / MethodRoundSWE-bench Pro Resolve rate (%) ↑TerminalBench 2.1 Success rate (%) ↑
Backbone and baselines without self-play
Qwen3-Coder-30B-A3B-Instruct–27.7 ± 1.311.0 ± 1.7
Qwen3.6-35B-A3B (Teacher reference)–38.5 ± 1.942.1 ± 2.4
SWE-MeM SFT+RL–31.1 ± 1.1–
Teacher Distillation †–30.8 ± 0.99.8 ± 2.2
Student-only self-play
Self-Play SWE-RL †028.2 ± 1.111.8 ± 1.7
128.6 ± 1.312.7 ± 1.6
229.1 ± 1.313.1 ± 2.1
Teacher-guided self-play
Self-Play SWE-RL + Teacher †028.4 ± 1.413.1 ± 1.7
129.2 ± 1.114.0 ± 1.3
229.9 ± 1.414.7 ± 1.9
SWE-Growth (Ours)028.5 ± 1.313.8 ± 2.2
130.2 ± 1.116.0 ± 1.9
231.7 ± 1.1 ∗17.2 ± 2.2 ∗

Notes: ∗ denotes significant improvement over Round-2 Self-Play SWE-RL + Teacher (paired t-test, p < 0.05). † Our implementation.

Learning Beyond Teacher Coverage

With a strong Teacher (Qwen3.6-35B-A3B), the Round-1 Student solves 20 tasks that neither the initial Student nor the Teacher solves (on 688 common tasks). Language-wise, gains exceed losses in all four languages (Go, Python, TypeScript, JavaScript).

With a weak Teacher (Qwen3.5-9B, which solves 192/731 tasks vs. 199 for the initial Student), the trained Student solves 208 tasks, exceeding both the initial Student (195) and the Teacher (192), including 25 tasks missed by both original models.

Ablation Results

RoundMethodSWE-bench Pro (%) ↑Terminal-Bench (%) ↑CCR (%) ↑CMR (%) ↓
Round 1SWE-Growth30.2 ± 1.116.0 ± 1.9100.00.0
vs. SFT only29.1 ± 0.911.3 ± 1.1——
vs. GRPO only28.5 ± 1.312.0 ± 1.3100.00.0
vs. No refresh28.7 ± 1.311.3 ± 0.958.115.1
vs. Fixed-pair gap28.2 ± 0.811.0 ± 1.129.442.4
Round 2SWE-Growth31.7 ± 1.117.2 ± 2.2100.00.0
vs. SFT only30.4 ± 1.613.7 ± 1.9——
vs. GRPO only31.4 ± 1.316.5 ± 1.5100.00.0
vs. No refresh31.0 ± 1.116.0 ± 1.378.59.5
vs. Fixed-pair gap30.8 ± 1.615.4 ± 2.144.344.9

Curriculum metrics:

CCRk=∣Ck∩Ek∣∣Ek∣,CMRk=∣Ck∖Ek∣∣Ck∣\text{CCR}_k = \frac{|C_k \cap E_k|}{|E_k|}, \quad \text{CMR}_k = \frac{|C_k \setminus E_k|}{|C_k|}

where EkE_k is the common candidate pool satisfying 0<s^stu,kcur(t)<10 < \hat{s}_{stu,k}^{cur}(t) < 1 and s^ref,k(t)>s^stu,kcur(t)\hat{s}_{ref,k}(t) > \hat{s}_{stu,k}^{cur}(t).

Key ablation findings:

  • SFT and GRPO are complementary: SFT-only performs better in Round 1 on SWE-bench Pro; GRPO-only approaches full method by Round 2
  • Refreshing task selection matters: "No refresh" (stale selections with fresh rollouts) misses newly eligible tasks (CCR drops to 58.1% in Round 1)
  • Fixed-pair gap fails: Using a gap between two frozen models (Qwen3.6 vs. Qwen3.5) instead of Teacher–current Student gap yields poor alignment (CCR 29.4%, CMR 42.4%)

Scaling Analysis

Source TasksSWE-bench Pro (best round)TerminalBench (best round)
10132.0%17.5%
40032.6%17.7%
100033.3%16.1%

Larger source-task pools support higher in-domain scores over more rounds, but cross-domain gains are not monotonic (1000 tasks yields lower TerminalBench than 400 tasks).

Theoretical and Practical Implications

Theoretical Contributions

  1. Task-level Teacher–Student gaps as curriculum signals: The paper demonstrates that gaps between a frozen Teacher and evolving Student provide a principled way to allocate supervision, targeting tasks where the Teacher demonstrates verified local strengths the Student has yet to acquire.

  2. Silent groups are learnable: The key theoretical insight is that all-fail groups (which provide zero group-relative advantage for GRPO) can still offer learning opportunities when a Teacher provides verified repairs, transforming them into useful SFT demonstrations.

  3. Complementarity of SFT and GRPO: The staged SFT→GRPO pipeline leverages Teacher demonstrations to bootstrap Student capabilities, then uses refreshed Student rollouts to identify tasks where the Student can now generate its own success/failure comparisons for reinforcement learning.

  4. Weak Teacher effectiveness: A Teacher with lower aggregate performance can still provide valuable task-level guidance, suggesting that task-level complementarity matters more than aggregate superiority.

Practical Implications

  1. Data efficiency: SWE-Growth achieves 31.7% on SWE-bench Pro from only 101 seed tasks, outperforming SWE-MeM SFT+RL (31.1%) which uses thousands of source training problems.

  2. Cross-domain generalization: The method improves TerminalBench 2.1 performance from 11.0% to 17.2% without training on TerminalBench tasks, while Teacher Distillation actually degrades cross-domain performance (11.0% → 9.8%).

  3. Curriculum design principles: Refreshing task selection against the evolving Student is critical—stale selections miss newly eligible tasks, and fixed model-pair gaps do not reliably track the current Student's needs.

Conclusion

SWE-Growth introduces a teacher-guided adaptive self-play framework that uses task-level Teacher–Student gaps to coordinate SFT, GRPO, and task generation. Starting from 101 seed issues, it achieves 31.7% on SWE-bench Pro and 17.2% on TerminalBench 2.1, outperforming Teacher-augmented self-play by 1.8 and 2.5 percentage points respectively. Results with a weaker Teacher highlight the value of task-level complementarity over aggregate capability.

Limitations and Future Directions

  • Teacher configurations: Limited set of Teacher configurations studied; effects of Teacher capability and demonstration format warrant further study
  • Gap estimation noise: Empirical gaps estimated from few rollouts (G=4G=4 or G=8G=8) are sensitive to sampling variability; incorporating uncertainty into task selection is future work
  • Selection bias: Selecting tasks based on frozen Teacher success may exclude tasks learnable through other supervision forms
  • Domain generalization: Evaluation covers repository-level code repair and terminal-based tasks; generalization to other domains remains to be established
  • Safety considerations: Automated bug injection could be misused; agent-generated code requires human review before deployment

Related papers