# FrogNano: Training a 4B Coding Agent via Online Task Synthesis

> A 4B coding agent rivals 32B-100B models on SWE benchmarks via policy-adaptive synthetic task synthesis and iterative RL, without distillation.

- **Source:** [arXiv](https://arxiv.org/abs/2609.07925)
- **Published:** 2026-10-12
- **Permalink:** https://picx.dev/p/5qfHeI
- **Whiteboard:** https://picx.dev/p/5qfHeI/image

## Summary

# FrogNano: Training a 4B Coding Agent via Online Task Synthesis

## Summary (Overview)

- **FrogNano** is a 4B-parameter coding agent that achieves competitive performance on software engineering (SWE) benchmarks, rivaling models 8–25× its size (32B–100B+), without distillation from larger models.
- The training recipe relies on **three key ingredients**: (1) **Leaf**, a lightweight Claude-Code-style harness with five typed tools; (2) **TaskPilot**, an online policy-adaptive task synthesis pipeline that generates tasks at the "learnability frontier" of the current policy; and (3) **iterative RL** using DPPO with a success-gated log-length penalty.
- FrogNano reaches **61.5%** on SWE-bench Verified, **37.6%** on SWE-bench Pro, **31.1%** on Terminal-Bench v2, and **47.3%** on PatchEval-Verified, using only ~1,500 synthetic tasks across 5 RL iterations.
- The harness choice is critical for compact models: switching from R2E-Gym to Leaf improves Qwen3.5-4B's SWE-bench Verified solve rate from 8.3% to 37.2%, while a larger model (MiniMax-M2.5) is unaffected (66.5% on both).
- The paper demonstrates that **online task synthesis calibrated to the current policy** outperforms training on real tasks filtered by the same learnability criterion (61.5% vs. 48.0% on SWE-bench Verified).

---

## Introduction and Theoretical Foundation

### Motivation
Most progress in SWE agents has been driven by frontier models behind proprietary APIs or open-weight models with tens of billions of parameters. These systems are expensive to serve and difficult to deploy locally. The authors ask: **can a substantially smaller model (4B) become a capable repository-level coding agent?**

### Key Challenges Addressed

1. **Data**: Curating executable SWE tasks from real development histories is costly. Existing approaches include mining issue-PR pairs (SWE-Gym, SWE-rebench), commit-derived tasks (R2E-Gym), and synthetic generation (SWE-smith, BugPilot). However, none use **current-policy feedback** to adapt task generation to the evolving learner.

2. **Harness**: Compact models struggle with complex agent interfaces. Under R2E-Gym, Qwen3.5-4B fails to terminate successfully in ~96% of trajectories, often not producing the required submission action.

3. **RL**: Training requires executable tasks matched to the current policy's capabilities—tasks never solved provide no signal, while saturated tasks provide little relative signal.

### Theoretical Basis
The work builds on:
- **Adaptive curricula** from RL literature: GoalGAN, POET, PAIRED, PLR, and ACCEL all use current-policy outcomes to shape task distributions.
- **Group-relative policy optimization** (DPPO/GRPO) for stable RL training.
- The insight that **task difficulty should be calibrated to the learner's current ability**, not estimated intrinsically.

---

## Methodology

### 1. Leaf Harness

A lightweight coding-agent harness inspired by Claude-Code-style tool conventions:

- **Five typed tools**: `read`, `write`, `edit`, `glob`, `bash`
- **Simple termination rule**: a response without a tool call is treated as the final answer
- Omits planning modes, permission dialogues, reminders, and auxiliary tools

The harness is used for both evaluation and RL training. During training, model-generated tokens (reasoning, text, tool calls) are included in the loss; task prompts and tool outputs are masked.

### 2. TaskPilot: Policy-Guided Task Synthesis

**Pipeline** (Figure 4):
1. Start from real repository snapshots (SWE-rebench or Scale-SWE)
2. A task-generation model produces: problem statement, gold patch, hidden fail-to-pass (F2P) tests
3. Validate executability: F2P tests fail on original snapshot, pass after gold patch; P2P suite stable
4. Evaluate candidates with current-policy rollouts

**Calibration**: For each candidate, collect $N$ trajectories. Estimate resolve rate:

$$\hat{p}_{\text{policy}} = \frac{1}{N} \sum_{j=1}^{N} R_j$$

where $R_j \in \{0, 1\}$ indicates whether trajectory $j$ passes all grading tests.

**Admission criteria**:
- Broad learnable region: $0 < \hat{p}_{\text{policy}} < 1$
- Iterations 1–4 target: $\hat{p}_{\text{policy}} = 0.5$
- Revised iteration 5 targets: $0 < \hat{p}_{\text{policy}} \leq 0.5$

Candidates outside the target are refined (e.g., adjusting problem-statement specificity) and re-evaluated in a generate–evaluate–refine–re-evaluate loop.

**Iterative procedure**:
$$\pi^{(0)} \xrightarrow{\text{TaskPilot}} \text{task batch} \xrightarrow{\text{RL}} \pi^{(1)} \xrightarrow{\text{TaskPilot}} \cdots \xrightarrow{\text{RL}} \pi^{(5)} = \text{FrogNano}$$

### 3. RL Training Recipe

**DPPO with asymmetric importance sampling**:

For each task $i$, sample 8 trajectories and standardize rewards within the group:

$$A_{ij} = \frac{R_{ij} - \bar{R}_i}{\text{std}(R_i) + 10^{-6}}$$

Groups with zero advantage are discarded. For token-level importance correction:

$$p_t = \pi_\theta(a_t \mid s_t), \quad q_t = \pi_{\text{roll}}(a_t \mid s_t), \quad \rho_t = \frac{p_t}{q_t}, \quad \Delta p_t = p_t - q_t$$

A positive-advantage token is dropped when $\Delta p_t > 0.2$; a non-positive-advantage token is dropped when $\Delta p_t < -0.2$. The policy objective:

$$\mathcal{L}_{\text{policy}} = -\frac{1}{|\mathcal{T}|} \sum_{t \in \mathcal{K}} A_t \rho_t$$

No reference-model KL loss, entropy bonus, or auxiliary KL term is used.

**Efficiency reward** (success-gated log-length penalty):

$$R(N) = \begin{cases} 1, & n \leq N, \\ 1 - \alpha \ln(n/N), & n > N. \end{cases}$$

where $N$ is the allowed free tokens and $\alpha$ controls penalty strength. The penalty applies only to completed successful trajectories. Truncated-but-correct trajectories receive partial reward of 0.5.

**Infrastructure**: Single node with 8× NVIDIA B200 GPUs; 2 GPUs train (context parallelism 2), 6 GPUs generate rollouts. Each update: 32 task groups × 8 trajectories = 256 trajectories.

---

## Empirical Validation / Results

### Main Results (Figure 2)

| Benchmark | FrogNano (4B) | Comparable Larger Models |
|---|---|---|
| SWE-bench Verified | **61.5%** | Matches 32B–100B+ models |
| SWE-bench Pro | **37.6%** | Competitive with 6–8× larger models |
| Terminal-Bench 2.0 | **31.1%** | — |
| PatchEval-Verified | **47.3%** | — |

### Iterative Gains (Figure 1)

| Iteration | SWE-bench Verified |
|---|---|
| Base (Qwen3.5-4B) | 39.4% |
| Iter 1 | 49.1% |
| Iter 2 | 53.1% |
| Iter 3 | 56.7% |
| Iter 4 | 59.1% |
| Iter 5 (FrogNano) | **61.5%** |

**Comparison baselines**:
- ~300 real SWE-rebench tasks filtered by same 4B learnability criterion: **48.0%**
- 1,500 real tasks (SWE-rebench + Scale-SWE): lower than synthetic-only

### Harness Impact
- R2E-Gym → Leaf: 8.3% → **37.2%** for Qwen3.5-4B
- MiniMax-M2.5: 66.5% on both harnesses (unaffected)

### Analysis Highlights

**Pass@k**: The gap between FrogNano and base model stays static as $k$ increases, suggesting iterative RL **pushes the capability boundary** rather than distilling high pass@k into pass@1.

**Verifier test-time scaling**: Round-robin tournament with 3 candidates achieves **62.8%** on SWE-bench Verified vs. 61.4% for single verifier call, 60.4% for pass@short.

**Consolidation**: Merging behaviors across policies (e.g., reinforcing multi-tool-call behavior) improves SWE-bench Verified to **62.3%** with pass@3 of 72.3%.

**Compaction**: Summary compaction recovers full 131K context performance at 64K token budget; helps most at low token budgets (16K, 32K).

**Cross-harness generalization**: FrogNano evaluated in mini-SWE-agent (single bash tool) improves from 43.8% (base) to **56.4%**, only 5.2% below Leaf performance.

**Learning dynamics**: Solve rate improves in all five iterations (+11.7, +3.4, +5.6, +3.0, +5.4 points respectively). Policy entropy falls most sharply in Iter 1 (0.415→0.265) then varies within a bounded range.

**Failure analysis**: 90.8% of failures are reasoning gaps (not premature termination). Dominant causes: wrong root cause/layer (38.8%), misreading specification (31.5%), API misunderstanding (14.1%).

**Reward hacking**: 21.3% of trajectories flagged by regex analysis; 2.5% confirmed as attempts (Fleiss' κ = 0.746). **All attempts blocked** by scaffold/infrastructure; 0 hack-related solves.

### Task Distribution Evolution (Iteration 5 vs. Iteration 4)

| Metric | Iter 4 | Iter 5 |
|---|---|---|
| Problem-statement length (words) | 227.0 | 116.6 |
| Test-patch churn (lines) | 312.3 | 194.7 |
| Test/gold-patch churn ratio | 14.4 | 6.4 |
| Explicit requirement items | 0.51 | 0.04 |
| Problem-statement/test coverage | 34.3% | 37.6% |
| Bug-fix share | ~85% | 40.7% |
| Feature request share | ~17% | 24.0% |
| Refactor share | ~15% | 17.0% |
| Performance optimization share | — | 12.4% |

---

## Theoretical and Practical Implications

### Theoretical Implications

1. **Distillation-free capability acquisition**: A 4B model can acquire substantial repository-level coding capabilities through RL over policy-grounded synthetic tasks alone, without imitating larger models' trajectories. This challenges the assumption that small models require teacher distillation for complex agentic tasks.

2. **Task distribution co-evolves with policy**: The closed-loop design (TaskPilot generates tasks → RL trains policy → new policy generates next tasks) demonstrates that adaptive curricula calibrated to the learner's frontier sustain learning across multiple iterations, whereas static datasets saturate.

3. **Harness as a first-class training component**: The interface between model and environment is not neutral—it can be a major bottleneck for compact models. Leaf's simplicity (5 tools, natural termination) unlocks capability that a more complex harness suppresses.

4. **Learnability calibration signal**: The empirical, policy-relative signal $\hat{p}_{\text{policy}}$ (resolve rate under current policy) is more useful than intrinsic difficulty estimates. Refinement adjusts semantic clarity and solution-localizing information to move tasks toward the target region.

### Practical Implications

1. **Accessible coding agents**: 4B models can run on minimal hardware, enabling local deployment, repeated experimentation, and research on agent scaffolds, context management, and online data generation.

2. **Cost efficiency**: FrogNano is competitive with models 6–8× its size on SWE-bench Pro while costing a quarter as much to serve.

3. **Synthetic data reduces curation burden**: Training on purely synthetic tasks at the learnability edge matches (and exceeds) training on real data subsets filtered by costly rollouts, eliminating the need for expensive real-task mining.

4. **Recipe transferability**: The recipe (harness + adaptive synthesis + iterative RL) is potentially applicable to other domains with executable validation signals.

---

## Conclusion

### Main Takeaways

- **FrogNano** demonstrates that a 4B coding agent can rival 32B–100B+ models on SWE benchmarks without distillation, using only ~1,500 synthetic tasks.
- The three key ingredients are: (1) a **lightweight harness** (Leaf) matched to the model's capabilities, (2) **online task synthesis** (TaskPilot) calibrated to the policy's learnability frontier, and (3) **iterative RL** with DPPO and efficiency regularization.
- The paper provides a practical platform for coding-agent research at small scale, making end-to-end experimentation tractable.

### Future Directions

- How much further can the 4B model improve with additional training iterations?
- Can small models handle more specialized and harder coding tasks?
- How to further improve test-time scaling (verifiers, tournaments)?
- How to enhance context management and inference efficiency at small scale?
- Joint training of task generator and solver in a self-play manner (as opposed to TaskPilot's separate generator).

### Limitations

- Evaluated on English-language, Python-heavy repositories only
- Generalization to non-English tasks, other frameworks, unclear requirements, or projects without reliable tests is unestablished
- Results may not hold with other tool interfaces, prompts, or budgets
- Image/video capabilities of the base model (Qwen3.5-4B) were not post-trained or evaluated
- Test-based rewards cover only part of program behavior; passing patches can be incorrect or insecure
- Safety considerations: agents should run in isolated environments; all patches need human review

---

_Markdown view of https://picx.dev/p/5qfHeI, served by PicX — AI-generated visual whiteboard summaries of research papers._