Summary (Overview)

• FastContext is a dedicated exploration subagent that separates repository exploration from the main agent's problem-solving, reducing token consumption and improving resolution rates in coding agents.
• The system uses specialized exploration models (4B–30B parameters) trained via supervised fine-tuning (SFT) and reinforcement learning (RL) with task-grounded rewards.
• Integrating FastContext into Mini-SWE-Agent improves end-to-end resolution rates up to 5.5% while reducing main-agent token consumption up to 60% across three benchmarks (SWE-bench Multilingual, SWE-bench Pro, SWE-QA).
• FastContext exposes only three language-agnostic read-only tools (READ, GLOB, GREP) and returns compact file-and-line citations as focused context.
• Standalone evaluation shows trained FastContext checkpoints reach 73.71 file-level F1 and 60.35 module-level F1, outperforming existing localization baselines.

---

Introduction and Theoretical Foundation

Background

Coding agents powered by Large Language Models (LLMs) have achieved strong results on software engineering tasks, but repository exploration remains a major bottleneck. The authors observe that:

• Locating relevant code consumes substantial token budget and pollutes the agent's context with irrelevant snippets.
• In most agents, the same model explores the repository and solves the task, leaving exploratory reads and searches in the solver's history.
• Benchmarks like SWE-bench and SWE-QA require navigating large, multi-file codebases, making exploration a critical factor in both task success and inference efficiency.

Preliminary Analysis

The paper presents a detailed analysis of 300 GPT-5.4-high trajectories on SWE-bench Multilingual:

• Reading and searching dominate the full trajectory, accounting for 56.2% of all tool-use turns and 46.5% of the main agent's total tokens.
• The agent starts editing at turn 8.47 on average, with a median of 6 sequential exploration turns and 15.5 exploration tool calls before the first edit.
• Unresolved trajectories are associated with more pre-edit exploration turns than resolved ones (8.34 vs. 6.67 turns on average).

This motivates the design of FastContext as a reusable exploration component that can be delegated to a subagent performing parallel search and returning compact evidence.

---

Methodology

3.1 FastContext Subagent Architecture

FastContext is a runtime delegation mechanism that separates repository exploration from solving:

• The main agent delegates exploration to the explorer, which returns evidence rather than a patch.
• The subagent exposes only three language-agnostic tools:
  • READ: reads line-numbered file contents
  • GLOB: discovers paths via pattern matching
  • GREP*: regex search over repository text
• Multiple tool calls in the same turn are executed in parallel.
• The output contract is a compact final answer block containing file paths and line ranges:

<final_answer>
/src/router.py:42-58 (Router definition)
/tests/test_router.py:101-119
</final_answer>

3.2 Policy Initialization with Supervised Fine-Tuning

The initial exploration policy is trained via SFT with 2,954 filtered examples from Sonnet 4.6 exploration traces, split into three sources:

1. parallel_toolcalls (990 examples): targets broad first-turn search with nonredundant parallel tool calls.
2. multiturn_traj (983 examples): targets multi-turn evidence gathering with full trajectories.
3. linerange (981 examples): targets precise citation generation with narrow <final_answer> blocks.

The SFT objective is:

$$\mathcal{L}_{\mathrm{SFT}} = - \frac{1}{|\mathcal{D}_{\mathrm{sft}}|} \sum_{(x, y) \in \mathcal{D}_{\mathrm{sft}}} \sum_{t = 1}^{|y|} m_{t} \log p_{\theta}(y_{t} \mid x, y_{<t})$$

where $m_t$ masks out non-assistant tokens.

3.3 Policy Refinement with Reinforcement Learning

The explorer is refined with task-grounded RL using a 400-prompt set derived from issue-resolution tasks with reference patches. The reward function is:

$$R = \underbrace{F_1(P_f, G_f) + F_1(P_l, G_l)}_{\text{task outcome}} + \underbrace{r_{\text{parallel}}}_{\text{parallel function call}} - \underbrace{r_{\text{format}}}_{\text{penalty}}$$

Where:

• $G_f$ and $G_l$: target file and line sets from the reference patch
• $P_f$ and $P_l$: predicted sets parsed from the model's final citations
• $r_{\text{parallel}}$: small bonus for bounded multi-call exploration
• $r_{\text{format}}$: penalty for empty, overly long, malformed, or excessive-fan-out outputs

The format penalty is:

$$r_{\text{format}} = 10 \cdot \mathbf{1}[n_C < 1 \lor n_C > 20 \lor b_C > 0 \lor p_{\max} > 6]$$

and the bounded-parallelism bonus is:

$$r_{\text{parallel}} = \mathbf{1}[3 < p_{\max} \leq 6]$$

The models are optimized with GRPO (Group Relative Policy Optimization), initialized from the SFT checkpoint.

---

Empirical Validation / Results

4.2 End-to-End Results

Table 1: End-to-end performance and efficiency across three benchmarks

Key findings:

• FastContext improves end-to-end accuracy for every main agent and benchmark.
• Largest gains on SWE-bench Pro: GPT-5.4 improves from 46.0 to 51.5, GLM-5.1 from 17.5 to 22.5.
• Largest token savings on SWE-QA: 60.3% for GPT-5.4.
• 4B-RL can outperform the larger 30B-SFT explorer in several settings.

4.4 Standalone Exploration Quality

Table 2: Standalone exploration quality on SWE-bench Verified

Trained FastContext checkpoints reach 73.71 file-level F1 and 60.35 module-level F1, compared with 68.57 and 50.88 for the best non-FastContext rows.

Cost Audit

Table 3: Token and cost audit for GPT-5.4 SWE-bench Multilingual run

The explorer accounts for only 2.1% of the augmented total cost.

---

Theoretical and Practical Implications

Theoretical Contributions

1. Modular view of coding agents: The paper demonstrates that repository exploration can be treated as a first-class, trainable component rather than an implicit cost inside monolithic solver trajectories.

2. Separation of concerns: By decoupling exploration from solving, FastContext shows that specialized small models can effectively handle navigation tasks, enabling collaboration between smaller specialized models and stronger main agents.

3. Task-grounded RL effectiveness: The results show that a 4B-parameter model trained with RL can outperform a 30B-SFT model, suggesting that task-grounded optimization is more important than raw model scale for exploration tasks.

Practical Implications

1. Token efficiency: FastContext reduces frontier-model token consumption by up to 60%, directly translating to inference cost savings.

2. Deployment feasibility: The 4B explorer is designed for local serving, eliminating per-token API costs while maintaining performance.

3. Cleaner context: The main agent receives focused file-line evidence rather than noisy exploratory context, improving both resolution rates and reasoning quality.

---

Conclusion

Main Takeaways

• Repository exploration should be separated from solving and handled by specialized models.
• FastContext's SFT + RL training recipe effectively bootstraps compact exploration models from reference trajectories.
• The architecture enables parallel tool use and returns compact file-and-line citations that are directly consumable by the main agent.

Future Directions

1. Broader integration: Adapting FastContext to other coding-agent frameworks with different tool interfaces and orchestration mechanisms.
2. Smaller main models: Exploring FastContext paired with 30B-class coding agents.
3. Smaller explorers: Investigating whether the same recipe supports 1.7B or 0.6B parameter models.
4. Benchmark decontamination: Addressing potential overlap between public benchmarks and frontier-model training data.

Limitations

• Evaluation is currently limited to Mini-SWE-Agent as the main-agent scaffold.
• Experiments focus on strong frontier models (GPT-5.4, GLM-5.1, Kimi-K2.6).
• The smallest explorer trained has 4B parameters; smaller variants remain to be explored.

The paper concludes that repository navigation can be optimized and evaluated separately from patch generation or answer synthesis, encouraging future systems to expose exploration as an explicit interface with clearer context boundaries.