# SWE-Trail: History-Guided Trajectory Collection for Coding Agent Training

> SWE-Trail uses prior attempt history to guide where to collect coding trajectories, improving student model resolution by 2.33 points on SWE-bench Multilingual without exposing history to the agent.

- **Source:** [arXiv](https://arxiv.org/abs/2611.02074)
- **Published:** 2026-11-07
- **Permalink:** https://picx.dev/p/Vm25T5
- **Whiteboard:** https://picx.dev/p/Vm25T5/image

## Summary

## Summary (Overview)

- **SWE-Trail** is a trajectory collection method for coding agent training that uses prior attempts (history) on the same issue to guide *where* additional collection effort should be spent, without exposing that history to the coding agent's context.
- The key insight is that **history should guide collection allocation, not agent reasoning** — exposing history as advice to the agent creates a train-test mismatch that hurts downstream performance.
- SWE-Trail compiles history into **detectors** that identify states warranting additional exploration, then compares candidate next actions by briefly executing them from the same checkpoint and selecting the route better supported by repository evidence.
- On a 124B student (Ling-3.0-flash-base), SWE-Trail improves mean resolution over independent sampling by **2.33 percentage points on SWE-bench Multilingual** and **1.60 points on SWE-bench Pro**.
- Ablation studies show that the collection pipeline without history (H0) produces more resolved trajectories but yields only **0.11 points** improvement on Multilingual, demonstrating that collection success alone does not explain downstream gains.

---

## Introduction and Theoretical Foundation

### Background

Resolving repository-level issues requires coding agents to explore repositories, modify code, and validate changes. These interaction trajectories are increasingly used as post-training data, making the **collection process** directly influential on the value of the data for training.

### The Problem with Independent Sampling

A common approach is to sample multiple trajectories independently per issue and select those that pass tests or a judge. However, the authors identify two key problems:

1. **Diminishing returns**: Later independent samples become increasingly redundant, often revisiting repair targets and behaviors already explored.
2. **Missed allocation signals**: Earlier trajectories contain evidence (e.g., disagreements between successful and failed attempts) that could guide where subsequent collection effort should be spent.

### The Train-Test Mismatch Concern

A natural way to use history is to expose it to the coding agent as advice. However, training on such trajectories makes the model rely on advice that is **unavailable at test time**, creating a mismatch that may reduce downstream performance.

### Central Research Question

> *How can history guide where collection effort is spent while the coding agent decides what to generate from its ordinary context?*

### Theoretical Foundation

The paper establishes two empirical motivations:

**1. Diminishing coverage gains from independent rollouts:**
- With one attempt per issue, 58.4 issues are solved in expectation.
- Increasing from 1 to 2 attempts adds 13.1 solved issues; increasing to 3, 4, and 5 adds only 8.0, 5.1, and 3.4 more, respectively.
- One attempt already covers 90.0% of modified files on stable-solved issues, 73.2% on mixed issues, and 80.1% on never-solved issues.

**2. History provides allocation signals:**
- Audits found decision points in 26 issues (GPT-5.6-Sol) and 31 issues (GLM-5.1) where successful and failed attempts diverge meaningfully.
- For never-solved histories, the mean pairwise Jaccard overlap of modified files across failed attempts is **0.841**, with 93 of 198 issues having a perfect overlap score of 1.0.

---

## Methodology

### SWE-Trail Architecture

SWE-Trail operates in two stages:

#### Stage 1: Offline Compilation

For issue $i$, with issue statement and repository snapshot $x_i$ and history $H_i$ (prior attempts):

- The **compiler** analyzes $(x_i, H_i)$ to extract allocation signals (decisions supported by successful attempts, differences across repair routes, recurring failures).
- It runs **bounded read-only repository checks** to resolve open questions (without modifying code or accessing reference patches).
- It stores evidence in a private **acquisition record** $R_i$.
- It builds a **detector** $D_i$ whose rules recognize states from the rollout's actions and observations (not fixed step numbers).
- It defines a **local comparison objective** $u_i(d)$ for each detector state $d$.
- It sets a **sampling-temperature schedule** $\lambda_i$ for candidate generation (mixed and never-solved histories use higher temperatures).

#### Stage 2: Online Collection

When a rollout reaches a state $d$ recognized by $D_i$:

1. **Candidate proposal**: The coding agent proposes candidate next actions from its ordinary context (history, $R_i$, and $u_i(d)$ are *not* added to the context).
2. **Candidate grouping**: Equivalent proposals are grouped; the default candidate (from ordinary temperature) is kept, and executable alternatives are shortlisted.
3. **Bounded comparison**: At most one alternative is chosen using $\phi_{R_i}$ and $u_i(d)$. Two isolated workspaces are restored from the same checkpoint, and both candidates run for the same short horizon.
4. **Route selection**: The two executed suffixes are presented in shuffled order (identity-blind). SWE-Trail prefers a route only when clear evidence shows better progress under $u_i(d)$; otherwise, it keeps the default.
5. **Trajectory completion**: The coding agent continues from the selected workspace until the rollout ends. Only the selected route is kept; the unselected suffix and private collector events are removed.

Each trajectory performs **at most one comparison** to avoid expanding into a multi-level search tree.

### Experimental Setup

- **Collection issues**: 300 SWE-rebench V2 issues, five trajectories per issue (1,500 total).
- **Collection model**: GLM-5.1 with OpenHands scaffold.
- **Primary student**: 124B Ling-3.0-flash-base, full-parameter SFT on 128 H800 GPUs, three independent training runs.
- **Second student**: Qwen3-14B-Base.
- **Evaluation**: SWE-bench Multilingual (300 tasks) and SWE-bench Pro (731 tasks).
- **Conditions**: SeedBank (independent baseline), H5 (five SeedBank attempts as history), H0 (no history), H10 (ten attempts as history), plus controlled adaptations of prior work.

---

## Empirical Validation / Results

### Main Results (Table 1)

| Condition | Multilingual Mean | SWE-Pro Mean |
|---|---|---|
| SeedBank independent rollouts | 63.33 | 41.22 |
| SWE-Gym (outcome-conditioned transform) | 64.00 (+0.67) | 40.86 (−0.36) |
| SWE-TRACE (critical-path reduction) | 64.44 (+1.11) | 40.77 (−0.46) |
| SWE-Lego (strict tool-error masking) | 64.56 (+1.22) | 41.63 (+0.41) |
| SWE-Exp (agent-visible experience) | 62.00 (−1.33) | 38.44 (−2.78) |
| SWE-TRAIL-H0 | 63.44 (+0.11) | 41.95 (+0.73) |
| SWE-TRAIL-H5 (random detection) | 64.22 (+0.89) | 41.54 (+0.32) |
| SWE-TRAIL-H5 (random retention) | 64.78 (+1.44) | 41.63 (+0.41) |
| **SWE-TRAIL-H5** | **65.67 (+2.33)** | **42.82 (+1.60)** |

**Key findings:**
- Every H5 run exceeds every SeedBank run on both benchmarks (exact one-sided permutation test: $p = 0.05$).
- **SWE-Exp** (agent-visible history) *falls below* SeedBank by 1.33 points on Multilingual and 2.78 on SWE-Pro, confirming the train-test mismatch concern.
- **Random detection** lowers mean resolution by 1.44 points (Multilingual) and 1.28 points (SWE-Pro), validating the history-guided choice of comparison location.
- **Random retention** lowers mean resolution by 0.89 and 1.19 points, validating the use of execution evidence for route selection.

### History Scale and Cost (Table 2)

| Condition | History | Resolved | Acquisition Tokens | Total Tokens | Multilingual | SWE-Pro |
|---|---|---|---|---|---|---|
| H0 | 0 | 329 | 164M† | ≥1.845B† | 63.44 | 41.95 |
| H5 | 5 | 345 (+16) | 307M (+143M†) | 1.914B (+69M†) | 65.67 (+2.22) | 42.82 (+0.87) |
| H10 | 10 | 341 (+12) | 510M (+346M†) | 2.160B (+315M†) | 65.89 (+2.44) | 42.95 (+1.00) |

- H10 costs **66.0% more acquisition tokens** than H5 but adds only 0.22/0.14 points.
- H5 uses at most **3.7% more total tokens** than H0.

### Selection vs. Collection (Table 3)

| Dataset | Multilingual | SWE-Pro |
|---|---|---|
| SeedBank | 63.33 | 41.22 |
| Top-5-of-10 | 64.11 | 41.86 |
| **SWE-TRAIL-H5** | **65.67** | **42.82** |
| Top-5-hybrid | 66.22 | 43.14 |

- **Top-5-of-10** (selecting from more independent rollouts) remains below H5 by 1.56 points on Multilingual, showing that post-hoc selection alone cannot match SWE-Trail.
- **Top-5-hybrid** (selection applied to combined SeedBank + H5 candidates) exceeds H5 by 0.56/0.32 points, showing selection and SWE-Trail are **complementary**.

### Collection Outcomes (Table 4)

| Method | Resolved | Solved Issues | Verified Positive |
|---|---|---|---|
| SeedBank | 292 | 88 | 263 |
| SWE-TRAIL-H0 | 329 | 92 | 272 |
| SWE-TRAIL-H5 | 345 (+16) | 98 (+6) | 285 (+13) |

**Critical finding**: H0 produces 37 more resolved trajectories than SeedBank but improves the student by only 0.11 points on Multilingual. H5 produces only 16 more resolved trajectories than H0 but improves the student by 2.22 points. **Collection success does not explain student gains.**

### Generalization to Qwen3-14B-Base

H5 improves mean resolution over SeedBank by **3.11 points on Multilingual** and **2.42 points on SWE-Pro**, confirming gains extend across student models.

### Mechanism Analysis

- History leads to more comparisons: candidate sampling occurs in 1,147 of 1,500 rollouts (H5) vs. 664 (H0).
- Candidate diversity remains similar (~90% with ≥2 distinct actions).
- Better local progress is observed in 40.7% of H5 rollouts vs. 28.7% of H0.
- **Most gains come from mixed histories**: 51 mixed-history issues contribute 13 of H5's 16 additional resolved trajectories and 5 of 6 additional solved issues.

---

## Theoretical and Practical Implications

### Theoretical Implications

1. **Allocation vs. Generation**: The paper provides evidence that the value of history lies in guiding *where* collection effort is spent, not in guiding *what* the agent generates. This reframes how prior attempts should be used in trajectory collection.

2. **Collection Success ≠ Training Value**: The H0 vs. H5 contrast (similar collection outcomes, very different student gains) demonstrates that the *process* of collection shapes what the model learns, not just the final outcome.

3. **Train-Test Mismatch**: SWE-Exp's degradation shows that agent-visible advice creates trajectories that depend on information unavailable at test time, reducing their supervisory value.

4. **Diminishing Returns on History**: H10's marginal gains (0.22/0.14 points) over H5 at 66% more acquisition cost suggest that beyond a certain point, additional history provides little new allocation signal.

### Practical Implications

1. **Data Collection Design**: Practitioners should design collection pipelines that use prior attempts to target exploration effort, rather than relying solely on independent sampling or post-hoc selection.

2. **Complementarity with Selection**: SWE-Trail can be combined with verifier-based selection (Top-5-hybrid) for additional gains, showing that collection and curation are complementary.

3. **Cost Efficiency**: H5 achieves most of the benefit at only 3.7% more total tokens than H0, making it a cost-effective improvement over independent sampling.

4. **Avoid Agent-Visible History**: The SWE-Exp results caution against exposing history to agents during collection if the trained model will not have access to that history at test time.

---

## Conclusion

SWE-Trail demonstrates that prior attempts are valuable not only for what they contain, but for guiding where new collection effort should be spent. The method:

1. **Compiles history into detectors** that identify states warranting additional exploration.
2. **Keeps history outside the coding agent's context**, avoiding train-test mismatch.
3. **Compares candidate actions by brief execution** from the same checkpoint, using repository evidence for selection.

With fixed training setup, SWE-Trail improves the 124B student's mean resolution over independent sampling by **2.33 points on SWE-bench Multilingual** and **1.60 points on SWE-bench Pro**, with gains also appearing for Qwen3-14B-Base. The H0 comparison shows collection success alone does not explain the gains, and randomization of either collection decision (where to compare, which route to keep) lowers performance.

**Future directions** include:
- Testing other collection models and scaffolds.
- Updating history across collection rounds.
- Allocating effort differently across issues.
- Extending beyond coding to environments supporting state restoration (e.g., web navigation).

---

_Markdown view of https://picx.dev/p/Vm25T5, served by PicX — AI-generated visual whiteboard summaries of research papers._