Summary (Overview)

• SWE-Stream is a scalable system for constructing executable software engineering (SWE) training data from real-world development histories, combining progressive task construction with elastic streaming execution.
• The system processes 203,905 initial PR candidates to produce 80,874 validated tasks from 10,728 repositories across six programming languages (Python, JavaScript, TypeScript, Java, Go, C++).
• Fine-tuning Qwen3-30B-A3B-Thinking-2507 on 118,688 successful trajectories achieves 69.0%, 49.3%, and 45.42% resolve rates on SWE-bench Verified, Multilingual, and Pro, respectively.
• In a controlled comparison with approximately matched training budgets (~160M tokens), SWE-Stream achieves 45.6% resolve rate on SWE-bench Verified, outperforming R2E-Gym (43.0%), SWE-smith (42.8%), and Scale-SWE (43.6%).
• The streaming construction strategy delivers 1.45× the accepted-task throughput of batch construction and accepts 13.2% more tasks than fixed-limit streaming.

---

Introduction and Theoretical Foundation

Background and Motivation

Software engineering agents solve repository-level tasks by modifying code, executing tests, and iteratively refining solutions through feedback. Training these agents requires executable tasks that combine four critical components:

1. A meaningful problem description
2. A reproducible environment
3. A reference change (repair patch)
4. Validation tests capable of distinguishing correct repairs

Key Challenges

Development histories provide a natural source of supervision but are rarely training-ready due to:

• Incompatible dependencies — historical dependencies may no longer resolve or build
• Insufficient test coverage — tests may not adequately distinguish correct from incorrect repairs
• Inconsistent artifacts — descriptions, patches, and validation artifacts may not reflect the same intended behavior

Limitations of Existing Approaches

Key insight: Heterogeneous development histories require candidate-specific reconstruction, completion, and validation. Candidates become ready at different times, so cohort-wide stage barriers delay those that could otherwise advance. Resource demands vary, making uniform per-task limits poorly suited to some workloads.

---

Methodology

System Architecture

SWE-Stream operates through two complementary designs:

1. Progressive Task Construction

The pipeline proceeds through five stages, where stage outcomes determine whether each candidate advances, receives targeted remediation, or is rejected:

1. Candidate Selection — Rule-based checks and an LLM judge screen repositories and merged PRs for coherent, self-contained objectives verifiable through execution. Documentation-only changes, formatting-only edits, automated dependency upgrades, and bundled unrelated objectives are excluded.

2. Environment Reconstruction — An environment agent reconstructs the base revision in an isolated sandbox, selecting runtimes and dependencies from version constraints, lock files, and CI configurations. Reuses previously constructed Dockerfiles when available.

3. Test Suite Preparation — The test agent evaluates PR and repository tests, classifying them into:

   • Fail-to-Pass (F2P): tests that fail before the repair and pass afterward
   • Pass-to-Pass (P2P): tests that pass in both states (regression checks)

   Tests are generated only for remaining coverage gaps, with instructions to exercise public behavior and avoid assertions tied to the reference repair.

4. Independent Execution Validation — The validator runs candidate tests in a fresh sandbox pair (pre-fix and post-fix), determining final F2P and P2P sets independently of the test agent's trials. A candidate passes only with ≥1 F2P test and P2P tests covering affected modules.

5. Consistency and Leakage Review — A dedicated agent checks whether description, repair, and tests address the same requirement, whether the test patch contains repair logic, and whether F2P transitions reflect intended behavior. Also flags repair instructions or solution code disclosed in the description.

2. Elastic Streaming Execution

The scheduler coordinates stages across candidates under a shared resource budget (CPU, memory, storage, network):

• Asynchronous stage transitions: No cohort-wide barrier — candidates proceed at different rates and may occupy different stages simultaneously
• Repository-level resource feedback: Adaptive limits based on observed usage:
  • Memory-limit OOM → retry with higher memory limit
  • Sustained high CPU load → raise repository–stage CPU limit for later tasks
• Admission control: Regulates intake using CPU, memory, and network pressure; reduces admissions under sustained pressure and prioritizes work in progress

Key Parameters

• Default sandbox limits: 4 CPU cores, 8 GiB memory, 16 GiB storage
• Max attempts per stage: 3 per candidate
• Max reviews: 3 failed reviews before candidate is discarded
• Max validated tasks per repository: 20 (to broaden coverage)

---

Empirical Validation / Results

Dataset Statistics

Stage Success Rates (share of stage entrants that ultimately passed):

Task Provenance:

• Original (57,943 tasks): description and all retained validation tests come from development records
• Augmented (22,931 tasks): any artifact is generated

Patch and Test Statistics

Large-Scale Training Results

Resolve rates (%) on SWE-bench benchmarks (mean over 3 runs, deterministic decoding):

Key comparisons with published results:

• Outperforms Open-SWE-Traces on Pro by +8.6 points (45.42 vs 36.8)
• Outperforms Orchard-SWE (SFT) on Verified by +4.7 points (69.0 vs 64.3)
• Outperforms SWE-HERO-32B on Multilingual by +5.2 points (49.3 vs 44.1)
• With 3B activated parameters, beats 72B dense OpenSWE by +3.0 points on Verified
• Exceeds SFT+RL scores of SWE-MeM and Granite 4.2 on all three benchmarks

Controlled Data Comparison (~160M tokens)

Notably, SWE-Stream achieves this with fewer trajectories (3,616 vs 4,224–5,923) but the highest mean token length (45.53K) and agent turn count (41.29), indicating longer, more complex repository interactions.

Construction Efficiency

• SWE-Stream reaches 500 accepted tasks 43.9% sooner than Batch and 17.6% sooner than Fixed
• Acceptance yield: 67.0% (SWE-Stream) vs 59.2% (Fixed)
• All 78 additional accepted tasks (vs Fixed) previously failed with memory-limit OOM, validating the adaptive resource policy
• CPU efficiency: 6.21% fewer CPU core-hours per accepted task than Fixed

---

Theoretical and Practical Implications

System Design Insights

1. Progressive validation beats batch processing: Using stage outcomes to decide advancement, remediation, or rejection—rather than applying uniform cohort-wide barriers—substantially improves throughput and yield.

2. Adaptive resource allocation is critical: The 13.2% acceptance improvement over fixed-limit streaming demonstrates that uniform per-task resource limits are poorly suited to heterogeneous repository workloads. Memory-limit OOM recovery alone accounts for all 78 additional accepted tasks.

3. Data quality scales with validation rigor: The combination of independent execution validation (fresh sandbox pairs) and consistency/leakage review provides automated quality control that enables large-scale construction without manual review.

Training Implications

1. Longer trajectories matter: SWE-Stream's superior performance at matched token budgets, despite fewer trajectories, suggests that trajectory quality and complexity (measured by token length and agent turns) may be as important as quantity.

2. Multilingual and long-horizon signal: The strong performance on SWE-bench Pro (+42.14 points over baseline) indicates that real-world development histories provide valuable training signal for complex, multi-file tasks.

3. SFT alone can rival RL-augmented models: With 3B activated parameters, SWE-Stream's SFT results exceed SFT+RL scores of larger models, suggesting that high-quality training data can substitute for additional training compute.

Practical Applications

• Overlapping construction with training: Accepted tasks can feed SFT trajectory collection or RL rollouts while construction continues, enabling continuous data pipeline operation.
• Resource-constrained environments: The admission control and adaptive limit mechanisms make the system robust under varying resource availability.

---

Conclusion

SWE-Stream demonstrates that scalable SWE agent training depends not only on collecting more development data, but also on building reliable systems that turn heterogeneous software histories into executable and verifiable learning experiences.

Key Takeaways

1. 80,874 validated tasks across six languages enable state-of-the-art performance on three SWE benchmarks
2. Streaming construction delivers 1.45× throughput vs batch and 13.2% more accepted tasks vs fixed-limit streaming
3. Progressive validation with targeted remediation provides an effective alternative to uniform pipeline processing

Limitations

• Generated artifacts are not manually reviewed at scale; automated checks may miss boundary cases or reject valid repairs differing from reference implementations
• Shared model risk: Construction and review agents share the same model, so correlated errors remain possible
• Evaluation scope: Only SFT of Qwen3-30B-A3B-Thinking-2507 with mini-swe-agent was tested; gains with other models, scaffolds, or reinforcement learning remain untested
• Pretraining exposure: Repository exclusion removes direct training overlap, but pretraining exposure to benchmark repositories remains possible

Future Directions

• Test SWE-Stream data with reinforcement learning and alternative scaffolds/models
• Expand coverage to additional programming languages and repository types
• Develop more robust automated checks for description sufficiency and test fairness
• Investigate the relationship between trajectory complexity and downstream task performance