# Scaling Agentic Data for Long-Horizon Terminal Intelligence

> TerminalHorizon's automated synthesis of long-horizon terminal trajectories with connected behavioral patterns enables a 27B model to surpass prior methods using 10x less data, improving all 12 benchmarks.

- **Source:** [arXiv](https://arxiv.org/abs/2610.13166)
- **Published:** 2026-10-12
- **Permalink:** https://picx.dev/p/plAdLZ
- **Whiteboard:** https://picx.dev/p/plAdLZ/image

## Summary

# Summary of "Scaling Agentic Data for Long-Horizon Terminal Intelligence"

## Summary (Overview)

- **Core contribution**: Introduces **TerminalHorizon**, a fully automated data synthesis engine that generates high-quality long-horizon terminal agent trajectories by preserving and connecting agentic behavioral patterns (exploration, diagnosis, strategy revision, verification) across multiple task stages.
- **Key innovation**: A two-stage pipeline—**Challenge-Guided Long-Horizon Synthesis** (using independent execution to identify and restore work that environment construction prematurely completed) and **Progressive Horizon Expansion** (extending tasks under the same goal with dependencies on prior results).
- **Dataset**: Constructs **TerminalHorizon-3K**, comprising 1,535 task environments and 2,852 training trajectories with an average of **52.8 interaction turns** (vs. 3.87–15.6 for prior methods), spanning 25 professional domains.
- **Results**: Supervised fine-tuning of Qwen3.5-27B on only ~3K trajectories yields **TerminalHorizon-27B**, improving performance on **all 12 benchmarks**—including +14.2% on Terminal-Bench 2.0, +13.85% on Terminal-Bench 2.1, +11.7% Pass Rate on AutomationBench, and gains on general reasoning benchmarks.
- **System-level capability**: 12-hour recursive self-improvement (RSI) case studies show the model continues experimenting beyond initial solutions, revises methods based on feedback, and retains improvements in final deliverables (e.g., +27.03% on a multi-view point cloud fusion task).

## Introduction and Theoretical Foundation

### Background and Motivation

LLM agents are evolving from dialogue-based question answering toward **autonomous work in digital environments**, with the terminal serving as a general interface that unifies code, data, tools, and execution feedback. As agents move from individual operations to **long-horizon tasks**, they must:

- Coordinate interdependent stages
- Adapt to execution feedback
- Accumulate evidence through exploration
- Continually adjust strategies based on feedback
- Transform intermediate results into final deliverables

### Key Insight: The Gap Between Short and Long Trajectories

> Short trajectories can demonstrate commands and tool calls, but they do not fully show how agents accumulate evidence through exploration, use it to inform judgments, continually adjust strategies based on feedback, and turn intermediate results into final deliverables.

The authors identify two key challenges in synthesizing long-horizon trajectories:

1. **Complex tasks don't guarantee intended behavioral patterns**: Supplied materials may already solve parts of the problem, allowing agents to skip exploration or implementation.
2. **Long trajectories must show patterns unfolding across stages**: More difficult tasks or more interaction turns don't necessarily extend patterns or strengthen their connections.

### Theoretical Foundation: Agentic Behavioral Patterns

The paper formalizes problem-solving processes using **27 agentic behavioral patterns** across five families:

- **Information exploration and judgment**: e.g., relationship tracing (following connections between objects to inform decisions)
- **Diagnosis and error correction**
- **Construction and work progress**
- **Verification and assurance**
- **Strategy and execution coordination**

The central thesis is that environment design must **retain evidence to explore, analysis and implementation left to complete, and feedback that helps agents revise earlier judgments**.

## Methodology

### Task Formulation

Each terminal task package comprises five components:

$$
T = (I, E, S, V, M), \qquad \mathcal{P}(T) = (I, E).\tag{1}
$$

- **$I$**: Public instruction (goal and constraints)
- **$E$**: Executable environment (initial state, tools, execution mechanisms)
- **$S$**: Reference solution
- **$V$**: Private verifier
- **$M$**: Execution configuration

During execution, the agent accesses only $\mathcal{P}(T)$.

### Stage 1: Challenge-Guided Long-Horizon Synthesis

An inner loop uses independent execution to guide task reconstruction; an outer loop plans subsequent research batches.

#### Autonomous Batch Planning
The Planner Agent selects research directions based on existing tasks and feedback:

$$
\mathcal{Q}_b = \operatorname{Plan}(\mathcal{D}_b, \mathcal{F}_b; n_b), \qquad \mathcal{Q}_b = \{q_{b,i}\}_{i=1}^{n_b}.\tag{2}
$$

#### Evidence-Grounded Deep Research
A Deep Research Agent investigates documentation, code repositories, and professional materials to establish real-world needs and technical constraints, producing a task seed $z_{b,i}$.

#### Joint Environment Construction
A Construction Agent synthesizes a concrete instance with controlled initial state and verification conditions. The **public completion contract** requires:

$$
\mathcal{C}_{\mathrm{sem}}(V) \subseteq \operatorname{Sem}(I, \mathcal{O}_{\mathrm{pub}}(E)).\tag{3}
$$

Completion requirements checked by the verifier must be supported by the instruction and discoverable evidence.

#### Independent Challenge
An Independent Challenger (uninvolved in construction) attempts the candidate task:

$$
a_t \sim \pi(\cdot | I, h_t), \quad (s_{t+1}, o_{t+1}) \sim P_E(\cdot | s_t, a_t).\tag{4}
$$

The resulting trajectory and scoring evidence support diagnosis:

$$
\tau_r \sim \mathrm{Rollout}(\pi_B; \mathcal{P}(T_r)), \qquad d_r = \mathcal{A}_{\mathrm{syn}}(T_r, \tau_r, \rho_r).\tag{5}
$$

#### Horizon Gap Analysis and Reconstruction
Gaps between intended and actual work are traced to initial materials, object relationships, or tool behavior, then the task is refined:

$$
T_{r+1} = \operatorname{Refine}(T_r; z, \tau_r, d_r).\tag{6}
$$

The outer loop updates the task collection and feedback records:

$$
\mathcal{D}_{b+1} = \mathcal{D}_b \cup \mathcal{A}_b, \qquad \mathcal{F}_{b+1} = \mathcal{F}_b \cup \Delta\mathcal{F}_b.\tag{7}
$$

### Stage 2: Progressive Horizon Expansion

The Evolution Agent extends accepted tasks under the same goal via **four content-expansion directions**:

1. **Broader roles**: Using existing artifacts to support related responsibilities
2. **Deeper mechanisms**: Making previously simplified relationships/dependencies part of the work
3. **Richer evidence**: Adding records and observations for cross-source analysis
4. **Method upgrades**: Requiring substantial extensions or redesign of the original approach

Added requirements are connected to prior work through **three interaction designs**:
- **Evidence-grounded decisions**: Later decisions depend on evidence gathered through public queries
- **Action-feedback coupling**: Observable tool effects link earlier operations to later diagnosis
- **Instance variation**: Objects/states vary across runs, requiring inspection and adaptation

Expansion produces a new candidate:

$$
\widetilde{T}^{(g+1)} = \operatorname{Expand}\left(T^{(g)}; \delta^{(g)}\right).\tag{8}
$$

## Empirical Validation / Results

### Dataset Characteristics

- **1,535 task environments**, **2,852 training trajectories**
- **52.82 mean interaction turns** (P50: 40, P90: 98) vs. 3.87–15.60 for prior methods
- **25 professional domains** (chips, physics, astronomy, etc.)
- **Median feedback chain length**: 25 turns
- **Average 8.6 core behavioral patterns** per trajectory

### Main Results on Terminal-Bench

| Model | Base Model | Traj. | TB 1.0 | TB 2.0 |
|---|---|---|---|---|
| Terminal-Universe-27B | Qwen3.5-27B | 32.0k | - | 52.8 |
| RST-27B | Qwen3.5-27B | 327.2k | - | 49.4 |
| **Qwen3.5-27B (base)** | - | - | 46.3 | 41.6 |
| **TerminalHorizon-27B** | Qwen3.5-27B | **3k** | **55.0** | **55.8** |

Key finding: **TerminalHorizon-27B outperforms Terminal-Universe-27B by 3.0% using the same base model and harness but fewer than one-tenth as many training trajectories**, demonstrating data efficiency.

### Results Across All Benchmark Categories

| Benchmark | Qwen3.5-27B | TerminalHorizon-27B |
|---|---|---|
| **Coding** | | |
| Terminal-Bench 2.1 | 46.82 | **60.67 (+13.85)** |
| **Agentic** | | |
| AutomationBench (Pass Rate) | 5.5 | **17.2 (+11.7)** |
| AutomationBench (Partial) | 26.4 | **49.7 (+23.3)** |
| τ²-bench Airline | 79.0 | **84.5 (+5.5)** |
| τ²-bench Retail | 84.9 | **87.5 (+2.6)** |
| VitaBench Cross-domain | 20.5 | **31.8 (+11.3)** |
| **General reasoning** | | |
| MATH-500 | 96.9 | **98.9 (+2.0)** |
| GSM8K | 96.3 | **97.3 (+1.0)** |
| AIME 2024 | 93.3 | **96.7 (+3.4)** |
| AIME 2025 | 86.7 | **93.3 (+6.6)** |
| KOR-Bench (Cipher) | 90.1 | **91.2 (+1.1)** |

### Ablation Studies

- **Teacher model comparison**: DeepSeek-V4-Pro produces longer feedback chains (30.3 vs. 19.2 mean turns), lower exact repetition (13.2% vs. 15.8%), and higher pass rate (50.0% vs. 44.0%) than GLM-5.2.
- **Model size scaling**: Gains of +13.85%, +13.11%, and +14.98% on Qwen3.5-27B, Qwen3.5-35B-A3B, and Qwen3.5-122B-A10B respectively.
- **Data scaling**: Performance rises from ~48% (1K trajectories) to 60.67% (3K trajectories) with no saturation observed.

### RSI Case Study: Multi-View Point Cloud Fusion

With a 12-hour budget, the base model **stopped after 12.1 minutes**, satisfied with a self-designed proxy metric. In contrast, TerminalHorizon-27B:
1. Continued experimenting beyond the initial solution
2. Introduced a **leave-one-view-out diagnostic** exposing insufficient coverage
3. Improved cross-view alignment and performed global refinement
4. Used confidence-based point selection
5. Achieved **+27.03%** higher hidden evaluation score

## Theoretical and Practical Implications

### Theoretical Implications

1. **Data quality over quantity**: The paper demonstrates that long-horizon trajectories with connected behavioral patterns are substantially more information-dense than short trajectories, enabling strong performance with ~10x less data than comparable methods.
2. **Environment construction matters**: The work formalizes that environments must preserve substantive work (exploration, implementation, verification) rather than completing it in advance—a principle extendable to other agent training domains.
3. **Cross-stage pattern connections**: Beyond individual behavioral patterns, the paper emphasizes the importance of how patterns connect across stages (findings inform decisions, feedback guides strategy revision, intermediate results support delivery).

### Practical Implications

1. **Data efficiency**: Practitioners can achieve strong agentic performance with far fewer training trajectories if those trajectories are long and behaviorally rich.
2. **Transfer to real-world productivity**: Gains on AutomationBench, VitaBench, and τ²-bench demonstrate that terminal training transfers to enterprise automation and customer service scenarios.
3. **General reasoning preservation**: Long-horizon agentic training can simultaneously improve general reasoning (AIME 2024: +3.4%, AIME 2025: +6.6%), addressing concerns about agentic fine-tuning degrading other capabilities.
4. **Autonomous research capability**: The RSI case studies demonstrate practical value for long-running autonomous work, showing the model can avoid premature convergence and deliver verified improvements.

## Conclusion

The paper introduces **TerminalHorizon**, a fully automated engine for synthesizing long-horizon trajectories that connect agentic behavioral patterns. Through challenge-guided reconstruction and progressive expansion, it links exploration, strategy revision, and verification around a shared goal. Training on these trajectories improves performance on **all 12 benchmarks** across coding, real-world productivity, and general reasoning, while RSI-Exam cases demonstrate sustained experimentation, feedback-driven improvement, and progress retained through final delivery.

**Future directions** suggested by the work include:
- Further scaling of high-quality long-horizon training trajectories (no saturation observed)
- Application of the synthesis principles to other agent domains beyond terminals
- Deeper investigation of how cross-stage behavioral pattern connections scale with task complexity

---

_Markdown view of https://picx.dev/p/plAdLZ, served by PicX — AI-generated visual whiteboard summaries of research papers._