Summary: Quality, Not Just Outcome: A Structural Scoring Framework for Selecting Coding-Agent Trajectories for Supervised Fine-Tuning

Summary (Overview)

• Core contribution: The paper introduces TQS (Trajectory Quality Score), a structural scoring framework for coding-agent trajectories that evaluates process quality rather than just final outcome, operating directly on trajectory structure in linear time without requiring LLM judges.
• Key finding: Models trained on top-ranked trajectories (by TQS) achieve 27.6 pass@1 on SWE-Bench Verified, outperforming random selection (24.8) by 2.8 points and bottom-ranked selection (21.8) by 5.8 points, demonstrating that successful trajectories vary substantially in their value as supervision.
• Framework design: TQS characterizes trajectory quality through three structural dimensions—information focus, intent–action alignment, and policy dynamics—operationalized via seven efficient, interpretable measures computed from event logs alone.
• Scalability advantage: TQS processes trajectories at ~43 ms per trajectory (87.5 trajectories/second with 4 workers), making corpus-scale selection practical (~49 minutes for 258K trajectories at ~$0.1–$0.2 in cloud CPU costs), versus ~$18.6K estimated for GPT-4o judging on the same corpus.
• Validation: TQS achieves 0.732 concordance in distinguishing successful from failed trajectories, 78.7% agreement with human evaluators and 75.4% with Claude Sonnet-5 on ordering successful trajectories, and monotonic resolution rate (54.1%→86.9%) across TQS deciles.

Introduction and Theoretical Foundation

Background and Motivation

Coding agents that interact with sandboxed environments to inspect repositories, execute tests, and edit files have achieved strong performance on software engineering benchmarks like SWE-Bench Verified. A critical ingredient is trajectory supervision: training on multi-turn agent demonstrations consisting of reasoning traces, tool calls, tool observations, and final patches. For medium-scale open-weight models, the common practice is to collect trajectories from more capable teacher agents and retain those whose final patches pass the benchmark test suite.

The Core Problem

The paper identifies a fundamental limitation of outcome-level filtering:

"Outcome-level correctness does not necessarily imply process-level quality."

Among successful trajectories, behavioral patterns vary substantially:

• Some localize bugs quickly, maintain coherent search scope, and transition efficiently from exploration to modification
• Others repeatedly inspect the same files, interrupt stalled shell commands, or postpone meaningful edits until late in the interaction budget

These differences matter because trajectory-level SFT exposes the student to the full interaction process. Inefficient or unstable interaction patterns become part of the training signal.

Theoretical Foundation: Structured Convergence

The paper proposes that high-quality trajectories exhibit structured convergence: as an agent approaches a solution, a strong trajectory becomes increasingly organized around the relevant fix. This organization manifests at three levels:

1. Information focus: The agent focuses observations and file operations on task-relevant evidence
2. Intent–action alignment: Stated intent (thoughts) aligns with tool actions
3. Policy dynamics: The interaction policy moves from broad exploration toward productive edit, test, and recovery behavior

Limitations of Existing Approaches

• Perplexity-style scores are confounded by length and tokenization
• LLM-based judges are costly at scale (hundreds of thousands of trajectories, billions of tokens)
• Neither approach characterizes how effectively an agent organizes its search process over time

Methodology

Trajectory Representation

A trajectory $\tau = (s_0, s_1, \ldots, s_{n-1})$ is a sequence of $n$ steps, where each step is:

$$s_t = (a_t, m_t, o_t, F_t, \varepsilon_t),$$

where $a_t \in \mathcal{A}$ is an action type (think, read_file, write_file, execute_bash, search, submit, other); $m_t$ is the free-text thought; $o_t$ is the tool observation; $F_t \subseteq \mathcal{F}$ is the set of file paths touched; and $\varepsilon_t \in \{0,1\}$ indicates whether $o_t$ matches a predefined error pattern.

The Seven TQS Components

(A1) Observation Entropy Collapse (OEC)

Character-level Shannon entropy of observations, measuring whether observations concentrate around a smaller body of evidence:

$$H(o_t) = -\sum_c p_t(c) \log_2 p_t(c),$$ $$s_{\mathrm{OEC}} = \operatorname{clamp}\left(1 - \min_t r_t, 0, 1\right),$$

where $r_t = \tilde{H}_t / \bar{H}_{\mathrm{open}}$ is the ratio of the three-step moving average entropy to the mean entropy of the first observations.

(A2) Progressive Scope Narrowing (PSN)

Kendall's $\tau_K$ between the distinct files touched in a five-step window and the step index over the second half of the trajectory:

$$\text{scope}_t = \left|\bigcup_{k=\max(0,t-4)}^{t} F_k\right|,$$ $$s_{\text{PSN}} = \text{clamp}\left(\frac{1-\tau_K}{2}, 0, 1\right).$$

A shrinking active file set gives negative $\tau_K$ and a higher score.

(B1) Intent–Action Consistency (IAC)

A lexicon-based parser maps thoughts to coarse intended actions, measuring consistency between stated intent and actual action:

$$s_{\mathrm{IAC}} = \frac{1}{|\mathcal{I}|} \sum_{t \in \mathcal{I}} \mathbf{1}[\hat{a}_t = a_t],$$

where $\mathcal{I}$ is the set of steps where the intended action is defined.

(C1) Tool Transition Entropy (TTE)

Normalized entropy of adjacent action transitions, penalizing long single-tool loops:

$$s_{\mathrm{TTE}} = \frac{-\sum_{i,j} p(i \rightarrow j) \log_2 p(i \rightarrow j)}{\log_2 49}.$$

(C2) Solution Commitment Point (SCP)

Gaussian-shaped score favoring the first substantive edit (error-free write_file) occurring between 20% and 50% of the trajectory:

$$s_{\mathrm{SCP}} = \begin{cases} 0, & \text{no substantive edit;} \\ \exp\left(-d(u)^2/(2\sigma^2)\right), & \text{otherwise,} \end{cases}$$

where $u = t^*/n$ is the relative time of the first substantive edit, $d(u) = \max(0, 0.2-u, u-0.5)$, and $\sigma = 0.2$.

(C3) Post-Error Diversity (PED)

Measures adaptation after errors by combining action-type change with Jaccard distance between successive file sets:

$$d_t = \frac{1}{2}\left(\mathbf{1}[a_{t+1} \neq a_t] + \text{Jacc}(F_{t+1}, F_t)\right),$$ $$s_{\mathrm{PED}} = \frac{1}{|\mathcal{E}_\tau|} \sum_{t \in \mathcal{E}_\tau} d_t.$$

(D1) Dirty Pattern Indicator (DPI)

Guardrail capturing categorical failures (early stopping, no substantive edit, repeated shell-command loops, repeated errors):

$$p = 0.4\mathbf{1}[\text{early}] + 0.4\mathbf{1}[\text{no edit}] + \min(0.3, 3\,\mathrm{lp}) + \min(0.3, 0.6\,\mathrm{er}),$$ $$s_{\mathrm{DPI}} = 1 - \min(1, p).$$

Aggregation

Fixed, hand-set weights: $w_{\mathrm{OEC}} = 0.30$, $w_{\mathrm{IAC}} = 0.25$, $w_{\mathrm{PSN}} = w_{\mathrm{TTE}} = w_{\mathrm{PED}} = w_{\mathrm{DPI}} = 0.10$, and $w_{\mathrm{SCP}} = 0.05$. The final TQS is:

$$\text{TQS}(\tau) = \frac{\sum_{c \in \mathcal{C}^*(\tau)} w_c \cdot s_c(\tau)}{\sum_{c \in \mathcal{C}^*(\tau)} w_c}, \tag{1}$$

where $\mathcal{C}^*(\tau)$ is the set of components defined on $\tau$. Trajectories with $n < 3$ or component coverage below $4/7$ are excluded.

Component Admission Protocol

Each component must pass three criteria:

1. Concordance > 0.5: Rate at which the measure assigns a higher score to a passing trajectory than a paired failing trajectory from the same task
2. Length correlation $|\rho| < 0.25$: Absolute Spearman correlation with trajectory length
3. Behavioral diversity preservation: No mode collapse in selected subsets

Empirical Validation / Results

Discriminant Validity

| Measure | Conc. | $|\rho|$ | Outcome | |---------|-------|---------|---------| | OEC | 0.640 | 0.103 | √ kept | | IAC | 0.612 | 0.087 | √ kept | | PSN | 0.585 | 0.142 | √ kept | | SCP | 0.598 | 0.058 | √ kept | | TTE | 0.563 | 0.218 | √ kept | | PED | 0.541 | 0.034 | √ kept | | DPI | 0.591 | 0.071 | √ guardrail | | TQS Composite | 0.732 | 0.092 | — | | HCR | 0.527 | 0.014 | × diversity | | BUS/TGCR/DCS-f. | ≤ 0.479 | < 0.25 | × conc. | | DCS-Jacc. | — | 0.729 | × length |

Main Results on SWE-Bench Verified

Direct Validation Among Successful Runs

Dimension Ablation

Behavioral Diagnosis on Rollouts

External Outcome Check

The resolved rate increases monotonically from 54.1% in D1 to 86.9% in D10 across TQS deciles (32.9-point spread, no inversion at decile boundaries).

Theoretical and Practical Implications

Theoretical Implications

1. Process quality is orthogonal to outcome: The paper demonstrates that "passing" is too coarse a signal for coding-agent SFT data—trajectories with identical outcomes can differ substantially in their value as supervision.

2. Structured convergence as a quality framework: The three-dimensional characterization (information focus, intent–action alignment, policy dynamics) provides a principled way to think about what makes agent trajectories good training data.

3. Length is correlated but insufficient: The Top-LM-K result (27.0 vs. Random-K's 24.8) shows that length captures some quality signal, but TQS provides additional discriminative power beyond length alone.

Practical Implications

1. Scalable data selection: TQS makes corpus-scale trajectory filtering practical (49 minutes CPU time vs. $18.6K for LLM judging), enabling repeated rescoring and iterative data curation.

2. Cost-quality tradeoff: TQS approaches the effectiveness of GPT-4o rubric-based judging (27.6 vs. 28.4 pass@1) at a fraction of the cost, with the gap narrowing on harder tasks.

3. Behavioral imprint: Selecting high-TQS supervision leaves measurable behavioral improvements in the fine-tuned policy—shorter interactions, more focused observations, and fewer dirty patterns.

4. Hard-task sensitivity: The Top–Random gap increases with task difficulty (1.6 on Easy → 3.4 on Medium → 4.5 on Hard), suggesting that process quality matters most for challenging problems.

Conclusion

Main Takeaways

• TQS provides a lightweight, effective alternative to LLM-based data selection for coding-agent trajectories, operating directly on event-log structure in linear time.
• Process-level trajectory quality is an important factor in coding-agent supervision, with top-ranked trajectories improving pass@1 by 3.2 points over base and 2.8 points over random selection.
• All three structural dimensions contribute to the downstream benefit, with intent–action alignment being the most critical (removing it drops performance below random selection).
• Low-quality successful trajectories can actively hurt downstream performance (Bottom-K: 21.8 vs. base: 24.4).

Future Directions

• Recalibrating parsers and thresholds for different action spaces, multilingual traces, or other step granularities
• Extending to GUI or multimodal agents (requiring new definitions of scope and intent)
• Testing on larger backbones (preliminary 7B experiments showed unstable selector differences)
• Multi-seed training runs for variance estimates
• Incorporating long-trajectory tails (>32K tokens) that may contain useful recovery behavior

Key Limitations

• Single teacher model (Qwen3-Coder-480B) under OpenHands scaffold
• Fixed-budget setting with 1,024 trajectories per condition
• Single training run per condition (no multi-seed variance)
• TQS scores observed process structure, not whether the patch addresses the underlying root cause