Summary (Overview)

• Core Contribution: Trajectory2Gym reframes trajectory generation for tool-use agents as the deliberate construction of learnable interaction experience rather than the collection of successful trajectories, based on two coupled requirements: compositional coverage (observation–action dependencies must be represented) and decision grounding (each supervised action must be supported by learner-visible interaction history).
• Three-Stage Framework: (1) Compositional interaction synthesis targets under-supported capability paths in native environments; (2) Grounded supervision masks decisions by visibility, execution validity, and workflow-order certification; (3) Structural-support learning allocates repeated support per learner scale with parent-normalized SFT.
• Key Empirical Result: Across five Qwen3 scales (1.7B to 30B-A3B), Trajectory2Gym improves Tau1, Tau2, and BFCL at every scale, raising the three-benchmark equal-weight mean by 16.9–29.8 percentage points over public initializations with SFT only (no RL).
• Scale-Beating Performance: The post-trained 4B model surpasses the public 30B-A3B initialization on Tau2 (69.17 vs. 28.33); the 8B model has the highest three-benchmark mean of any evaluated checkpoint.
• Key Findings: Gains concentrate in stateful, multi-turn interaction; parent-normalized SFT provides the largest isolated ablation gain (8.04 points); more structural support is not automatically better (non-monotonic depth sensitivity).

---

Introduction and Theoretical Foundation

Background and Motivation

The paper addresses a fundamental flaw in standard tool-use agent training: successful trajectories are not necessarily useful supervision. Even successful interactions may:

• Underrepresent multi-step dependencies (compositional coverage failure)
• Contain actions unjustified by the agent's decision-time visibility (decision grounding failure)

The authors identify two specific challenges from their candidate pool (34,772 certified candidates):

• Challenge 1 (Compositional Coverage): 40.9% of trajectories contain at most two supervised decisions; only 10.0% contain a certified decision after an authentic error.
• Challenge 2 (Decision Grounding): Successful API execution alone does not establish that an action was justified by visible information. A collector may use an unseen identifier, and native tool replay may accept actions that contradict later user confirmations.

Theoretical Formulation

The policy acts from a decision context that includes only agent-visible information:

$$c_{t} = \big(x, \mathcal{T}_{t}, V(H_{<t})\big), \qquad a_{t} \sim \pi_{\theta}(\cdot | c_{t}), \tag{1}$$

where $H_{<t}$ is the history preceding decision $t$, $V$ projects the history onto the agent-visible subspace, and $\mathcal{T}_{t}$ is the tool schema.

The supervised dataset for learner scale $s$ is:

$$\mathcal{D}_{s} = \{(\tau_{i}, \mathbf{m}_{i})\}_{i=1}^{N_{s}}, \qquad \mathbf{m}_{i} \in \{0, 1\}^{T_{i}}, \tag{2}$$

where $T_i$ is the number of assistant decisions and $m_{it}=1$ marks decision $t$ as a supervised target.

Two required properties:

1. Compositional coverage: training data must provide sufficient support for observation–action dependencies and multi-step structures
2. Decision grounding: each supervised action must be supported by the realized interaction and decision-time context $c_t$

Key Proposition (Collector-Private Uncertainty)

When a demonstrated action depends on collector-private information after conditioning on $c_t$, a policy restricted to $c_t$ faces additional uncertainty (characterized formally as Proposition 1 in the paper). This formalizes why execution validity alone is insufficient for supervision eligibility.

---

Methodology

Stage 1: Compositional Interaction Synthesis

A shared capability vocabulary $\mathcal{K}$ covers semantic action roles (lookup, error, clarification, retry, confirmation, cancellation, update, etc.). An ordered path $p = (\kappa_1, \ldots, \kappa_\ell)$ composes these requirements. A generator $G$ constructs request and initial state, and a teacher $\pi_{\mathrm{T}}$ executes:

$$(x, s_{0}) \sim G(\cdot | p, \mathcal{E}), \qquad \tau \sim P_{\mathcal{E}}(\cdot | x, s_{0}, \pi_{\mathrm{T}}). \tag{3}$$

The support count of a path in a selected set $\mathcal{S}$:

$$n_{\mathcal{S}}(p) = \sum_{\tau \in \mathcal{S}} \mathbf{1}[p(\tau) = p]. \tag{4}$$

Key properties:

• Paths are targeted when support falls below the scale's target (deficit-driven synthesis)
• Uniform random selection misses rare paths (Proposition 3: a path represented once is missed with probability $1 - B/N$)
• Paths are certified only when action/observation dependencies appear in the executed log
• Teacher hints and private events never enter the student trace

Stage 2: Grounded Supervision

Target Eligibility: Distinguishes native execution validity $e_t \in \{0,1\}$ from decision-time grounding $v_t \in \{0,1\}$, with local eligibility:

$$g_{t} = e_{t} v_{t}. \tag{5}$$

• For a tool call: $e_t = 1$ requires authenticated non-error execution; $v_t = 1$ requires tool availability to the agent and identifier-like arguments supported by preceding visible information
• Rejected calls and authentic errors remain visible context (no imitation loss); the grounded recovery becomes a target

Workflow Completion Mask:

$$m_{t} = R_{\tau} \mathbf{1}[q_{\tau} \preceq A_{\tau}] g_{t}, \tag{6}$$

where $R_\tau$ is native terminal success, $q_\tau$ is the task's ordered requirements, $A_\tau$ is the canonicalized sequence of relevant operations, and $\preceq$ denotes ordered subsequence inclusion. Only trajectories with $\sum_t m_t > 0$ enter the eligible pool.

Stage 3: Structural-Support Learning

Structural Selection: For scale $s$, target depth $d_s$ (distinct trajectories per canonical path) grows with learner scale (1, 2, 3, 4, 8 from 1.7B to 30B-A3B). The deficit:

$$\delta_{s}(p; \mathcal{S}) = \left[d_{s} - n_{\mathcal{S}}(p)\right]_{+}. \tag{7}$$

Selection starts from a certified core, adds completing exemplars, closes uncovered atoms/paths, then reduces deficits—exactly maximizing capped support $\sum_p \min(n_{\mathcal{S}}(p), d_s)$ under cardinality budget (Proposition 2).

Parent-Normalized SFT: Each trajectory's contribution is weighted at the parent level:

$$\ell_{\theta}(\tau) = -\frac{1}{N_{\tau}}\sum_{u=1}^{N_{\tau}} \log \pi_{\theta}(y_u \mid h_u), \qquad \mathcal{L}_{\mathrm{SFT}}(\theta) = \frac{1}{|\mathcal{D}_s|}\sum_{\tau \in \mathcal{D}_s} \ell_{\theta}(\tau). \tag{8}$$

This prevents long multi-turn interactions from dominating the loss via rendering multiplicity (Proposition 4 shows per-example averaging weights a parent proportionally to its rendered children count).

---

Empirical Validation / Results

Experimental Setup

• Models: Qwen3 scales (1.7B, 4B, 8B, 14B, 30B-A3B) from public initialization, one epoch of full-parameter parent-normalized SFT
• Benchmarks: Tau1, Tau2, BFCL (Task Success / Function Calling)
• Data: 11.6k–19.9k trajectories per scale
• Evidence standard: Paired comparison with own initialization, same task population, native interface, decoding, simulator, scorer

Table 1: Tau1/Tau2 Results (Pass@1, %)

Key comparisons: Post-trained 4B (Tau2: 69.17) > public 30B-A3B (Tau2: 28.33); post-trained 30B-A3B (Tau2: 84.17) within 1.2 points of closed-source references (84.76, 85.31). 24 of 25 paired domain comparisons improve.

Table 2: BFCL Results (Overall, %)

Gains concentrate in Multi-turn Base (rises 7–41 points). Every post-trained model from 4B exceeds GPT-5.5's reported Multi-turn Base (60.00). The 30B model's 80.93 Overall numerically exceeds Gemini-3.0-Pro's 79.97.

Table 3: Ablation Study (Qwen3-8B)

Table 4: Support-Depth Sensitivity (Qwen3-8B)

Finding: Performance is non-monotonic in support depth—the shallowest setting wins despite fewer data. Structural support is an allocation problem, not a scaling rule.

---

Theoretical and Practical Implications

Theoretical Implications

1. Success gates, does not license, supervision: The paper formalizes why outcome-filtered trajectory training is insufficient—success certifies episode completion, not what the episode teaches or whether each action was justified by visible evidence.
2. Three independent levels of data design: (a) which interaction structures are available, (b) which decisions are eligible for imitation, (c) how much optimization mass each interaction receives. These can vary independently even with similar trajectory counts.
3. Decoupling execution validity from decision grounding: Native execution validity ($e_t$) and decision-time grounding ($v_t$) are formally distinct ($g_t = e_t v_t$); both are required for supervision eligibility.

Practical Implications

1. Data construction beats data collection: Targeted synthesis of under-supported paths yields gains that outcome filtering alone cannot provide, across all scales.
2. Authentic errors are context, not targets: Rejected calls with authentic errors remain visible context without imitation loss; the grounded recovery that follows becomes a target—preserving recovery reasoning without rewarding failure.
3. Per-scale allocation matters: Larger learners benefit from more realizations per path ($d_s$ growing from 1 to 8), but there is an optimal depth—more repetition is not automatically better.
4. Rendering format should not affect optimization weight: Parent-normalized training makes a semantic interaction the unit of training weight, independent of how many rendered children it produces.

---

Conclusion

Main Takeaways

The central claim is that what a trajectory teaches is decided not by its outcome, but by the dependencies it realizes and the decisions its visible history can justify. Trajectory2Gym operationalizes this through:

1. Compositional interaction synthesis: Builds the interactions a pool lacks (targeted expansion on deficit structures)
2. Grounded supervision: Certifies targets one decision at a time via visibility, execution, and workflow-order checks
3. Structural-support learning: Allocates repeated support per learner scale with parent-normalized training weight

Key Results Summary

• Standard SFT (no RL) on constructed interactions improves all three benchmarks at all five scales (16.9–29.8 points on the three-benchmark mean)
• Gains are strongest in stateful and multi-turn tasks where compositional dependencies matter most
• Parent normalization provides the largest isolated gain (8.04 points)
• The post-trained 4B model exceeds the public 30B-A3B initialization on stateful tasks

Future Directions

• Extending decision-level certification to on-policy rollouts (RL settings)
• Application to web, software, and terminal agents
• Tracking structural coverage and decision eligibility alongside data volume in future pipelines
• Keeping training weight of an interaction independent of its rendering format

Broader Lesson

"Scaling tool-use agents depends not only on collecting more successful trajectories, but on constructing interactions whose dependencies are represented and whose decisions are learnable from the agent's visible history."