# TurnCost: Turns, Not Observations: Cache-Aware Loop Engineering for Coding Agents

> Tool observations in coding-agent trajectories are small, so cache-aware pricing shows turn count, not observation size, drives cost.

- **Source:** [arXiv](https://arxiv.org/abs/2610.15093)
- **Published:** 2026-10-17
- **Permalink:** https://picx.dev/p/i6Ol52
- **Whiteboard:** https://picx.dev/p/i6Ol52/image

## Summary

# TurnCost: Turns, Not Observations: Cache-Aware Loop Engineering for Coding Agents

## Summary (Overview)

- **Key finding**: Tool observations in coding-agent trajectories are surprisingly small (median 118–286 tokens), while the large build logs that motivate compression techniques occur only near the 99th percentile.
- **Cache-aware pricing**: 98.6% of input tokens are served from prefix cache, meaning flat input pricing overestimates loop cost by ~7.1× on average under near-ideal caching conditions.
- **Break-even condition**: The paper derives a closed-form condition for when model delegation is profitable, showing that the relevant comparison is against the strong model's *cached-read* price, not its list price—making the constraint ~10× tighter than flat-price calculations suggest.
- **Cost decomposition**: Turn count explains 84% of variance in log trajectory cost, while mean observation size explains only 4%—yet all existing optimization techniques target the observation axis.
- **Tool release**: The authors release **TurnCost**, an open-source tool for re-scoring trajectory corpora under alternative caching, pricing, compression, and routing policies.

---

## Introduction and Theoretical Foundation

### Background

AI coding agents (e.g., SWE-agent, OpenHands) operate in iterative **ReAct loops**: propose an edit → apply it → compile → run tests → inspect tool output (the *observation*) → decide next step. The design of this iterative control loop—including context management, model selection, compaction, and termination—is termed **loop engineering** by the authors.

### The Anomaly

The paper is motivated by an empirical puzzle from **AgentDiet** (a context-reduction system):
- A learned reducer achieved 0.652 of baseline cost at 60% pass rate
- A trivial **Delete** baseline (removing observations with no summarization) achieved 0.655 of baseline cost at 58% pass rate
- The learned reducer invoked an additional language model and was selected from five candidates, yet gained only 0.3% cost saving over simple deletion

**Why does a sophisticated learned reducer provide so little cost advantage over simply deleting the observation?**

### Research Questions

- **RQ1**: How large are tool observations in real agentic coding trajectories?
- **RQ2**: By how much does flat input pricing overstate loop cost under cache-aware pricing?
- **RQ3**: Does delegating observation handling to a cheaper model outperform deterministic truncation?
- **RQ4**: Does turn count or observation size contribute more to trajectory cost variation?

---

## Methodology

### Corpus

- **Source**: nvidia/Open-SWE-Traces (207,489 published trajectories, CC BY 4.0)
- **Design**: 2 scaffolds (OpenHands, SWE-agent) × 2 models (Qwen3.5-122B-A10B, Minimax-M2.5) = 4 cells
- **Sample**: 150 trajectories per cell, 600 total, deterministically sampled (first 150 rows in file order)

### Reconstruction

TurnCost reconstructs per-turn context from message sequences, tokenizes with `o200k_base`, and simulates prefix caching exactly using the append-only structure:

$$h_t = C_t - r_{t-1}$$

where $h_t$ is cache-served tokens at turn $t$, $C_t$ is total context, and $r_{t-1}$ is the observation appended after the previous turn.

### Conditions Evaluated

| Condition | Description |
|-----------|-------------|
| **B0** | Unmodified baseline |
| **A\*** | In-band swapping (weak model receives full transcript) |
| **R1** | Out-of-band delegation to weak model |
| **B2** | Same delegation by strong model (isolates compression from routing) |
| **B3** | Deterministic truncation, no model call (analogue of AgentDiet's Delete) |

### Pricing

Prices normalized with $p_S^{in} = \$1$ per million tokens. Defaults: $p_S^{cache} = 0.1 \cdot p_S^{in}$, $p_W^{in} = 0.05 \cdot p_S^{in}$, compression ratio $\sigma = 0.05$.

---

## Empirical Validation / Results

### RQ1: Observations Are Small

**Table 1: Corpus shape and observation-size distribution**

| Model / scaffold | Turns med. (IQR) | Ctx med. | Obs. p50 | p90 | p99 | max |
|---|---|---|---|---|---|---|
| qwen35 / OpenHands | 97 (85–112) | 61,699 | 189 | 854 | 4,274 | 28,604 |
| minimax / SWE-agent | 71 (47–90) | 46,783 | 166 | 1,092 | 6,208 | 33,494 |
| minimax / OpenHands | 60 (44–80) | 45,395 | 286 | 1,258 | 5,194 | 12,854 |
| qwen35 / SWE-agent | 134 (114–159) | 73,336 | 118 | 749 | 5,219 | 32,980 |

Median observation size: **118–286 tokens** across cells. Large logs are rare (99th percentile: 4,274–6,208 tokens).

### RQ2: Flat Pricing Overstates Cost by 7.1×

- 98.1–99.0% of input tokens served from cache
- Flat pricing overstates cost by **6.5× to 7.8×** (mean 7.14×)
- Inflation grows with loop length: 5.33× (<40 turns) → 7.99× (>130 turns)

### RQ3: Truncation and Delegation Are Cost-Indistinguishable

- **B3 (truncation) matches or beats R1 (delegation)** across all 20 cell-threshold combinations
- Maximum gap: 0.17 percentage points at default prices
- At aggressive thresholds: both save 11–19%
- At conservative thresholds: savings decline toward zero and turn negative in 2 of 4 cells

### RQ4: Turn Count Drives Cost

- Turn count alone: $R^2 = 0.844$ with exponent 1.23 (superlinear, confirming re-read effect)
- Mean observation size alone: $R^2 = 0.040$ with exponent −0.31 (slightly negative!)
- Together: $R^2 = 0.945$

**The most expensive trajectories have the most turns AND the smallest observations.**

### Key Distributional Finding

**Table 2: Turn types pooled across all cells (56,164 turns)**

| Turn type | % of turns | % of obs. tokens | Mean obs. |
|---|---|---|---|
| read / exec | 67.4% | 43.7% | 294 |
| edit | 29.9% | 51.8% | 787 |
| terminate | 1.6% | 4.5% | 1,259 |
| think | 1.1% | 0.0% | 6 |

The largest observations come from **edit operations** (editor tools echo updated file context, ~787 tokens mean), not build logs. The motivating example in the literature is misplaced twice over.

### Oracle Ceiling

- Observations contribute 66–74% of transcript growth
- Oracle compression (all observations to 5% size, free, no quality loss): saves 47.8–60.3% maximum
- This is the theoretical ceiling for observation-focused methods

---

## Theoretical and Practical Implications

### The Cache-Aware Cost Model

The trajectory cost under prefix caching:

$$\text{Cost} = \sum_{t=1}^{T} \left[ p_S^{\text{cache}} h_t + p_S^{\text{in}} (C_t - h_t) + p_S^{\text{out}} o_t \right]$$

### The Break-Even Condition (Eq. 3)

For out-of-band delegation at turn $j$, with compression ratio $\sigma$ and prompt overhead fraction $e$:

$$T - j > \frac{p_W^{\text{in}} (1 + e) + p_W^{\text{out}} \sigma}{(1 - \sigma) \, p_S^{\text{cache}}} \xrightarrow[e \to 0]{\sigma \to 0} \frac{p_W^{\text{in}}}{p_S^{\text{cache}}}$$

**Key insights from Eq. (3)**:
1. **Observation size is absent**—it governs how much delegation saves, not whether it's worth doing
2. **The strong model's list price is absent**—only its cached-read price matters (typically 10% of list)
3. **Poor compression penalizes superlinearly** (appears in both numerator and denominator)

### Worked Example

With $p_S^{in} = \$3.00$, $p_S^{cache} = \$0.30$, $p_W^{in} = \$0.15$:
- Correct break-even horizon: $H = 0.15/0.30 = 0.5$ (one remaining turn suffices)
- Using list price instead: $0.15/3.00 = 0.05$ (understates threshold by 10×)

### Three Recommendations for Loop Engineering

1. **Price the cache before optimizing anything**—98–99% of input tokens are cache hits
2. **Use Eq. (3) as a deployment rule**—delegate only when remaining turns exceed the break-even horizon
3. **Always compare against a truncation baseline**—a method that doesn't beat deletion is buying quality, not cost

### Why the Opportunity Is Small

The median observation is small by *design*: SWE-agent's agent–computer interface (ACI) returns concise, structured output rather than raw shell dumps. The compression opportunity was largely removed by interface design *before* context reduction or routing was applied.

---

## Conclusion

### Main Takeaways

1. **Context reduction saturation has a measurable cause**: tool observations are small (118–286 tokens median)
2. **Flat pricing creates illusory savings**: routing results measured against flat-priced baselines cannot be read directly
3. **The positive finding**: turn count (84% of cost variance) is the unexamined opportunity—every existing technique targets observation size (4% of variance)

### Key Caveats

- The turn-count association is **partly mechanical** (cost is a sum over turns), but the exponent >1 and negative observation-size correlation are not definitional
- **Compression quality was not measured**—the parity result concerns cost only
- The corpus excludes trajectories that hit iteration limits, potentially truncating the heavy tail of observation sizes
- Results use open-weight models; frontier proprietary agents may differ

### Future Directions

The open question: **Why does an agent need a hundred turns to fix one bug, and can redundant turns be removed without costing task success?** Removing a turn eliminates not only that turn's cost but every subsequent re-read of its content—making turn reduction potentially far more impactful than observation compression.

> "The most useful thing a loop engineer can know before choosing where to spend effort is that the compression opportunity had largely been removed by interface design before either method was applied."

---

_Markdown view of https://picx.dev/p/i6Ol52, served by PicX — AI-generated visual whiteboard summaries of research papers._