Summary (Overview)

• This paper introduces a quality-gated physical-accounting protocol for evaluating coding agents, measuring not just prompt tokens but also uncached prefill, KV cache residency, wall time, and GPU energy.
• The authors decompose executed prefill into logical prompt tokens minus reused prefix tokens (Equation 1), showing that token savings can paradoxically increase physical work when prefix reuse is lost.
• On 450 localization cases from SWE-rebench V2, both EAGER (preloading ten retrieved blocks) and COMPACT (anchored history compaction) pass a five-point non-inferiority quality gate.
• EAGER reduces model calls by 35.7% and GPU-energy proxy by 33.2% versus on-demand search (GREP), while COMPACT adds no further savings.
• MASK-5 (keeping only five observations) cuts prompt volume by 33.3% but increases uncached prefill by 608.5% and energy by 125.6%, failing the quality gate—demonstrating that editing reusable history can multiply physical work.

---

Introduction and Theoretical Foundation

Background and Motivation

Repository localization maps an issue report to files, classes, and functions. Modern coding agents use iterative retrieval, structure-aware search, and graph navigation within ReAct-style loops. These agents repeatedly search, read, and reassess retrieved source code, creating long trajectories of tool outputs and intermediate hypotheses.

The core problem: token counts do not determine physical cost. Serving runtimes like vLLM and SGLang maintain KV caches; when a new request begins with a cached token prefix, the runtime reuses those entries and prefills only the remaining suffix. A shorter prompt can therefore cost more if it rewrites cached history, forcing recomputation of all subsequent tokens.

Theoretical Foundation

The paper formalizes the relationship between logical and executed work. Let $n_p$ be the number of model requests under policy $p$, $\ell_{p,t}$ the logical prompt tokens in request $t$, and $h_{p,t}$ the tokens served from a reusable prefix. Then:

$$T_{\text{prompt}}(p) = \sum_{t=1}^{n_p} \ell_{p,t}, \quad T_{\text{uncached}}(p) = T_{\text{prompt}}(p) - \underbrace{\sum_{t=1}^{n_p} h_{p,t}}_{T_{\text{hit}}(p)}. \tag{1}$$

For a treatment–control contrast:

$$\Delta T_{\text{uncached}} = \Delta T_{\text{prompt}} - \Delta T_{\text{hit}}.$$

Key insight: Prompt savings increase executed prefill when a policy loses more reusable tokens than it removes. Because $n_p$ also depends on the policy, fewer calls can reduce total work even when each call is no cheaper.

Three Research Challenges

• C1 (Trajectory attribution): Evidence supply, delivery time, and history retention can each change what the agent does next.
• C2 (Logical-to-physical translation): A policy changes both request count and prefix reuse; logical tokens can order policies differently from uncached prefill, KV occupancy, latency, or energy.
• C3 (Quality-constrained comparison): Resource comparisons must first bound task-quality loss.

---

Methodology

Experimental Setup

The study uses an NVIDIA DGX Spark with an NVIDIA GB10 SoC and 128 GB coherent unified memory. The localization stack uses Qwen3.6-35B-A3B-FP8 (35B total, 3B active) with vLLM, temperature zero, and 65,536-token context limit.

Policy Algebra

Let $\sigma \in \{\text{loc}, \text{patch}\}$ identify the agent scaffold, $S_\sigma$ its system prompt, and $H_{\sigma,t} = (a_1, o_1, \ldots, a_t, o_t)$ its canonical action–observation history. An experimental arm is $p = (\sigma, e, r)$ where $e \in \{\emptyset, R_{10}\}$ is the evidence supplied before the first turn and $r$ controls visible serialization. The visible state and serialized model input are:

$$V^{(e)}_{\sigma,t} = q \oplus e \oplus H_{\sigma,t}, \quad P^{(e,r)}_{\sigma,t} = S_\sigma \oplus r\left(V^{(e)}_{\sigma,t}\right), \tag{2}$$

where $\oplus$ denotes serialization.

Three deterministic context-serialization operators:

$$\mathcal{I}\left(V^{(e)}_{\sigma,t}\right) = V^{(e)}_{\sigma,t},$$ $$\mathcal{A}_\sigma\left(V^{(R_{10})}_{\sigma,t}\right) = \begin{cases} V^{(R_{10})}_{\sigma,t}, & t < \tau, \\ D_{\tau,\sigma} \oplus H_{\sigma,\tau+1:t}, & t \geq \tau, \end{cases}$$ $$\mathcal{L}_5\left(V^{(R_{10})}_{\text{patch},t}\right) = q \oplus R_{10} \oplus H_{\text{patch},t}[o_i \leftarrow \mu(|o_i|) \text{ for } i \leq t-5]. \tag{3}$$

Here $\tau$ is the first successful source read, $D_{\tau,\sigma}$ is a scaffold-specific anchor containing the task query, the successful read, and a bounded pre-rewrite assessment. $\mathcal{L}_5$ follows SWE-agent's LastNObservations: it keeps actions and tool IDs but replaces older outputs with line-count markers $\mu(|o_i|)$.

The arm definitions:

$$G = (\text{loc}, \emptyset, \mathcal{I}), \quad N = (\text{patch}, \emptyset, \mathcal{I}),$$ $$E_\sigma = (\sigma, R_{10}, \mathcal{I}), \quad C_\sigma = (\sigma, R_{10}, \mathcal{A}_\sigma), \quad M_5 = (\text{patch}, R_{10}, \mathcal{L}_5). \tag{4}$$

Quality-Constrained Resource Objective

The resource vector collected for each policy:

$$\mathcal{R}(p) = (T_{\text{prompt}}, T_{\text{output}}, N_{\text{req}}, T_{\text{uncached}}, L_{\text{TTFT}}, L_{\text{wall}}, K_R, K_{\text{max}}, U_{\text{max}}, E_{\text{gpu}}) \tag{5}$$

where $K_R$ is the KV time-area (integrated KV bytes over wall time, in GiB·s), $K_{\text{max}}$ is peak KV occupancy, $U_{\text{max}}$ is peak unified-memory pressure, and $E_{\text{gpu}}$ is sampled GPU energy.

A policy is eligible only if:

$$\text{LCB}_{.95}[Q(p) - Q(c(p))] > -\epsilon, \quad \epsilon = 0.05, \tag{6}$$

where LCB$_{.95}$ is the lower endpoint of a paired 95% interval obtained by resampling whole repositories, and $Q(p)$ is the task-quality endpoint.

Datasets

---

Empirical Validation / Results

RQ1: Quality-Eligible Policies and Shorter Trajectories

Across 450 localization cases, the frozen retriever reaches .431 Recall@5 and .548 Recall@10. Agent Recall@5 is .470, .478, and .494 for GREP, EAGER, and COMPACT respectively.

• EAGER: paired difference +0.8 points; lower 95% endpoint −2.6 (clears the −5-point gate)
• COMPACT: paired difference +2.4 points; lower endpoint −0.9 (meets the criterion)

Table 1: Localization-only profile for $P_{\text{loc}} = \{G, E, C\}$

Key finding: COMPACT uses 8.5% more prompt tokens, 13.3% more calls, 8.2% more wall time, and 9.4% more energy than EAGER. The anchor drops remaining candidates, and the agent may search again for evidence that EAGER still shows.

RQ2: Physical Savings Follow Fewer Calls

For all four primary cost metrics, intervals for both policies exclude zero. Mean TTFT varies by less than 2%, and latency per successful call rises from 5.218 to 5.721 seconds (EAGER) and 5.468 (COMPACT), so task-time savings come from fewer calls, not faster calls.

Cache intervention results (600 cells, toggling vLLM prefix-cache flag):

• EAGER's logical prompt changes by +8.3% (95% CI [−5.7, +24.7]) but uncached prefill by −67.7% ([−71.0, −63.8])
• The planned $C_{\text{loc}} - E_{\text{loc}}$ cache interaction is −7.1k uncached tokens per task; 95% interval [−17.3k, +2.3k] (includes zero)

RQ3: Token Savings Reverse under External Retention

Table 2: Patch-generation-only profile for $P_{\text{patch}}$: mini-SWE-agent on 100 SWE-bench Verified instances

Critical finding: MASK-5 removes 75.1k logical prompt tokens per attempt but loses 135.9k device-hit tokens, adding 60.8k uncached tokens. Cached prefixes serve about 96% of EAGER's prompt but only 53% of MASK-5's. The paired quality difference is −3.0 points with lower 95% endpoint −8.6, crossing the −5-point gate and thus failing non-inferiority.

---

Theoretical and Practical Implications

Break-Even Analysis for History Deletion

The paper derives a first-order estimate: if a runtime served a fraction $\rho$ of the hits observed here in both arms, masking would change uncached prefill by:

$$-75.1\text{k} + 135.9\text{k} \cdot \rho \text{ tokens per attempt}$$

This turns positive above $\rho \approx 0.55$. At $\rho = 0$ (no prefix cache), masking would save prefill; batching or cache pressure that evicts prefixes lowers $\rho$ and with it the masking penalty.

Three Key Recommendations

1. Separate evidence delivery from retention (C1): Eager uses the fewest resources among eligible localization arms; the anchor remains eligible but adds no saving because shorter requests don't offset additional calls.

2. Measure prefix reuse (C2): Reports should expose request count, prefix reuse, and uncached prefill alongside prompt volume.

3. Constrain quality before comparing resources (C3): Apply Equation (6) before interpreting resource differences. Token count alone would favor the policy that the quality gate excludes.

---

Conclusion

A context policy changes both what an agent does and how much of its prompt the runtime can reuse. The paper demonstrates that:

• Early evidence (EAGER) saved work by avoiding calls, reducing model calls by 35.7% and GPU energy by 33.2%.
• Repeated observation masking (MASK-5) turned a 33.3% prompt saving into a 608.5% rise in uncached prefill and a 125.6% energy increase, while failing the quality gate.
• Resource comparisons should measure executed work (uncached prefill, KV time-area, energy) and admit only policies that pass a quality gate.

Limitations and Future Directions

• Results cover two stacks on one DGX Spark serving one request at a time; batching, cache pressure, and cloud GPUs remain untested.
• The GPU-energy proxy omits CPU, host-memory, and whole-system energy.
• Only four fixed policies were tested; window sweeps, LLM summarization, learned pruning, and external memory remain unmeasured, though Equation (1) applies to each.
• The 32-call patch cap measures fixed-budget performance, not unrestricted SWE-bench ability.