Summary (Overview)

• AttnCompress is a dynamic, attention-guided trajectory compression framework for Autonomous Software Engineering (ASE) agents, addressing context window limits and cost bottlenecks in long-horizon SE tasks.
• The framework employs three key mechanisms: PPL-spike-based structure-aware segmentation, proxy attention-based relevance estimation, and a dynamic rolling window for context maintenance.
• On SWE-Bench-Verified, AttnCompress achieves a pass rate of 53.17%, outperforming prior SOTA (AgentDiet: 51.17%) while reducing token consumption by 21.6% and total costs by 33.6%.
• The framework is model-agnostic, generalizing across diverse proxy models (Qwen, Llama, Gemma) and seven programming languages on Multi-SWE-Bench-Flash.
• AttnCompress is a training-free, plug-and-play middleware that preserves syntactic integrity and semantic dependencies critical for SE tasks.

---

Introduction and Theoretical Foundation

Background and Motivation

The advent of LLMs has shifted software engineering from human-centric assistance to Autonomous Software Engineering (ASE) agents. These agents operate through long-horizon, iterative "trial-and-error" workflows (codebase analysis, file editing, test execution, debugging), generating lengthy interaction trajectories that create severe bottlenecks:

• Context window limits: Verbose logs and redundant file contents accumulate rapidly.
• Cost: Growing token consumption leads to substantial API expenditures.
• Performance degradation: The "Lost-in-the-Middle" phenomenon shows that excessive noise degrades reasoning.

Key Challenges Identified

The paper identifies two fundamental challenges specific to SE tasks:

1. Information Density Variance: Within a trajectory, information density is highly uneven. A real example from django-11133 shows that within a 700+ line file output, only a ~15-line function (make_bytes) is useful. This creates a granularity mismatch—message-level pruning retains noise, while token-level pruning breaks syntax.

2. Dynamic Relevance (Focus Shifting): The relevance of historical information fluctuates as debugging focus evolves. An audit of 100 trajectories found that 40% exhibited focus shifting, where the agent abandons one hypothesis and needs to revisit previously discarded context.

Limitations of Prior Work

---

Methodology

Framework Overview

AttnCompress is formalized as a plug-and-play middleware between the SE agent's trajectory manager and the backend LLM. Given a raw trajectory $T = \{ (a_1, o_1), \dots, (a_t, o_t) \}$ where $a_t$ is the agent's action and $o_t$ is the observation, the goal is to maintain a compressed trajectory $T' = \{ (a_1, o_1'), \dots, (a_t, o_t') \}$ such that $|T'| \ll |T|$ while maximizing retention of semantically relevant information.

Phase 1: Structure-Aware Segmentation via PPL Spikes

The algorithm uses a small proxy model to detect semantic boundaries via Perplexity (PPL) spikes. For each line $l_i$ in a tool output:

$$PPL(l_i) = \exp\left(-\frac{1}{|l_i|} \sum_{x \in l_i} \log P(x \mid x_{<context})\right)\tag{1}$$

Boundary detection uses adaptive thresholding:

1. Compute PPL differences: $D_i = |PPL(l_i) - PPL(l_{i-1})|$
2. Set spike threshold: $\tau = \mu + h \cdot \sigma$ (where $\mu$ = mean, $\sigma$ = std, $h$ = sensitivity coefficient)
3. Mark local maxima where $D_i > \tau$ as split boundaries; merge single-line blocks with predecessors

Phase 2: Importance Estimation via Proxy Attention

The input sequence concatenates: (1) context history, (2) new observation $O_{new}$, and (3) a special Query Token $q_{gen}$ (e.g., <|im_start|>) appended at the end. The attention weight from this token to each block quantifies relevance:

$$Score(b_i) = \frac{1}{|b_i|} \sum_{t \in b_i} A(q_{gen}, t)\tag{2}$$

Selection strategy: Given compression ratio $\rho \in (0,1)$, set token budget $L_{budget} = \rho \cdot \sum_{i=1}^{m} |b_i|$. Sort blocks by descending score and greedily select until budget is reached.

Phase 3: Dynamic Rolling Maintenance

The trajectory is partitioned into three regions:

$$T = \{\underbrace{(a_1, o_1), \ldots, (a_l, o_l)}_{\text{Long-term}}, \underbrace{\ldots, (a_{t-k}, o_{t-k})}_{\text{Short-term}}, \underbrace{\ldots, (a_t, o_t)}_{\text{Immediate}}\}\tag{3}$$

• Immediate Memory (last $k$ turns): Kept raw.
• Short-term Memory (rolling buffer): Re-evaluated every turn against current intent.
• Long-term Memory (fixed archive): Static for prefix caching; only reconstructed during Global Refresh.

Two operating modes:

• Incremental Update: Re-compress only short-term region; concatenate with pre-existing compressed archive (maximizes prefix cache hits).
• Global Refresh (triggered when short-term buffer ≥ M turns): Merge long-term + short-term, perform global selection, allowing "resurrection" of previously discarded blocks.

Implementation Details

• Proxy model: Qwen3-4B-Instruct (chosen for code understanding + low overhead)
• Default hyperparameters: Tail size $k=2$, compression ratio $\rho=0.2$, PPL threshold = -2, attention layer = -1 (last), rolling window $M=10$
• Hardware: 8 NVIDIA A100 GPUs

---

Empirical Validation / Results

RQ1: Cost-Effectiveness Trade-off (SWE-Bench-Verified)

Table 1: Main Results on SWE-Bench Verified (per-instance averages)

Key findings:

• AttnCompress achieves 3.9% relative improvement over AgentDiet in pass rate
• 21.6% token reduction vs. AgentDiet; 42.3% reduction vs. Original
• 33.6% total cost reduction vs. AgentDiet; 20.2% reduction vs. Original
• Latency: 366.7s total (66.6s compression) vs. AgentDiet's 498.5s (248.1s compression) — 26.4% faster

RQ2: Component Contribution (Ablation Study)

Table 3: Ablation Study Results

The attention mechanism is the primary driver (+7.0%), followed by the rolling window (+5.0%) and PPL segmentation (+3.0%).

RQ3: Generalization Capabilities

Proxy model robustness (Table 6, Qwen3-Coder-30B backend):

Multi-language results (Multi-SWE-Bench-Flash): AttnCompress achieves 19.67% pass rate, near-parity with Original (20.33%), outperforming all compression baselines. It matches or exceeds the Original baseline in Java, Rust, and C++.

---

Theoretical and Practical Implications

Theoretical Contributions

1. Bridging semantic integrity and dynamic adaptability: AttnCompress demonstrates that attention-based relevance estimation from small proxy models can effectively align with backend agent needs (Spearman $\rho \approx 0.6$ across proxy families).

2. Granularity resolution: PPL-based segmentation provides a principled middle ground between token-level and message-level compression, respecting the syntactic structure of code and structured logs.

3. Dynamic context maintenance: The three-tier rolling window formalizes the need for non-linear information recall in debugging workflows, addressing the "focus shifting" problem identified in 40% of real trajectories.

Practical Implications

• Cost efficiency: Total cost of $0.0949 per instance (vs.$0.1429 for AgentDiet) makes large-scale deployment feasible.
• Model-agnostic design: Works across diverse proxy models (Qwen, Llama, Gemma) and backend agents (Qwen3-Coder-30B, Qwen3-235B, Gemini-3-Flash).
• Training-free middleware: No fine-tuning required, enabling plug-and-play integration with existing ReAct-style agents.
• Hyperparameter robustness: Performance plateaus across architectural choices (threshold, proxy layer), requiring minimal task-specific tuning.

---

Conclusion

AttnCompress addresses the context scalability bottleneck in ASE agents through dynamic attention-guided trajectory compression. The framework's three mechanisms—PPL-based structural segmentation, proxy attention relevance scoring, and dynamic rolling window maintenance—collectively preserve syntactic integrity and semantic dependencies critical for SE tasks.

Key results:

• 53.17% pass rate on SWE-Bench-Verified (SOTA among compression methods)
• 21.6% token reduction and 33.6% cost reduction vs. prior SOTA
• Model-agnostic generalization across proxy models and programming languages

Future directions (implied by the paper):

• Investigating the exact mechanism by which attention alignment influences downstream agent outcomes
• Exploring overlapping segmentation schemes for improved cross-boundary coherence
• Extending evaluation to additional agent frameworks beyond Trae-Agent

The framework's ability to balance compression efficiency with semantic preservation offers a practical solution for scalable, long-horizon autonomous software engineering.