# AttnCompress: Dynamic Attention-Guided Trajectory Compression for Software Engineering Agents > AttnCompress achieves 53.17% pass rate on SWE-Bench-Verified, beating prior compression SOTA while cutting tokens by 21.6% and costs by 33.6%. - **Source:** [arXiv](https://arxiv.org/abs/2609.08318) - **Published:** 2026-09-12 - **Permalink:** https://picx.dev/p/qRqs6r - **Whiteboard:** https://picx.dev/p/qRqs6r/image ## Summary ## Summary (Overview) - **AttnCompress** is a dynamic, attention-guided trajectory compression framework for Autonomous Software Engineering (ASE) agents, addressing context window limits and cost bottlenecks in long-horizon SE tasks. - The framework employs three key mechanisms: **PPL-spike-based structure-aware segmentation**, **proxy attention-based relevance estimation**, and a **dynamic rolling window** for context maintenance. - On SWE-Bench-Verified, AttnCompress achieves a **pass rate of 53.17%**, outperforming prior SOTA (AgentDiet: 51.17%) while reducing token consumption by **21.6%** and total costs by **33.6%**. - The framework is **model-agnostic**, generalizing across diverse proxy models (Qwen, Llama, Gemma) and seven programming languages on Multi-SWE-Bench-Flash. - AttnCompress is a **training-free, plug-and-play middleware** that preserves syntactic integrity and semantic dependencies critical for SE tasks. --- ## Introduction and Theoretical Foundation ### Background and Motivation The advent of LLMs has shifted software engineering from human-centric assistance to **Autonomous Software Engineering (ASE) agents**. These agents operate through long-horizon, iterative "trial-and-error" workflows (codebase analysis, file editing, test execution, debugging), generating lengthy interaction trajectories that create severe bottlenecks: - **Context window limits**: Verbose logs and redundant file contents accumulate rapidly. - **Cost**: Growing token consumption leads to substantial API expenditures. - **Performance degradation**: The "Lost-in-the-Middle" phenomenon shows that excessive noise degrades reasoning. ### Key Challenges Identified The paper identifies two fundamental challenges specific to SE tasks: 1. **Information Density Variance**: Within a trajectory, information density is highly uneven. A real example from django-11133 shows that within a 700+ line file output, only a ~15-line function (`make_bytes`) is useful. This creates a **granularity mismatch**—message-level pruning retains noise, while token-level pruning breaks syntax. 2. **Dynamic Relevance (Focus Shifting)**: The relevance of historical information fluctuates as debugging focus evolves. An audit of 100 trajectories found that **40% exhibited focus shifting**, where the agent abandons one hypothesis and needs to revisit previously discarded context. ### Limitations of Prior Work | Category | Methods | Limitations | |----------|---------|-------------| | **Selection-based** | RECOMP, CPC, LLMLingua-2, FilCo | Token-level granularity breaks syntax; requires training | | **Heuristic-based** | ObsMask, FIFO eviction | Lacks semantic awareness; irreversible information loss | | **Summarization-based** | AgentDiet, Claude Code, Gemini-Cli | Risks hallucination; loses precise details (line numbers, variable names); high overhead | --- ## Methodology ### Framework Overview AttnCompress is formalized as a plug-and-play middleware between the SE agent's trajectory manager and the backend LLM. Given a raw trajectory $T = \{ (a_1, o_1), \dots, (a_t, o_t) \}$ where $a_t$ is the agent's action and $o_t$ is the observation, the goal is to maintain a compressed trajectory $T' = \{ (a_1, o_1'), \dots, (a_t, o_t') \}$ such that $|T'| \ll |T|$ while maximizing retention of semantically relevant information. ### Phase 1: Structure-Aware Segmentation via PPL Spikes The algorithm uses a small proxy model to detect semantic boundaries via **Perplexity (PPL) spikes**. For each line $l_i$ in a tool output: $$ PPL(l_i) = \exp\left(-\frac{1}{|l_i|} \sum_{x \in l_i} \log P(x \mid x_{ \tau$ as split boundaries; merge single-line blocks with predecessors ### Phase 2: Importance Estimation via Proxy Attention The input sequence concatenates: (1) context history, (2) new observation $O_{new}$, and (3) a special **Query Token** $q_{gen}$ (e.g., `<|im_start|>`) appended at the end. The attention weight from this token to each block quantifies relevance: $$ Score(b_i) = \frac{1}{|b_i|} \sum_{t \in b_i} A(q_{gen}, t)\tag{2} $$ **Selection strategy**: Given compression ratio $\rho \in (0,1)$, set token budget $L_{budget} = \rho \cdot \sum_{i=1}^{m} |b_i|$. Sort blocks by descending score and greedily select until budget is reached. ### Phase 3: Dynamic Rolling Maintenance The trajectory is partitioned into three regions: $$ T = \{\underbrace{(a_1, o_1), \ldots, (a_l, o_l)}_{\text{Long-term}}, \underbrace{\ldots, (a_{t-k}, o_{t-k})}_{\text{Short-term}}, \underbrace{\ldots, (a_t, o_t)}_{\text{Immediate}}\}\tag{3} $$ - **Immediate Memory** (last $k$ turns): Kept raw. - **Short-term Memory** (rolling buffer): Re-evaluated every turn against current intent. - **Long-term Memory** (fixed archive): Static for prefix caching; only reconstructed during **Global Refresh**. **Two operating modes**: - **Incremental Update**: Re-compress only short-term region; concatenate with pre-existing compressed archive (maximizes prefix cache hits). - **Global Refresh** (triggered when short-term buffer ≥ M turns): Merge long-term + short-term, perform global selection, allowing "resurrection" of previously discarded blocks. ### Implementation Details - **Proxy model**: Qwen3-4B-Instruct (chosen for code understanding + low overhead) - **Default hyperparameters**: Tail size $k=2$, compression ratio $\rho=0.2$, PPL threshold = -2, attention layer = -1 (last), rolling window $M=10$ - **Hardware**: 8 NVIDIA A100 GPUs --- ## Empirical Validation / Results ### RQ1: Cost-Effectiveness Trade-off (SWE-Bench-Verified) **Table 1: Main Results on SWE-Bench Verified** (per-instance averages) | Method | Pass (%) | Input (k) | $C_{total}$ ($) | Step | |--------|----------|-----------|-----------------|------| | Original | 55.17 | 1118.16 | 0.1189 | 45.93 | | Random | 48.17 | 904.42 | 0.1064 | 61.42 | | Lingua | 48.67 | 817.03 | 0.0912 | 58.31 | | LLMSummary | 50.83 | 810.52 | 0.1126 | 52.54 | | ObsMask | 47.17 | 628.93 | 0.0443 | 63.47 | | SlidingWindow | 49.83 | 588.06 | 0.0968 | 51.21 | | AgentDiet | 51.17 | 822.44 | 0.1429 | 48.12 | | **AttnCompress** | **53.17** | **644.66** | **0.0949** | **52.43** | **Key findings**: - AttnCompress achieves **3.9% relative improvement** over AgentDiet in pass rate - **21.6% token reduction** vs. AgentDiet; **42.3% reduction** vs. Original - **33.6% total cost reduction** vs. AgentDiet; **20.2% reduction** vs. Original - Latency: 366.7s total (66.6s compression) vs. AgentDiet's 498.5s (248.1s compression) — **26.4% faster** ### RQ2: Component Contribution (Ablation Study) **Table 3: Ablation Study Results** | Method | Pass (%) | $\Delta$ | |--------|----------|---------| | AttnCompress (Full) | 42.0 | — | | w/o PPL (Token-level) | 39.0 | -3.0% | | w/o Attention (Random) | 35.0 | -7.0% | | w/o Rolling (Fixed) | 37.0 | -5.0% | The **attention mechanism** is the primary driver (+7.0%), followed by the **rolling window** (+5.0%) and **PPL segmentation** (+3.0%). ### RQ3: Generalization Capabilities **Proxy model robustness** (Table 6, Qwen3-Coder-30B backend): | Proxy Model | Pass (%) | Analysis Time (s) | |-------------|----------|-------------------| | Qwen3-1.7B-Instruct | 43.50 | 71.6 | | Qwen3-4B-Instruct | 44.50 | 530.9 | | Qwen3-8B-Instruct | 46.50 | 384.1 | | Gemma3-4B-Instruct | 45.00 | 515.6 | | Llama3.2-3B-Instruct | 45.50 | 100.4 | **Multi-language results** (Multi-SWE-Bench-Flash): AttnCompress achieves **19.67%** pass rate, near-parity with Original (20.33%), outperforming all compression baselines. It matches or exceeds the Original baseline in Java, Rust, and C++. --- ## Theoretical and Practical Implications ### Theoretical Contributions 1. **Bridging semantic integrity and dynamic adaptability**: AttnCompress demonstrates that attention-based relevance estimation from small proxy models can effectively align with backend agent needs (Spearman $\rho \approx 0.6$ across proxy families). 2. **Granularity resolution**: PPL-based segmentation provides a principled middle ground between token-level and message-level compression, respecting the syntactic structure of code and structured logs. 3. **Dynamic context maintenance**: The three-tier rolling window formalizes the need for non-linear information recall in debugging workflows, addressing the "focus shifting" problem identified in 40% of real trajectories. ### Practical Implications - **Cost efficiency**: Total cost of $0.0949 per instance (vs. $0.1429 for AgentDiet) makes large-scale deployment feasible. - **Model-agnostic design**: Works across diverse proxy models (Qwen, Llama, Gemma) and backend agents (Qwen3-Coder-30B, Qwen3-235B, Gemini-3-Flash). - **Training-free middleware**: No fine-tuning required, enabling plug-and-play integration with existing ReAct-style agents. - **Hyperparameter robustness**: Performance plateaus across architectural choices (threshold, proxy layer), requiring minimal task-specific tuning. --- ## Conclusion AttnCompress addresses the context scalability bottleneck in ASE agents through **dynamic attention-guided trajectory compression**. The framework's three mechanisms—PPL-based structural segmentation, proxy attention relevance scoring, and dynamic rolling window maintenance—collectively preserve syntactic integrity and semantic dependencies critical for SE tasks. **Key results**: - **53.17% pass rate** on SWE-Bench-Verified (SOTA among compression methods) - **21.6% token reduction** and **33.6% cost reduction** vs. prior SOTA - **Model-agnostic** generalization across proxy models and programming languages **Future directions** (implied by the paper): - Investigating the exact mechanism by which attention alignment influences downstream agent outcomes - Exploring overlapping segmentation schemes for improved cross-boundary coherence - Extending evaluation to additional agent frameworks beyond Trae-Agent The framework's ability to balance compression efficiency with semantic preservation offers a practical solution for scalable, long-horizon autonomous software engineering. --- _Markdown view of https://picx.dev/p/qRqs6r, served by PicX — AI-generated visual whiteboard summaries of research papers._