CLIFFCOMPACTION: Cost-Efficient Compaction for Long-Horizon Coding Agents

Summary (Overview)

• CLIFFCOMPACTION is a rule-based, training-free autocompaction technique for coding agents that reduces inference cost by up to 50% while maintaining or improving performance on coding benchmarks (SWE-bench Verified, Terminal-Bench) and achieving state-of-the-art results on KernelBench for continual learning.
• The method's key design principle is high compaction precision over recall: it truncates or drops content verbatim (never rephrasing or summarizing), and each compaction discards previous compacted history entirely, preventing context drift from accumulating across compaction events.
• CLIFFCOMPACTION makes test-time scaling economically viable: three compacted Kimi K2.6 rollouts on Terminal-Bench 2.0 match Opus 4.7 and exceed Opus 4.6 and GPT-5.3 Codex at lower cost, gaining +10.5 points at only 1.9× the cost of a single uncompacted run.
• For continual learning on KernelBench Level 3, CLIFFCOMPACTION achieves CUDA kernel speedups of 2.23× after 200 steps and 3.58× after 400 steps, surpassing specialized search algorithms and trained agents despite being a general-purpose technique.
• The authors open-source a scaffold-agnostic API-proxy implementation usable with Claude Code, Codex, and other harnesses.

Introduction and Theoretical Foundation

Background and Motivation

Coding agents solve complex tasks requiring contexts spanning millions of tokens, creating two fundamental challenges:

1. Context management: How to handle information once an agent session exceeds its context window
2. Efficiency: How to maximize performance while reducing the computational cost of long contexts

The authors identify that the KV-cache is the primary bottleneck for long-context sessions, being both memory-intensive and bandwidth-bound. While backend solutions like FlashAttention reduce KV-cache memory directly, frontend token-based algorithms can reduce costs indirectly by manipulating context content.

Key Theoretical Concepts

Compaction Precision vs. Recall: The paper introduces a crucial trade-off:

• Compaction precision: How faithfully preserved information matches the original (avoiding omission/distortion)
• Compaction recall: How much information remains retrievable across many compactions

LLM-based summarization suffers from low precision because summaries of summaries compound losses, leading to subtle context drift. CLIFFCOMPACTION optimizes precision at the cost of recall, based on the finding that "removing stale context can sometimes improve agent performance rather than simply reducing memory pressure."

Residual Propagation: Though CLIFFCOMPACTION discards old context, information propagates forward implicitly:

$$C_t \leftarrow L_t \leftarrow C_{t-1} \leftarrow L_{t-1} \leftarrow \dots \leftarrow C_1 \leftarrow L_1$$

Each live session $L_t$ is generated with compacted history $C_{t-1}$ in view, so the agent's actions carry implicit influence of discarded context.

Methodology

The CLIFFCOMPACTION Algorithm

The core algorithm operates as follows. After the $(t-1)$-th compaction, the context $S_t$ consists of compacted history $C_{t-1}$ followed by live session $L_t$:

$$S_t = C_{t-1} \oplus L_t, \quad C_t = \text{CLIFFCOMPACTION}(L_t), \quad S_{t+1} = C_t \oplus L_{t+1}$$

Critical design choices:

1. Token threshold triggering: Context grows naturally until exceeding a preset budget, then compacts
2. Verbatim retention: Keep exact excerpts, never summaries
3. Flat compaction: Each new compaction discards previous compactions entirely, operating only on the live session since the last event

Compaction Rules

• Tool results: Drop those exceeding 500 characters; retain shorter ones (grep matches, exit codes, concise output)
• Tool calls: Reduce to compact signatures (tool name, target file/path, essential arguments); remove long embedded content
• Thoughts: Truncate to 300 characters
• System prompt and task description: Keep in full
• Recent turns: Keep the $K$ most recent turn pairs unchanged

Algorithm Pseudocode

Algorithm 1 CLIFFCOMPACTION
1: function CLIFFCOMPACTION(turns)
2:    old, recent ← turns[:-2K], turns[-2K:]  ▷ K turn pairs
3:    parts ← []
4:    for m ∈ old
5:    if m is a prior compaction
6:    skip  ▷ discard, keep flat
7:    else if m is ASSISTANT
8:    parts += [TRUNCATE(m.thinking, 300), SIGNATURE(m.toolcall, 150)]
9:    else if m is TOOLRESULT and |m| ≤ 500
10:   parts += [m]
11:   end if
12:   end for
13:   return JOIN(parts), recent
14: end function

KV-Cache Efficiency

CLIFFCOMPACTION preserves cache efficiency by never modifying context between compactions. The cache remains valid across each growth segment and is invalidated only at compaction points. Since uncached input tokens cost 5–6× more than cached ones, this design is crucial for cost efficiency.

Experimental Setup

• Benchmarks: SWE-bench Verified, Terminal-Bench 2.0/2.1, KernelBench Level 3
• Scaffolds: mini-swe-agent, OpenHands, Terminus-2, Claude Code (via API proxy)
• Models: Kimi (K2.5, K2.6, K2.7), GLM (5, 5.1, 5 Turbo, 5.3 Flash, 4.7 Flash), GPT-5-mini
• Compaction thresholds: $B \in \{45\text{K}, 32\text{K}, 16\text{K}, 8\text{K}\}$

Empirical Validation / Results

SWE-bench Verified Performance

CLIFFCOMPACTION preserves most full-context performance at moderate thresholds:

Terminal-Bench Results

On Terminal-Bench 2.0, CLIFFCOMPACTION improves over full-context baselines:

• Kimi K2.6: 61.42% at both 32K and 16K thresholds vs. 59.16% full-context baseline (+2.26 points)
• At 16K, replacing Terminus-2's native LLM summarization with CLIFFCOMPACTION increases success from 55.45% to 61.42%
• GLM 5.1: Improves from 49.83% (full) to 53.20% (32K) and 54.33% (16K)

On Terminal-Bench 2.1 with Claude Code, CLIFFCOMPACTION reaches 76.69% at matched ~45K context, exceeding both Claude Code's autocompaction (70.97%) and default 200K configuration (73.03%).

Test-Time Scaling Results

CLIFFCOMPACTION makes multi-rollout scaling cost-effective:

The SOFT GROUP VERIFICATION (SGV) selector uses a LightGBM classifier with features $\phi(T) \in \mathbb{R}^d$ including line-overlap agreement features across rollouts, selecting $\hat{T} = \arg\max_T s(\phi(T))$.

Continual Learning on KernelBench Level 3

CLIFFCOMPACTION enables sustained improvement beyond context limits:

Without compaction, 98% of runs terminate early (median 99 steps) due to context exhaustion, capping speedup at 1.30×. CLIFFCOMPACTION outperforms specialized kernel-optimization systems by 25% despite being general-purpose.

Comparison with Existing Compaction Strategies

CLIFFCOMPACTION is the only method that stays competitive on quality while remaining cheap on every benchmark.

Theoretical and Practical Implications

Theoretical Contributions

1. Precision-recall framework for compaction: The paper formalizes the trade-off between compaction precision (faithfulness) and recall (information retention), arguing that precision is more critical for agent performance than previously recognized.

2. Residual knowledge propagation: Demonstrates that discarding context entirely doesn't lose all information—the agent's subsequent behavior carries implicit influence of prior context, enabling effective continual learning over millions of tokens.

3. Cache-aware algorithm design: Highlights the importance of minimizing re-prefill events, showing that infrequent but substantial compactions are more cost-effective than frequent small ones.

Practical Implications

• Cost reduction: Up to 50% cost reduction on Terminal-Bench and 90% reduction in cache-read costs
• Democratized test-time scaling: Makes multi-rollout scaling affordable, allowing smaller models to compete with frontier models at lower cost
• Scaffold-agnostic deployment: The API-proxy implementation works with closed-source agents like Claude Code without modification
• Continual learning without specialized training: A general-purpose compaction technique outperforms task-specific kernel optimization systems

Conclusion

CLIFFCOMPACTION demonstrates that a simple, rule-based compaction strategy can outperform sophisticated summarization and specialized systems across coding benchmarks. By prioritizing compaction precision (verbatim retention) over recall, and by discarding previous compactions to prevent context drift, the method maintains performance while dramatically reducing costs.

The authors identify several limitations: benefits depend on scaffold complexity (fixed components consume budget), and the method is most valuable for medium-to-long-horizon tasks. Future work could explore comparisons with trained context-management approaches and external memory systems.

Key takeaway: CLIFFCOMPACTION shows that "less context can be more"—removing stale information not only saves money but can improve agent performance, making it a practical and effective technique for long-horizon coding agents.