Issue 3: The Ceiling of Selectors and the Audit of Evaluations

Papers in this issue

1. 

   1

   Trust the Mass: Forced Weights in KV-Cache Eviction

   Forced-weight KV-cache eviction is near-optimal with top-mass selection, closing only 2–5% of the gap to dense attention; published method margins stem from memory accounting artifacts, not selection quality.

   Editor's note

   First to systematically quantify the "selection ceiling" under forced weight renormalization: enumerating optimal subsets over 168k attention rows proves top-mass selection is near-optimal (only closing a median gap of 2-5%), and decomposes gaps among published eviction methods into memory accounting artifacts (mask storage occupying cache) and query visibility leakage (87.6 points of retrieval gap attributable to the ordering of visible questions). Relative to [#2] RestoreKV and [#1] How Query Visibility, it provides an upper bound for any selector improvement and proves that many published gaps are measurement artifacts rather than algorithmic gains; ContourKV's allocator is a minor contribution, but the measurement framework and negative results are essential reading.
2. 

   2

   Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus

   Massive activations in hybrid linear attention LLMs spike before full attention layers (PAS) and persist through linear layers (ISP), governed by hybridization ratio and cancellation timing.

   Editor's note

   First to systematically characterize the layer-wise organization of massive activations in hybrid linear attention: identifies pre-attention spikes and inter-spike plateaus, both aligned with full attention layer positions, and explains their continuous spectrum with the mixing ratio via a unified lifecycle of write-sink-cancel and delayed cancellation. Relative to [#1] Rethinking's "large-window laziness" and [#2] RoPE frequency axes, it advances the optimization prior of efficient attention to the layer-wise organization and cancellation timing of massive activations, providing causal evidence through controlled pretraining and output gating interventions.
3. 

   3

   When Do Attention Circuits Form? Developmental Trajectories of Capability and Attention-Sink Emergence

   Induction capability emerges early in 1B models, but the specific heads implementing it remain unstable throughout training, unlike BOS-sink heads which form later and vary greatly by model.

   Editor's note

   First to construct a developmental panel tracking attention sinks and induction circuits across 30 checkpoints: proves that induction circuits and BOS-attractors form in two independent phases (with a 10-20× difference in token count) and that the L0/L1 zero BOS floor is an architectural property, not a learning outcome. It provides direct constraints for architectures that deliberately avoid sinks and transforms "when sinks form" from a static observation into an intervenable training-phase question.
4. 

   4

   ATLAS: All-round Testing of Long-context Abilities across Scales

   ATLAS reveals long-context model quality is a capability- and length-dependent profile, not a single score, with rankings reshuffling substantially between 128K and 1M scopes.

   Editor's note

   Reframes long-context evaluation from single-point scores to a length-capability dual-axis profile: uses length-aware AUC integrals over a fixed 8K-1M grid instead of single-window scores, and provides the first end-to-end confidence interval propagation for long-context evaluation. Relative to [#2] PredicateLongBench's difficulty-axis decomposition, its increment is making the length degradation curve itself the scoring object and providing ranking reordering evidence across 26 models (7 models move by ≥2 positions between 128K and 1M).
5. 

   5

   Distractor-Aware Truncation: Disentangling Context-Length Effects from Signal Loss in Long-Context LLM Benchmarks

   Distractor-aware truncation reveals that naive middle-removal conflates signal loss with distractor reduction, producing false "shorter context helps" conclusions across all tested models and benchmarks.

   Editor's note

   Uses paired statistical designs and cross-provider replication to prove that naive middle truncation confounds "distractor reduction" with "signal loss," leading to the opposite conclusion of "short context is better"; the distractor-aware protocol preserves signal by construction, showing performance retention or improvement. It is a methodological warning for anyone evaluating context length effects or compression methods—naive truncation is not a measurement of context window effects.
6. 

   6

   KVpop — Key-Value Cache Compression with Predictive Online Pruning

   KVpop trains lightweight scorers with future-attention supervision and delayed scoring, keeping 97–100% of model quality at 88% KV-cache compression on math reasoning.

   Editor's note

   Moves KV eviction scoring from insertion time to eviction boundary, using future attention quality as the supervision signal, and computes future attention targets efficiently via transposed attention to avoid dense attention graphs. Relative to DMS's delayed eviction, it is the first to use mLSTM states at the eviction boundary to integrate near-future context, providing degradation curves on AIME/HMMT and OOD benchmarks rather than just NIAH scores.
7. 

   7

   DART: Decoded Attention over Recurrent States for Efficient Long-Context Sequence Modeling

   DART augments Mamba-2 with attention-style key-value retrieval over compressed recurrent states, achieving 75% cache memory savings while substantially improving long-range recall and retrieval tasks.

   Editor's note

   Observes that Mamba-2 decodes values but not keys from the compressed state, and proposes state-memory attention (SMA), which decodes token-conditioned keys/values from chunk-level memory states, unifying recurrent compression and attention retrieval within a single chunk-level loop. Relative to Jamba/Samba's layer-wise interleaving and Attamba/RAT's chunk-level attention, it places retrieval and compression in the same memory representation and provides compute-matched comparisons with Mamba-2 and full attention.
8. 

   8

   LKV: End-to-End Learning of Head-wise Budgets and Token Selection for LLM KV Cache Eviction

   LKV achieves state-of-the-art KV cache compression via end-to-end learned budgeting and token selection, recovering 98.4% of full-cache performance at 15% retention with zero inference overhead.

   Editor's note

   Unifies KV eviction budget allocation and token selection into an end-to-end differentiable optimization: LKV-H uses learnable embeddings to compete for a global budget, while LKV-T uses a lightweight MLP to directly predict token importance from KV states (matrix-free), trained with self-distillation. Relative to [#2] KV-CAT's training-time compressibility incentive, it is the first to make budget allocation itself differentiable and query-free; ablations show that budget allocation is the dominant factor in fidelity.
9. 

   9

   NestedKV: Nested Memory Routing for Long-Context KV Cache Compression

   NestedKV reframes training-free KV cache compression as multi-time-scale anomaly detection, outperforming single-anchor methods by up to 19 points under aggressive compression.

   Editor's note

   Reframes training-free KV compression as multi-timescale anomaly detection on the key stream: maintains three anchors (stable, episodic, current) and combines their rankings via head-adaptive mixing and surprise-gated routing. Relative to KeyDiff's single global key-uniqueness anchor and [#2] OasisKV's lookahead prefetch, its increment is a training-free, query-agnostic continuous memory score that yields real degradation curves on RULER/LongBench under aggressive compression ratios (r=0.75-0.95).