Summary

Research Question

This paper investigates whether accuracy is a sufficient metric for evaluating KV cache compression methods in large reasoning models (LRMs), questioning the assumption that final-answer accuracy alone validates compression quality.

Key Findings

The authors identify an "answer-evidence gap": compressed models can maintain high accuracy while losing the reasoning evidence that justifies their answers. The paper reveals that accuracy is an asymmetric diagnostic—while accuracy collapse is informative, preserved accuracy can conceal evidential degradation.

Experimental Setup

• Models: Qwen3-8B (primary), plus DeepSeek-R1-Distill-Llama-8B
• Tasks: AIME26 (math reasoning), GPQA (question answering), MedCalc (medical calculations), RULER (retrieval)
• Methods tested: 10 KV compression methods including SnapKV, HeadKV, AdaKV, PyramidKV, StreamingLLM, and quantization baseline (KIVI-2bit)
• Budget: 256-token cache compression

Key Metrics

1. Accuracy (Acc.): Final answer correctness
2. Reasoning Correctness (RWAC/crct.): Whether the reasoning chain supports the answer
3. Fidelity (Fid.): Robustness to injected misleading evidence

Key Findings

1. The Answer-Evidence Gap

• Compression preserves answer accuracy but degrades reasoning quality and robustness
• AIME baseline: Full-KV achieves 56.7% accuracy with 11.8% RWAC; compressed methods (AdaKV, HeadKV, SnapKV) show 23-27% accuracy with dramatically reduced Fidelity (30-35% vs 91.1% full-KV)

2. Accuracy is Asymmetric (Question 1)

• Accuracy collapse reveals compression damage, but preserved accuracy can hide invalid reasoning
• Failure modes are compression-method-specific:
  • SnapKV: mostly fails on chain inconsistencies
  • StreamingLLM: fails through low fidelity and poor accuracy simultaneously
  • MedCalc: formula-triggering info survives, but verification anchors don't

3. Rank Correlation Failure (Question 2)

• Accuracy rankings correlate poorly with reasoning-quality rankings
• Example on GPQA: LagKV achieves even better accuracy than full-KV but performs catastrophically on reasoning consistency (RWAC/crct ≈ 55% vs 10. propagation)
• At 256-token budget, ranking by accuracy versus chain consistency reverses the top performers

4. Robustness to Perturbations

• Fixed-trace perturbation testing shows compression affects resistance to misleading information
• Different methods fail at different perturbation positions:
  • StreamingLLM over-retains early tokens (high vulnerable to early intervention)
  • SnapKV ignores late content, making it vulnerable to end perturbations
• Attention-based methods bias toward conclusion-like patterns, sometimes hallucinating new "sinks" after perturbation

5. Task-Specific Divergence

• Retrieval-heavy tasks (RULER): accuracy collapse is visible → accuracy is informative
• Reasoning problems: partial evidence suffices for correct answers but not for verification or robustness → accuracy hides the gap

6. Key Insight for Compression Methods

Figure 3 shows that compression strategies differ dramatically in their retention patterns—neither recency-only (TOVA) nor attention-based (SnapKV) methods account for answer–evidence entanglement, leading to:

• Loss of proximal verification steps
• Distortion of reasoning chains
• Vulnerability to misleading perturbations

Main Recommendations

1. Evaluation: Always pair accuracy with faithfulness metrics (RWAC/correct, fidelity)
2. Design: Future compressors must preserve dependency structure and provenance, not just salient tokens
3. No method currently achieves both high accuracy and high faithfulness simultaneously

Significance

This work challenges the adequacy of accuracy-only evaluation in KV compression research, highlighting that for reasoning tasks, evidence preservation is as important as answer accuracy.