Attention-Weighted Value Projection for KV-Cache Compression

Abstract:Rank reduction discards dimensions; quantization keeps them at lower precision. Comparing the two requires a choice of what compression should preserve. For attention values, we study reconstruction of the attention output rather than reconstruction of the values alone. With fixed attention weights, the optimal orthogonal rank-$r$ projection uses the leading eigenvectors of $V^\top\alpha^\top\alpha V$, and its error is exactly the discarded eigenvalue sum. We extend this objective to calibration datasets and grouped query attention, and describe rank allocation under an additive local error budget. We also examine the limits of using local error to predict downstream loss. Historical weight-perturbation experiments favor coefficient rounding over the value projections tested, but do not establish a comparison at equal cache storage. The resulting distinction is practical: the projection objective has an exact solution, while a comparison with cache quantization requires separate activation-level experiments.

10 pages, 3 figures. Corrected general optimality and cache-storage claims; exact fixed-attention value-projection theorem and equal-cost allocation corollary. Detailed changes in Appendix A. No new model evaluations

Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

arXiv:2604.11501 [cs.LG]

(or arXiv:2604.11501v2 [cs.LG] for this version)

From: Samuel M.A Salfati [view email] [v1] Mon, 13 Apr 2026 14:06:18 UTC (414 KB) [v2] Tue, 8 Sep 2026 17:08:37 UTC (72 KB)