analiza aiqbench
This research proposes an exact mathematical solution for compressing Key-Value cache projections in attention mechanisms by optimizing the reconstruction of attention outputs rather than raw values. The study distinguishes this approach from standard quantization methods, offering a precise error metric based on discarded eigenvalues that is particularly relevant for improving inference efficiency in large language models.
Attention-Weighted Value Projection for KV-Cache Compression
arXiv:2604.11501v2 Announce Type: replace Abstract: Rank reduction discards dimensions; quantization keeps them at lower precision. Comparing the two requires a choice of what compression should preserve. For attention values, we study reconstruction of the attention output rather than reconstruction of the values alone. With fixed attention weights, the optimal orthogonal rank-$r$ projection uses the leading eigenvectors of $V^\top\alpha^\top\alpha V$, and its error is exactly the discarded eigenvalue sum. We extend this objective to calibration datasets and grouped query attention, and describe rank allocation under an additive local error budget. We also examine the limits of using local error to predict downstream loss. Historical weight-perturbation experiments favor coefficient rounding over the value projections tested, but do not establish a comparison at equal cache storage. The resulting distinction is practical: the projection objective has an exact solution, while a comparison with cache quantization requires separate activation-level experiments.
Abstract:Rank reduction discards dimensions; quantization keeps them at lower precision. Comparing the two requires a choice of what compression should preserve. For attention values, we study reconstruction of the attention output rather than reconstruction of the values alone. With fixed attention weights, the optimal orthogonal rank-$r$ projection uses the leading eigenvectors of $V^\top\alpha^\top\alpha V$, and its error is exactly the discarded eigenvalue sum. We extend this objective to calibration datasets and grouped query attention, and describe rank allocation under an additive local error budget. We also examine the limits of using local error to predict downstream loss. Historical weight-perturbation experiments favor coefficient rounding over the value projections tested, but do not establish a comparison at equal cache storage. The resulting distinction is practical: the projection objective has an exact solution, while a comparison with cache quantization requires separate activation-level experiments.
10 pages, 3 figures. Corrected general optimality and cache-storage claims; exact fixed-attention value-projection theorem and equal-cost allocation corollary. Detailed changes in Appendix A. No new model evaluations
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2604.11501 [cs.LG]
(or arXiv:2604.11501v2 [cs.LG] for this version)
From: Samuel M.A Salfati [view email] [v1] Mon, 13 Apr 2026 14:06:18 UTC (414 KB) [v2] Tue, 8 Sep 2026 17:08:37 UTC (72 KB)
Text integral sub licența deschisă a publicației · 209 cuvinte
Subiecte conexe
Model Weight Profiles: Where Do the Parameters Go?
AMD · 25 sept. 2026
[AINews] not much happened today
Latent Space · 3 oct. 2026
Enabling DeepSeek-V4-Flash Training on AMD Instinct MI355X GPUs with Primus
AMD · 3 sept. 2026
Can we change an AI model’s architecture without retraining it from scratch?
Amazon Science · 12 oct. 2026