analiza aiqbench
WUSH introduces a data-dependent linear transform that provably outperforms fixed Hadamard rotations by addressing outliers in low-bit LLM quantization. The method achieves significant accuracy gains on models like Llama-3.1-8B while delivering up to 5.8× throughput improvements via efficient fused GPU implementations. This work provides a practical, near-optimal solution for deploying quantized large language models with reduced error amplification.
WUSH: Near-Optimal Adaptive Transforms for LLM Quantization
arXiv:2512.00956v3 Announce Type: cross Abstract: Quantizing LLM weights and activations is a standard approach for efficient deployment, but a few extreme outliers can stretch the dynamic range and amplify low-bit quantization errors. Prior transform-based mitigations (e.g., Hadamard rotations) are fixed and data-agnostic, and their optimality for quantization has remained unclear. We derive closed-form optimal linear blockwise transforms for joint weight-activation quantization under standard RTN AbsMax-scaled block quantizers, covering both integer and floating-point formats. The resulting construction, WUSH, combines a Hadamard backbone with a data-dependent second-moment component to form a non-orthogonal transform that is provably near-optimal for FP and INT quantizers under mild assumptions while admitting an efficient fused GPU implementation. Empirically, WUSH improves W4A4 accuracy over the strongest Hadamard-based baselines (e.g., on Llama-3.1-8B-Instruct in MXFP4, it gains +2.8 average points with RTN and +0.7 with GPTQ) while delivering up to 5.8$\times$ per-layer throughput over BF16 via FP4 MatMul. Source code is available at https://github.com/IST-DASLab/WUSH.
Abstract:Quantizing LLM weights and activations is a standard approach for efficient deployment, but a few extreme outliers can stretch the dynamic range and amplify low-bit quantization errors. Prior transform-based mitigations (e.g., Hadamard rotations) are fixed and data-agnostic, and their optimality for quantization has remained unclear. We derive closed-form optimal linear blockwise transforms for joint weight-activation quantization under standard RTN AbsMax-scaled block quantizers, covering both integer and floating-point formats. The resulting construction, WUSH, combines a Hadamard backbone with a data-dependent second-moment component to form a non-orthogonal transform that is provably near-optimal for FP and INT quantizers under mild assumptions while admitting an efficient fused GPU implementation. Empirically, WUSH improves W4A4 accuracy over the strongest Hadamard-based baselines (e.g., on Llama-3.1-8B-Instruct in MXFP4, it gains +2.8 average points with RTN and +0.7 with GPTQ) while delivering up to 5.8$\times$ per-layer throughput over BF16 via FP4 MatMul. Source code is available at this https URL.
Machine Learning (cs.LG); Computation and Language (cs.CL)
arXiv:2512.00956 [cs.LG]
(or arXiv:2512.00956v3 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 166 cuvinte
Subiecte conexe
WUSH-KV: KV Cache Quantization with Data-Adaptive Transforms
arXiv · 30 sept. 2026
[AINews] Reality Checks on AI News (Yegge shuts down Gas Town, Databricks’ +60% Astra cost)
Latent Space · 17 sept. 2026
[AINews] High Taste Testers Ep. 1
Latent Space · 17 oct. 2026
Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers
Hugging Face · 26 aug. 2026