aiqbench analysis
YAQA introduces a novel adaptive rounding approach that directly optimizes end-to-end output error rather than relying on per-layer proxies, theoretically proving superior convergence and error bounds compared to existing methods like GPTQ. Empirical results show it reduces quantization error by approximately 30% while matching or exceeding Quantization Aware Training performance without adding inference overhead. This advancement offers a more robust framework for model compression that maintains state-of-the-art accuracy across downstream tasks.
Model-Preserving Adaptive Rounding
arXiv:2505.22988v3 Announce Type: replace Abstract: The goal of quantization is to produce a compressed model whose output distribution is as close to the original model's as possible. To do this tractably, most quantization algorithms minimize the immediate activation error of each layer as a proxy for the end-to-end error. However, this ignores the effect of future layers, making it a poor proxy. In this work, we introduce Yet Another Quantization Algorithm (YAQA), an adaptive rounding algorithm that directly considers the error at the network's output. YAQA introduces a series of theoretical results that culminate in the first end-to-end error bounds for quantization algorithms. First, we characterize the convergence time of adaptive rounding algorithms via the structure of their Hessian approximations. We then show that the end-to-end error can be bounded by the approximation's cosine similarity to the true Hessian. This admits a natural Kronecker-factored approximation with corresponding near-optimal Hessian sketches. YAQA is provably better than GPTQ/LDLQ and empirically reduces the error by $\approx 30\%$ over these methods. YAQA even achieves a lower error than quantization aware training. This translates to state of the art performance on downstream tasks, all while adding no inference overhead.
Abstract:The goal of quantization is to produce a compressed model whose output distribution is as close to the original model's as possible. To do this tractably, most quantization algorithms minimize the immediate activation error of each layer as a proxy for the end-to-end error. However, this ignores the effect of future layers, making it a poor proxy. In this work, we introduce Yet Another Quantization Algorithm (YAQA), an adaptive rounding algorithm that directly considers the error at the network's output. YAQA introduces a series of theoretical results that culminate in the first end-to-end error bounds for quantization algorithms. First, we characterize the convergence time of adaptive rounding algorithms via the structure of their Hessian approximations. We then show that the end-to-end error can be bounded by the approximation's cosine similarity to the true Hessian. This admits a natural Kronecker-factored approximation with corresponding near-optimal Hessian sketches. YAQA is provably better than GPTQ/LDLQ and empirically reduces the error by $\approx 30\%$ over these methods. YAQA even achieves a lower error than quantization aware training. This translates to state of the art performance on downstream tasks, all while adding no inference overhead.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2505.22988 [cs.LG]
(or arXiv:2505.22988v3 [cs.LG] for this version)
Full text under the publisher's open licence · 203 words
Related coverage
Does preservation make sense before we know how to revive?
LessWrong · Jun 15, 2026
Controllable-CoT leads to covert reasoning capabilities
LessWrong · Sep 22, 2026
Optimizing ATOM and vLLM-ATOM for High-Interactivity Inference
AMD · Sep 1, 2026
Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT
METR · Jul 21, 2026