analiza aiqbench
YAQA introduces a novel adaptive rounding approach that directly optimizes end-to-end output error rather than relying on per-layer proxies, theoretically proving superior convergence and error bounds compared to existing methods like GPTQ. Empirical results show it reduces quantization error by approximately 30% while matching or exceeding Quantization Aware Training performance without adding inference overhead. This advancement offers a more robust framework for model compression that maintains state-of-the-art accuracy across downstream tasks.
Model-Preserving Adaptive Rounding
arXiv:2505.22988v3 Announce Type: replace Abstract: The goal of quantization is to produce a compressed model whose output distribution is as close to the original model's as possible. To do this tractably, most quantization algorithms minimize the immediate activation error of each layer as a proxy for the end-to-end error. However, this ignores the effect of future layers, making it a poor proxy. In this work, we introduce Yet Another Quantization Algorithm (YAQA), an adaptive rounding algorithm that directly considers the error at the network's output. YAQA introduces a series of theoretical results that culminate in the first end-to-end error bounds for quantization algorithms. First, we characterize the convergence time of adaptive rounding algorithms via the structure of their Hessian approximations. We then show that the end-to-end error can be bounded by the approximation's cosine similarity to the true Hessian. This admits a natural Kronecker-factored approximation with corresponding near-optimal Hessian sketches. YAQA is provably better than GPTQ/LDLQ and empirically reduces the error by $\approx 30\%$ over these methods. YAQA even achieves a lower error than quantization aware training. This translates to state of the art performance on downstream tasks, all while adding no inference overhead.
Abstract:The goal of quantization is to produce a compressed model whose output distribution is as close to the original model's as possible. To do this tractably, most quantization algorithms minimize the immediate activation error of each layer as a proxy for the end-to-end error. However, this ignores the effect of future layers, making it a poor proxy. In this work, we introduce Yet Another Quantization Algorithm (YAQA), an adaptive rounding algorithm that directly considers the error at the network's output. YAQA introduces a series of theoretical results that culminate in the first end-to-end error bounds for quantization algorithms. First, we characterize the convergence time of adaptive rounding algorithms via the structure of their Hessian approximations. We then show that the end-to-end error can be bounded by the approximation's cosine similarity to the true Hessian. This admits a natural Kronecker-factored approximation with corresponding near-optimal Hessian sketches. YAQA is provably better than GPTQ/LDLQ and empirically reduces the error by $\approx 30\%$ over these methods. YAQA even achieves a lower error than quantization aware training. This translates to state of the art performance on downstream tasks, all while adding no inference overhead.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2505.22988 [cs.LG]
(or arXiv:2505.22988v3 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 203 cuvinte
Subiecte conexe
Does preservation make sense before we know how to revive?
LessWrong · 15 iun. 2026
Controllable-CoT leads to covert reasoning capabilities
LessWrong · 22 sept. 2026
Optimizing ATOM and vLLM-ATOM for High-Interactivity Inference
AMD · 1 sept. 2026
Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT
METR · 21 iul. 2026