analiza aiqbench
This research introduces BenQ, a novel post-training quantization method that leverages Benford's Law to optimize codebooks for transformer weights without requiring training data. By distinguishing between scale-broad transformational layers and stability-critical parameters like LayerNorm, the technique achieves superior 4-bit compression compared to standard uniform or NF4 methods while maintaining model fidelity. The work highlights a fundamental dichotomy in weight distributions that offers a simpler, more robust alternative to complex optimization-based quantization strategies.
Benford's Law as a Distributional Prior for Post-Training Quantization of Large Language Models
arXiv:2602.00165v2 Announce Type: replace Abstract: Post-training quantization (PTQ) is a practical way to reduce the memory footprint of large language models, but low-bit quantization is sensitive to mismatches between the quantization codebook and the empirical weight/activation distributions. We revisit Benford-like leading-digit statistics as a lightweight diagnostic of scale-broad behavior in transformer tensors. Across several model families, we observe a consistent functional dichotomy: transformational nn.Linear weights tend to be Benford-like, whereas LayerNorm parameters systematically deviate. Motivated by this observation, we propose BenQ, a data-free PTQ codebook that uses a simple log-spaced grid as a proxy for scale-broad distributions and applies it selectively to transformational layers while keeping stability-critical parameters in higher precision. In 4-bit group-wise PTQ, BenQ consistently improves over uniform RTN and trades wins with NF4 across architectures and tasks, while remaining substantially simpler than optimization-based methods. We additionally report dynamic activation quantization as an exploratory stress test: the results show that log-spaced grids can reduce RTN failures in some families, but also reveal that outlier handling remains essential for reliable low-bit activation PTQ. Code is available at https://github.com/ufopcsilab/benford-quant.
Abstract:Post-training quantization (PTQ) is a practical way to reduce the memory footprint of large language models, but low-bit quantization is sensitive to mismatches between the quantization codebook and the empirical weight/activation distributions. We revisit Benford-like leading-digit statistics as a lightweight diagnostic of scale-broad behavior in transformer tensors. Across several model families, we observe a consistent functional dichotomy: transformational this http URL weights tend to be Benford-like, whereas LayerNorm parameters systematically deviate. Motivated by this observation, we propose BenQ, a data-free PTQ codebook that uses a simple log-spaced grid as a proxy for scale-broad distributions and applies it selectively to transformational layers while keeping stability-critical parameters in higher precision. In 4-bit group-wise PTQ, BenQ consistently improves over uniform RTN and trades wins with NF4 across architectures and tasks, while remaining substantially simpler than optimization-based methods. We additionally report dynamic activation quantization as an exploratory stress test: the results show that log-spaced grids can reduce RTN failures in some families, but also reveal that outlier handling remains essential for reliable low-bit activation PTQ. Code is available at this https URL.
Accepted paper in Transactions on Machine Learning Research (TMLR). Link to OpenReview: this https URL
Machine Learning (cs.LG)
arXiv:2602.00165 [cs.LG]
(or arXiv:2602.00165v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2602.00165 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 285 cuvinte
Subiecte conexe
[AINews] High Taste Testers Ep. 1
Latent Space · 17 oct. 2026
[AINews] Meta Muse Wonder & Wonder Full: MSL’s Claude 5.5-tier Frontier Model
Latent Space · 13 oct. 2026
[AINews] The Future of Latent Space
Latent Space · 25 sept. 2026
[AINews] Opus 5.5 is good at explainer videos
Latent Space · 29 sept. 2026