aiqbench analysis
GaugeQuant introduces a novel training-time technique that exploits Transformer symmetries to learn quantization-optimal bases, significantly reducing activation outliers without requiring calibration datasets or freezing model weights. By modifying the loss function with a LogSumExp term and using stop-gradient operators on rotation matrices, this method achieves perplexity improvements comparable to post-training approaches while adding negligible overhead during standard language modeling tasks. This advancement offers an efficient pathway for deploying highly compressed large language models in resource-constrained environments without sacrificing performance.
GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries
arXiv:2607.20757v1 Announce Type: cross Abstract: Transformers are known to have internal continuous symmetries that leave outputs invariant, while modifying quantization. GaugeQuant leverages this in-training by introducing a LogSumExp term to the loss that breaks the symmetries, thus selecting a basis that minimizes activation outliers. A stop-gradient operator ensures that only rotation matrices are updated, yielding the language modeling objective completely unaltered. Our requires no specific calibration data, no quantization simulation, and adds negligible training overhead. With the LLaMA-2 7B model under W4A4 quantization with group size 128, perplexity drops from 8.22 to 6.73, competing with post-training methods that require frozen models and calibration datasets. Under W4A16, perplexity drops from 11.16 to 5.45. Code is available at https://github.com/MPedraBento/gauge-quant.
Abstract:Transformers are known to have internal continuous symmetries that leave outputs invariant, while modifying quantization. GaugeQuant leverages this in-training by introducing a LogSumExp term to the loss that breaks the symmetries, thus selecting a basis that minimizes activation outliers. A stop-gradient operator ensures that only rotation matrices are updated, yielding the language modeling objective completely unaltered. Our requires no specific calibration data, no quantization simulation, and adds negligible training overhead. With the LLaMA-2 7B model under W4A4 quantization with group size 128, perplexity drops from 8.22 to 6.73, competing with post-training methods that require frozen models and calibration datasets. Under W4A16, perplexity drops from 11.16 to 5.45. Code is available at this https URL.
Machine Learning (cs.LG); Computation and Language (cs.CL)
arXiv:2607.20757 [cs.LG]
(or arXiv:2607.20757v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2607.20757 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 212 words
Related coverage
Productionizing TurboQuant on AMD GPUs for KV-Cache-Bound LLM Inference
AMD · Jun 11, 2026
GEM Training: How Meta Doubled the Efficiency of Its LLM-Scale Ads Foundation Model
Meta Engineering · Aug 3, 2026
ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs
arXiv · Oct 2, 2026
ESSA: Evolutionary Strategies for Scalable Alignment
arXiv · Sep 10, 2026