analiza aiqbench
This study reveals that LLM arithmetic errors stem from specific geometric slippages in the residual stream rather than random noise, offering a new framework to detect and correct quantization failures during inference. By modeling carry operations as continuous latent potentials, the research enables lightweight probes to disentangle ground truth from hallucinations within single activation vectors. These insights provide a concrete mechanism for improving fundamental reliability in large language models through geometric consistency checks.
The Shape of Addition: Geometric Structures of Arithmetic in Large Language Models
arXiv:2606.03645v1 Announce Type: cross Abstract: Large Language Models exhibit paradoxical fragility in fundamental arithmetic, implying a disconnect between internal computation and discrete output. By analyzing the residual stream geometry during multi-operand addition, we identify the Iso-Raw-Sum Trajectory (IRST), a geometric structure where representations are anchored by semantic digits and modulated by continuous carry fibers. We propose the Noisy Quantization Model to explain this geometry, framing arithmetic errors as Geometric Slippages caused by internal neural noise pushing a continuous, latent Carry Potential across quantization thresholds. This geometric framework further elucidates Probe Versatility, explaining how lightweight probes can disentangle coexisting latent signals (such as ground truth versus hallucination) from a single activation vector. Finally, we validate these insights through a geometric consistency check method that effectively detects and corrects these quantization failures during inference. Our code is available at https://github.com/RL-MIND/Shape-of-Addition.
Abstract:Large Language Models exhibit paradoxical fragility in fundamental arithmetic, implying a disconnect between internal computation and discrete output. By analyzing the residual stream geometry during multi-operand addition, we identify the Iso-Raw-Sum Trajectory (IRST), a geometric structure where representations are anchored by semantic digits and modulated by continuous carry fibers. We propose the Noisy Quantization Model to explain this geometry, framing arithmetic errors as Geometric Slippages caused by internal neural noise pushing a continuous, latent Carry Potential across quantization thresholds. This geometric framework further elucidates Probe Versatility, explaining how lightweight probes can disentangle coexisting latent signals (such as ground truth versus hallucination) from a single activation vector. Finally, we validate these insights through a geometric consistency check method that effectively detects and corrects these quantization failures during inference. Our code is available at this https URL.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2606.03645 [cs.LG]
(or arXiv:2606.03645v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.03645 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 232 cuvinte
Subiecte conexe
IntSeqBERT: Learning Arithmetic Structure in OEIS via Modulo-Spectrum Embeddings
arXiv · 15 iun. 2026
Structured Synthetic Reasoning Data for Arithmetic Fine-Tuning of Small Language Models
arXiv · 23 iul. 2026
BRo-JEPA: Learning Modular Transformations in Latent Space
arXiv · 20 aug. 2026
Prime Fourier Embeddings: A Principled Basis for Modular Arithmetic
arXiv · 7 oct. 2026