analiza aiqbench
This paper is squarely about AI: it compresses n-gram embedding lookup tables in large language models (the example is DeepSeek) using a shared 2-bit quantization scheme. Its reported results include a 41–63% reduction in output KL versus 2-bit round-to-nearest and a drop in table storage from 203GB to 51GB, though these are preprint claims that still need independent replication. For practitioners, the main takeaway is that conditional memory tables, a growing part of LLM capacity, may become much cheaper to host on GPU if compression of this kind holds up.
N-Gram Memories at Two Bits: Quantizing Lookup Tables in Large Language Models
arXiv:2610.21810v1 Announce Type: cross Abstract: Conditional memory expands language-model capacity through selective lookup, but the full embedding table must remain available for every possible input. We introduce Conditional Program (CP), a compressed representation that reconstructs only the rows an input requests. CP represents each row by a short code and a local scale, with a reconstruction program shared across the table. We instantiate this representation with a Gaussian trellis, norm restoration and independently decodable row records. CP's two-bit code lowers output KL by 41--63\% relative to 2-bit round-to-nearest. Analysis across three trained n-gram memory families shows that Gaussian-like coordinate statistics coexist with learned relationships among rows; at matched storage budgets, the shared trellis outperforms the fitted codebooks and low-rank residual variants we test. For DeepSeek, it reduces table storage from 203GB to 51GB, leaving room for larger workloads when the table is kept on GPU.
Abstract:Conditional memory expands language-model capacity through selective lookup, but the full embedding table must remain available for every possible input. We introduce Conditional Program (CP), a compressed representation that reconstructs only the rows an input requests. CP represents each row by a short code and a local scale, with a reconstruction program shared across the table. We instantiate this representation with a Gaussian trellis, norm restoration and independently decodable row records. CP's two-bit code lowers output KL by 41--63\% relative to 2-bit round-to-nearest. Analysis across three trained n-gram memory families shows that Gaussian-like coordinate statistics coexist with learned relationships among rows; at matched storage budgets, the shared trellis outperforms the fitted codebooks and low-rank residual variants we test. For DeepSeek, it reduces table storage from 203GB to 51GB, leaving room for larger workloads when the table is kept on GPU.
Machine Learning (cs.LG); Computation and Language (cs.CL)
arXiv:2610.21810 [cs.LG]
(or arXiv:2610.21810v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.21810 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 238 cuvinte
Subiecte conexe
[AINews] DeepSeek v4.1-Flash: 763B-P8B-D16B novel causal Encoder–Decoder architecture with vision marks the Return of the Whale
Latent Space · 12 sept. 2026
Model Weight Profiles: Where Do the Parameters Go?
AMD · 25 sept. 2026
[AINews] Claude Opus 5.5, the new default model for AINews — and everybody cuts prices 40-50%
Latent Space · 23 sept. 2026
[AINews] High Taste Testers Ep. 1
Latent Space · 17 oct. 2026