analiza aiqbench
This research proposes inducing sparse neural activity in quantized linear-attention models to enable efficient deployment on neuromorphic hardware, addressing the memory and compute bottlenecks of standard transformers. By nullifying low-magnitude activations while preserving critical outliers, the method achieves significant throughput gains and power reductions compared to edge GPU inference. The work highlights a pathway for event-driven architectures to outperform traditional GPUs in LLM inference by leveraging unstructured sparsity.
Event-Driven Language Models with Sparse Neural Activity for Neuromorphic Hardware
arXiv:2608.30439v1 Announce Type: cross Abstract: Inference with transformer-based large language models (LLMs) is often limited by the memory-bound KV cache and quadratic attention cost. State-space models (SSMs) mitigate this through linear attention and fixed-size recurrent states, but their large dense linear projections remain computationally expensive even after quantization. We introduce a method that induces sparse neural activity in heavily quantized linear-attention models with minimal performance loss. Activations below a per-projection trainable threshold ($\pm \Delta$) are nullified while preserving crucial outliers, achieving comparable performance to dense models with up to 4$\times$ fewer effective arithmetic operations. Targeting a multi-core, multi-chip neuromorphic platform, where event-driven execution converts unstructured sparsity into throughput at both the compute and communication levels, a capability GPU architectures fundamentally lack, we project up to 37$\times$ higher throughput and 16$\times$ lower power versus edge GPU inference of a comparable transformer-based model, and up to 5.4$\times$ improvements over the non-sparsified baseline. These results position sparse, quantized linear-attention models as a natural fit for deploying LLMs on event-driven multi-core platforms.
Abstract:Inference with transformer-based large language models (LLMs) is often limited by the memory-bound KV cache and quadratic attention cost. State-space models (SSMs) mitigate this through linear attention and fixed-size recurrent states, but their large dense linear projections remain computationally expensive even after quantization. We introduce a method that induces sparse neural activity in heavily quantized linear-attention models with minimal performance loss. Activations below a per-projection trainable threshold ($\pm \Delta$) are nullified while preserving crucial outliers, achieving comparable performance to dense models with up to 4$\times$ fewer effective arithmetic operations. Targeting a multi-core, multi-chip neuromorphic platform, where event-driven execution converts unstructured sparsity into throughput at both the compute and communication levels, a capability GPU architectures fundamentally lack, we project up to 37$\times$ higher throughput and 16$\times$ lower power versus edge GPU inference of a comparable transformer-based model, and up to 5.4$\times$ improvements over the non-sparsified baseline. These results position sparse, quantized linear-attention models as a natural fit for deploying LLMs on event-driven multi-core platforms.
Neural and Evolutionary Computing (cs.NE); Machine Learning (cs.LG)
arXiv:2608.30439 [cs.NE]
(or arXiv:2608.30439v1 [cs.NE] for this version)
https://doi.org/10.48550/arXiv.2608.30439 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 262 cuvinte
Subiecte conexe
What's Continual Learning, and Why Might We Expect To See It In Advanced LLM Agents?
LessWrong · 12 iun. 2026
The Sparsity Ceiling: Where Spiking Networks Can and Cannot Trade Activity for Energy
arXiv · 30 iul. 2026
SpikeMoE: Brain-Inspired Competitive Routing for Flexible Spiking Mixture-of-Experts
arXiv · 2 oct. 2026
Spider - Man : Brand New Day Wearable AI Cannot Exist Yet : Real Power - Budget Wall Stands in Its Way
techtimes.com · 3 aug. 2026