aiqbench analysis
This paper introduces L0-MoE, a technique leveraging L0-regularization to accelerate dense large language models without significant performance degradation. By combining domain-aware dataset curation with dynamic batching, the method achieves up to a 2.5x speedup over standard dense inference while maintaining competitive accuracy. This approach addresses critical bottlenecks in LLM deployment by offering a lightweight alternative that reduces computational costs for high-throughput applications.
Accelerating Dense LLMs via L0-regularized Mixture-of-Experts
arXiv:2609.21672v1 Announce Type: new Abstract: Large language models (LLMs) achieve strong performance but suffer from slow and costly inference. Existing acceleration methods often lead to noticeable performance degradation, while Mixture-of-Experts (MoE) models require extensive computational resources. In this paper, we propose L0-MoE, a lightweight MoE approach using L0-regularization to accelerate dense LLMs nearly without performance loss. Our method introduces a cluster confusion matrix for domain-aware dataset curation and applies dynamic batching for efficient training. Experiments show that L0-MoE achieves up to 2.5x speedup over dense models while maintaining competitive performance, outperforming existing LLM acceleration baselines.
Abstract:Large language models (LLMs) achieve strong performance but suffer from slow and costly inference. Existing acceleration methods often lead to noticeable performance degradation, while Mixture-of-Experts (MoE) models require extensive computational resources. In this paper, we propose L0-MoE, a lightweight MoE approach using L0-regularization to accelerate dense LLMs nearly without performance loss. Our method introduces a cluster confusion matrix for domain-aware dataset curation and applies dynamic batching for efficient training. Experiments show that L0-MoE achieves up to 2.5x speedup over dense models while maintaining competitive performance, outperforming existing LLM acceleration baselines.
Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2609.21672 [cs.AI]
(or arXiv:2609.21672v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.21672 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 188 words
Related coverage
Scaling MoE reinforcement learning on Amazon EKS with EFA and DeepEP with 40% more throughput
AWS · Sep 25, 2026
Opaque recurrence, and other AI terms that you should probably know
TechCrunch · Sep 7, 2026
AI Model Co-Design: Hardware-Friendly LLM Design
NVIDIA · Jul 10, 2026
NVIDIA AI Releases Nemotron 3 Ultra: An Open 550B Mixture-of-Experts Hybrid Mamba-Transformer for Long-Running Agents
MarkTechPost · Jun 4, 2026