analiza aiqbench
NIRVANA introduces a hardware-aware structured pruning framework that leverages Neural Tangent Kernel-inspired saliency to maintain LLM performance without computationally expensive retraining. By optimizing the balance between attention heads and MLP neurons, it achieves state-ofthe-art compression results across major architectures like Llama3 while preserving critical training dynamics for future fine-tuning. This method offers a theoretically grounded solution to accelerate inference in large language models with minimal capability degradation.
NIRVANA: Structured Pruning Reimagined for Large Language Model Compression
arXiv:2509.14230v2 Announce Type: replace Abstract: While structured pruning presents a highly effective pathway for accelerating Large Language Model (LLM) inference, existing methods frequently suffer from significant performance degradation and demand computationally retraining to recover capabilities. To overcome these barriers, we present NIRVANA, a novel, hardware-aware structured pruning framework designed to preserve both zero-shot performance and the optimization landscape for downstream fine-tuning. Departing from traditional loss-based heuristics, our approach evaluates structural importance through a first-order function-space saliency inspired by the Neural Tangent Kernel (NTK), effectively safeguarding the model's critical training dynamics. To prevent structural collapse at high compression rates, we introduce a global unit-ranking strategy coupled with an analytically derived allocation mechanism, which optimally balances the pruning aggressiveness between attention heads and MLP neurons. Furthermore, we eliminate the instability typically associated with random data sampling by employing a lightweight, KL-divergence-driven calibration data selection process. Extensive evaluations across Llama3, Qwen, and T5 architectures demonstrate that NIRVANA consistently establishes new state-of-the-art results on different benchmarks, providing a theoretically sound and practical approach to LLM compression. The code is available at https://github.com/iDEA-iSAIL-Lab-UIUC/NIRVANA.
Abstract:While structured pruning presents a highly effective pathway for accelerating Large Language Model (LLM) inference, existing methods frequently suffer from significant performance degradation and demand computationally retraining to recover capabilities. To overcome these barriers, we present NIRVANA, a novel, hardware-aware structured pruning framework designed to preserve both zero-shot performance and the optimization landscape for downstream fine-tuning. Departing from traditional loss-based heuristics, our approach evaluates structural importance through a first-order function-space saliency inspired by the Neural Tangent Kernel (NTK), effectively safeguarding the model's critical training dynamics. To prevent structural collapse at high compression rates, we introduce a global unit-ranking strategy coupled with an analytically derived allocation mechanism, which optimally balances the pruning aggressiveness between attention heads and MLP neurons. Furthermore, we eliminate the instability typically associated with random data sampling by employing a lightweight, KL-divergence-driven calibration data selection process. Extensive evaluations across Llama3, Qwen, and T5 architectures demonstrate that NIRVANA consistently establishes new state-of-the-art results on different benchmarks, providing a theoretically sound and practical approach to LLM compression. The code is available at this https URL.
Accepted to COLM 2026. Camera-ready
Machine Learning (cs.LG)
arXiv:2509.14230 [cs.LG]
(or arXiv:2509.14230v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2509.14230 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 272 cuvinte
Subiecte conexe
[AINews] not much happened today
Latent Space · 10 sept. 2026
Tweaking Local Language Model Settings with Ollama
KDnuggets · 28 mai 2026
Quantization and Pruning Methods to Make Your LLM Leaner
KDnuggets · 28 aug. 2026
Compressing LLMs with MoP: Mixture of Pruners
arXiv · 28 iul. 2026