analiza aiqbench
This research introduces PGSVD, a novel low-rank compression framework that theoretically bounds activation-based errors to optimize the trade-off between model accuracy and inference speed. By formulating compression as a bi-objective optimization problem, the method achieves superior performance on both LLMs and VLMs without requiring fine-tuning. The work directly addresses critical deployment challenges regarding memory efficiency and computational cost in large-scale AI systems.
Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM
arXiv:2510.05544v3 Announce Type: replace Abstract: Large language models (LLM) and vision-language models (VLM) have achieved state-of-the-art performance, but they impose significant memory and computing challenges in deployment. We present a novel low-rank compression framework to address this challenge. First, we upper bound the change of network loss via layer-wise activation-based compression errors, filling a theoretical gap in the literature. We then formulate low-rank model compression as a bi-objective optimization and prove that a single uniform tolerance yields surrogate Pareto-optimal heterogeneous ranks. Based on our theoretical insights, we propose Pareto-Guided Singular Value Decomposition (PGSVD), a zero-shot pipeline that improves activation-aware compression via Pareto-guided rank selection and alternating least-squares implementation. We apply PGSVD to both LLM and VLM, showing better accuracy at the same compression levels and inference speedup.
Abstract:Large language models (LLM) and vision-language models (VLM) have achieved state-of-the-art performance, but they impose significant memory and computing challenges in deployment. We present a novel low-rank compression framework to address this challenge. First, we upper bound the change of network loss via layer-wise activation-based compression errors, filling a theoretical gap in the literature. We then formulate low-rank model compression as a bi-objective optimization and prove that a single uniform tolerance yields surrogate Pareto-optimal heterogeneous ranks. Based on our theoretical insights, we propose Pareto-Guided Singular Value Decomposition (PGSVD), a zero-shot pipeline that improves activation-aware compression via Pareto-guided rank selection and alternating least-squares implementation. We apply PGSVD to both LLM and VLM, showing better accuracy at the same compression levels and inference speedup.
Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2510.05544 [cs.CL]
(or arXiv:2510.05544v3 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2510.05544 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 218 cuvinte
Subiecte conexe
Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM
arXiv · 5 iun. 2026
[AINews] not much happened today
Latent Space · 10 sept. 2026
[AINews] Reflection Beam - 501B-A23B American Open Model
Latent Space · 6 oct. 2026
[AINews] not much happened today
Latent Space · 3 oct. 2026