IPPRO: Importance-based Pruning with PRojective Offset for Magnitude-indifferent Structural Pruning

Abstract:Importance-based structured pruning overwhelmingly relies on filter magnitude. This proxy is fundamentally flawed: due to scale invariance, functionally identical filters can receive arbitrarily different importance scores under rescaling. We propose IPPRO (Importance-based Pruning with PROjective Offset), a scale-invariant pruning framework grounded in projective geometry. By embedding filters into real projective space ($\mathbb{RP}^N$), IPPRO resolves the singularity at the origin, placing all filters at an equal angular distance from the zero filter. We define PROscore, which captures functional importance by measuring a filter's angular displacement toward zero under a single gradient step (directional collapse). We further connect PROscore to exact $L_0$ relaxation, proving this one-shot criterion reliably predicts multi-step pruning dynamics. Extensive experiments across CNNs, Vision Transformers, and LLMs (e.g., ResNet, DeiT, LLaMA) demonstrate that IPPRO consistently outperforms existing methods, yielding particularly striking gains under high compression and no-fine-tuning regimes, IPPRO establishes a robust, architecture-agnostic paradigm for neural network compression.

Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

arXiv:2507.14171 [cs.LG]

(or arXiv:2507.14171v3 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2507.14171 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.