aiqbench analysis
This paper introduces Compressed Active Subspaces (CAS), a novel method that reduces memory overhead by embedding model parameters into a compressed space before performing Bayesian inference. By mitigating the prohibitive storage requirements of full-dimensional gradients, CAS enables scalable uncertainty quantification for increasingly large neural networks without sacrificing predictive performance. The work represents a significant methodological advance in making active subspace techniques viable for modern deep learning architectures.
Compressed Active Subspaces for Scalable Bayesian Inference
arXiv:2609.19539v1 Announce Type: cross Abstract: Active subspace methods provide a framework for quantifying predictive uncertainty in high-dimensional models by identifying and performing inference along parameter directions that have the greatest influence on the model output. However, the construction of active subspaces requires storing many full-dimensional model gradients, which becomes prohibitive as model size increases. We address this limitation by proposing Compressed Active Subspaces (CAS), a scalable approach that first maps the model parameters to a compressed space using a structured isometric embedding and then constructs the active subspace within this reduced parameterization. Our approach substantially reduces the memory required for active subspace construction and enables Bayesian inference for large models where standard active subspace methods become impractical. We demonstrate the scalability of CAS on neural networks of increasing size while maintaining predictive performance and robust uncertainty estimates.
Abstract:Active subspace methods provide a framework for quantifying predictive uncertainty in high-dimensional models by identifying and performing inference along parameter directions that have the greatest influence on the model output. However, the construction of active subspaces requires storing many full-dimensional model gradients, which becomes prohibitive as model size increases. We address this limitation by proposing Compressed Active Subspaces (CAS), a scalable approach that first maps the model parameters to a compressed space using a structured isometric embedding and then constructs the active subspace within this reduced parameterization. Our approach substantially reduces the memory required for active subspace construction and enables Bayesian inference for large models where standard active subspace methods become impractical. We demonstrate the scalability of CAS on neural networks of increasing size while maintaining predictive performance and robust uncertainty estimates.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Methodology (stat.ME); Machine Learning (stat.ML)
arXiv:2609.19539 [cs.LG]
(or arXiv:2609.19539v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.19539 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 234 words
Related coverage
Subspace Inference Enables Efficient Active Reward Learning from Preferences
arXiv · Sep 5, 2026
Scalable AI Uncertainty Quantification via Generalized Laplace Active Subspaces
arXiv · Oct 10, 2026
QuanVI: Score-based Variational Inference via Quantum Maximally Mixed States
arXiv · Oct 1, 2026
[AINews] not much happened today
Latent Space · Oct 3, 2026