analiza aiqbench
VPOS introduces a novel greedy framework for unsupervised feature selection by operating directly in the weighted PCA loading space, utilizing orthogonal deflation to ensure subsequent selections capture distinct variance components. This approach provably reduces the covariance matrix rank at each step and demonstrates significant speedups over graph-based methods while achieving superior reconstruction error on multiple benchmarks. The method offers a computationally efficient alternative for dimensionality reduction tasks where preserving data structure is critical without relying on labeled supervision.
Variance-Preserving Orthogonal Selection (VPOS): Greedy Feature Selection via Orthogonal Deflation in PCA Loading Space
arXiv:2607.23198v1 Announce Type: new Abstract: We propose Variance-Preserving Orthogonal Selection (VPOS), a greedy framework for unsupervised feature selection that operates in the weighted PCA loading space. After each selection, VPOS projects out the chosen feature's variance direction via null-space deflation, forcing subsequent selections to cover orthogonal parts of the covariance structure. Each step provably reduces the loading matrix rank by one, and the greedy objective connects to monotone submodular maximization. The single hyperparameter $d$ is selected via a reproducible rule: the value minimising reconstruction MSE in a sensitivity sweep. On eight benchmarks, VPOS achieves the lowest reconstruction MSE on all eight while running 10-140x faster than graph-based methods at scale. Comparing against PCA (no deflation) at matched $d$ confirms deflation as the primary driver, reducing MSE by 10-73%.
Abstract:We propose Variance-Preserving Orthogonal Selection (VPOS), a greedy framework for unsupervised feature selection that operates in the weighted PCA loading space. After each selection, VPOS projects out the chosen feature's variance direction via null-space deflation, forcing subsequent selections to cover orthogonal parts of the covariance structure. Each step provably reduces the loading matrix rank by one, and the greedy objective connects to monotone submodular maximization. The single hyperparameter $d$ is selected via a reproducible rule: the value minimising reconstruction MSE in a sensitivity sweep. On eight benchmarks, VPOS achieves the lowest reconstruction MSE on all eight while running 10-140x faster than graph-based methods at scale. Comparing against PCA (no deflation) at matched $d$ confirms deflation as the primary driver, reducing MSE by 10-73%.
Machine Learning (cs.LG)
arXiv:2607.23198 [cs.LG]
(or arXiv:2607.23198v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2607.23198 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 217 cuvinte
Subiecte conexe
One-Sided Quantile Coupling for Flow Matching
arXiv · 5 aug. 2026
Geometric and Behavioral Stratification in Transformer Residual Streams
arXiv · 20 aug. 2026
A Nuclear-Norm Lower Bound for Dithered Scalar Quantization of Matrix Products
arXiv · 9 sept. 2026
The Entangled Dimensions of Decision Theory
LessWrong · 30 iul. 2026