analiza aiqbench
This paper introduces GRIT, a novel training method that enhances out-of-distribution robustness by geometrically projecting out unstable subspaces derived from finite and noisy invariant pairs. By decoupling pair processing from predictor fitting, the approach avoids discarding predictive information while enforcing invariance only where necessary, offering a practical improvement over standard prediction-consistency penalties. The method demonstrates superior mean accuracy across multiple learning objectives on benchmark datasets like ColoredMNIST-INV and Waterbirds-INV.
GRIT: Out-of-Distribution Robustness from Finite and Noisy Invariant Pairs
arXiv:2505.24843v3 Announce Type: replace Abstract: Predictors often exploit spurious correlations that fail under distribution shift, undermining out-of-distribution (OOD) robustness. Invariant pairs, such as the same object photographed against different backgrounds, directly indicate which variation should not affect prediction. Yet real pair collections are finite and noisy, so they can miss relevant variation or lead a learner to discard predictive information. To learn from such collections, we propose Geometric Robustness Invariant Training (GRIT), which estimates an unstable subspace from the covariance of pair differences in fixed pretrained features and projects it out before fitting a predictor. Because pair processing is separate from predictor fitting, the same projection can accompany ERM or other robustness objectives. Unlike quadratic prediction consistency, which penalizes variation along every direction with nonzero pair covariance, including variation introduced by pair noise, GRIT enforces exact invariance only to the estimated subspace. We bound GRIT's target risk in terms of pair count, noise relative to the weakest clean signal, and selected rank, assuming that the label distribution given a linear projection of the features is invariant across environments. Empirically, GRIT improves the mean OOD accuracy of each of eight learning objectives on ColoredMNIST-INV and Waterbirds-INV and has higher mean accuracy than a prediction-consistency penalty using the same pairs in 15 of 16 settings. On UrbanCars, combining pairs that vary the background with pairs that vary a co-occurring object raises worst-group accuracy above that of either pair type alone.
Abstract:Predictors often exploit spurious correlations that fail under distribution shift, undermining out-of-distribution (OOD) robustness. Invariant pairs, such as the same object photographed against different backgrounds, directly indicate which variation should not affect prediction. Yet real pair collections are finite and noisy, so they can miss relevant variation or lead a learner to discard predictive information. To learn from such collections, we propose Geometric Robustness Invariant Training (GRIT), which estimates an unstable subspace from the covariance of pair differences in fixed pretrained features and projects it out before fitting a predictor. Because pair processing is separate from predictor fitting, the same projection can accompany ERM or other robustness objectives. Unlike quadratic prediction consistency, which penalizes variation along every direction with nonzero pair covariance, including variation introduced by pair noise, GRIT enforces exact invariance only to the estimated subspace. We bound GRIT's target risk in terms of pair count, noise relative to the weakest clean signal, and selected rank, assuming that the label distribution given a linear projection of the features is invariant across environments. Empirically, GRIT improves the mean OOD accuracy of each of eight learning objectives on ColoredMNIST-INV and Waterbirds-INV and has higher mean accuracy than a prediction-consistency penalty using the same pairs in 15 of 16 settings. On UrbanCars, combining pairs that vary the background with pairs that vary a co-occurring object raises worst-group accuracy above that of either pair type alone.
Machine Learning (cs.LG)
arXiv:2505.24843 [cs.LG]
(or arXiv:2505.24843v3 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2505.24843 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 325 cuvinte
Subiecte conexe
News
Quantum Computing Report · 25 iul. 2026
A simpler circuit design for purifying entanglement in atom arrays
Quantum Zeitgeist · 3 sept. 2026
GRAIN: Bridging Name and Narrative Shifts in Real-World Graph Reasoning through Invariance-Rewarded Agentic RL
arXiv · 28 aug. 2026
Quantifying mixed-state entanglement via partial transpose and realignment moments
Quantum Journal · 19 aug. 2026