analiza aiqbench
This research introduces Invariance Pair Guidance (IPG), a novel training technique that enhances model robustness by correcting optimization trajectories using sparse counterfactual pairs rather than extensive data manipulation. By dynamically isolating spurious attributes, the method offers a theoretically grounded and data-efficient solution to prevent models from relying on non-causal shortcuts during inference.
Invariance Pair Guidance: Robustness to Spurious Correlations via Corrective Gradients
arXiv:2502.18975v3 Announce Type: replace Abstract: Machine learning models are inherently bound to the distribution of the training data, often exploiting non-causal shortcuts. As a result, achieving robustness to spurious correlations remains a challenge. While existing approaches rely on data manipulation or re-weighting strategies to achieve robustness, they typically require dense group labels, multiple training domains, or specialized pre-processing. We propose Invariance Pair Guidance (IPG), a method to mitigate reliance on spurious correlations using a sparse set of counterfactual pairs. Unlike other methods demanding extensive supervision, IPG utilizes a novel dual-update mechanism to dynamically correct the optimization trajectory. We generate input pairs that isolate the spurious attribute to define the invariance, a characteristic that should not affect the outcome of the model. Based on these pairs, we define a corrective gradient that complements the traditional gradient descent approach. The correction adapts via a predefined invariance condition. Experiments on ColoredMNIST, Waterbirds-100, and CelebA datasets demonstrate the effectiveness of our approach and its robustness to group shifts, supported by a theoretical convergence analysis. IPG offers a data-efficient and theoretically grounded path to robustness.
Abstract:Machine learning models are inherently bound to the distribution of the training data, often exploiting non-causal shortcuts. As a result, achieving robustness to spurious correlations remains a challenge. While existing approaches rely on data manipulation or re-weighting strategies to achieve robustness, they typically require dense group labels, multiple training domains, or specialized pre-processing. We propose Invariance Pair Guidance (IPG), a method to mitigate reliance on spurious correlations using a sparse set of counterfactual pairs. Unlike other methods demanding extensive supervision, IPG utilizes a novel dual-update mechanism to dynamically correct the optimization trajectory. We generate input pairs that isolate the spurious attribute to define the invariance, a characteristic that should not affect the outcome of the model. Based on these pairs, we define a corrective gradient that complements the traditional gradient descent approach. The correction adapts via a predefined invariance condition. Experiments on ColoredMNIST, Waterbirds-100, and CelebA datasets demonstrate the effectiveness of our approach and its robustness to group shifts, supported by a theoretical convergence analysis. IPG offers a data-efficient and theoretically grounded path to robustness.
This version of the article has been accepted for publication, after peer review (when applicable) but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: this https URL
Machine Learning (cs.LG)
arXiv:2502.18975 [cs.LG]
(or arXiv:2502.18975v3 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 228 cuvinte
Subiecte conexe
Do Student LLMs Inherit OOD Robustness? Invariance-Weighted Distillation for Reliable Knowledge Transfer
arXiv · 22 sept. 2026
Evaluating task vectors, unlearning and inoculation
LessWrong · 20 sept. 2026
Vision: Data-Centric Anchoring for Robust and Interpretable Agentic AI
arXiv · 9 sept. 2026
Robust Mixed-State Cluster States and Spurious Topological Entanglement Negativity
arXiv · 11 iun. 2026