analiza aiqbench
This paper introduces GENIE, an optimizer designed to improve domain generalization by balancing parameter updates through the One-Step Generalization Ratio. By preventing specific parameters from dominating gradient descent, it effectively reduces overfitting to spurious correlations while maintaining standard convergence rates. The method demonstrates superior performance across various existing domain-generalized learning frameworks without requiring architectural changes.
One-Step Generalization Ratio Guided Optimization for Domain Generalization
arXiv:2606.16301v1 Announce Type: new Abstract: Domain Generalization (DG) aims to train models that generalize to unseen target domains but often overfit to domain-specific features, known as undesired correlations. Gradient-based DG methods typically guide gradients in a dominant direction but often inadvertently reinforce spurious correlations. Recent work has employed dropout to regularize overconfident parameters, but has not explicitly adjusted gradient alignment or ensured balanced parameter updates. We propose GENIE (Generalization-ENhancing Iterative Equalizer), a novel optimizer that leverages the One-Step Generalization Ratio (OSGR) to quantify each parameter's contribution to loss reduction and assess gradient alignment. By dynamically equalizing OSGR via a preconditioning factor, GENIE prevents a small subset of parameters from dominating optimization, thereby promoting domain-invariant feature learning. Theoretically, GENIE balances convergence contribution and gradient alignment among parameters, achieving higher OSGR while retaining SGD's convergence rate. Empirically, it outperforms existing optimizers and enhances performance when integrated with various DG and single-DG methods.
Abstract:Domain Generalization (DG) aims to train models that generalize to unseen target domains but often overfit to domain-specific features, known as undesired correlations. Gradient-based DG methods typically guide gradients in a dominant direction but often inadvertently reinforce spurious correlations. Recent work has employed dropout to regularize overconfident parameters, but has not explicitly adjusted gradient alignment or ensured balanced parameter updates. We propose GENIE (Generalization-ENhancing Iterative Equalizer), a novel optimizer that leverages the One-Step Generalization Ratio (OSGR) to quantify each parameter's contribution to loss reduction and assess gradient alignment. By dynamically equalizing OSGR via a preconditioning factor, GENIE prevents a small subset of parameters from dominating optimization, thereby promoting domain-invariant feature learning. Theoretically, GENIE balances convergence contribution and gradient alignment among parameters, achieving higher OSGR while retaining SGD's convergence rate. Empirically, it outperforms existing optimizers and enhances performance when integrated with various DG and single-DG methods.
Machine Learning (cs.LG); Machine Learning (stat.ML)
arXiv:2606.16301 [cs.LG]
(or arXiv:2606.16301v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.16301 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 242 cuvinte
Subiecte conexe
p-PSO: A Penalized Particle Swarm Optimization Technique for Finding D-Optimal Designs with Mixed Factors in Generalized Linear Models
arXiv · 16 iun. 2026
Regret Optimality of Sample Average Approximation for Data-Driven Newsvendor Problems: A General Optimization Perspective
arXiv · 21 iul. 2026
Balancing multiscale similarity and cartographic constraints: A similarity-driven optimization framework for line generalization
arXiv · 29 iul. 2026
tidyHEBO: Robust General-Purpose Bayesian Optimization with Model-Consistent Warping and Pareto Search
arXiv · 24 sept. 2026