analiza aiqbench
This research extends the theory of benign overfitting in linear classifiers by incorporating bias terms, revealing how intercepts alter noise constraints and dimensionality requirements under anisotropic conditions. The findings clarify that while isotropic noise regimes remain largely unaffected, label-noisy environments introduce specific covariance-dependent limitations to generalization thresholds. This theoretical advancement is crucial for understanding the robustness of overparameterized models in practical settings where bias terms are standard.
Benign Overfitting in Linear Classifiers with a Bias Term
arXiv:2511.12840v2 Announce Type: replace-cross Abstract: Overparameterized models often generalize well even when they interpolate noisy training data. This is known as benign overfitting. For linear classification, Hashimoto et al. (2025) analyzed the phenomenon under a broad class of mixture distributions, but only for homogeneous classifiers without a bias term. We extend their framework to classifiers with an intercept. Benign overfitting still occurs, but the intercept perturbs the normalized Gram matrix of the noise and creates extra constraints on the covariance. These constraints are strongest with label noise. Their effect depends on the covariance: under isotropic noise they are dominated by the homogeneous conditions, while in anisotropic or noisy regimes they can raise the dimensionality needed for benign generalization. Thus the bias term changes the theory in some covariance regimes and leaves the asymptotic thresholds unchanged in others.
Abstract:Overparameterized models often generalize well even when they interpolate noisy training data. This is known as benign overfitting. For linear classification, Hashimoto et al. (2025) analyzed the phenomenon under a broad class of mixture distributions, but only for homogeneous classifiers without a bias term. We extend their framework to classifiers with an intercept. Benign overfitting still occurs, but the intercept perturbs the normalized Gram matrix of the noise and creates extra constraints on the covariance. These constraints are strongest with label noise. Their effect depends on the covariance: under isotropic noise they are dominated by the homogeneous conditions, while in anisotropic or noisy regimes they can raise the dimensionality needed for benign generalization. Thus the bias term changes the theory in some covariance regimes and leaves the asymptotic thresholds unchanged in others.
Machine Learning (stat.ML); Machine Learning (cs.LG)
arXiv:2511.12840 [stat.ML]
(or arXiv:2511.12840v2 [stat.ML] for this version)
https://doi.org/10.48550/arXiv.2511.12840 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 227 cuvinte
Subiecte conexe
Benign Overfitting for General Norms and Distributions
arXiv · 30 sept. 2026
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · 8 oct. 2026
Evaluating task vectors, unlearning and inoculation
LessWrong · 20 sept. 2026
Spectral-Transport Stability and Benign Overfitting in Interpolating Learning
arXiv · 21 iul. 2026