How Far Can Sharpness and Complexity Jointly Explain Generalization?

Title:How Far Can Sharpness and Complexity Jointly Explain Generalization?

Abstract:Large-scale empirical studies have revealed that many widely used generalization measures correlate only weakly with generalization, suggesting that relying on a single factor is insufficient to explain it. In this work, we examine sharpness and complexity jointly, placing the reach of the two-factor view at the center: how far can they jointly explain generalization, what determines when they succeed, and where do they fall short? We investigate these questions quantitatively, using sixteen alternative metric pairs as probes and Pareto analysis and linear regression as evaluation tools. Our results show that suitably defined $(S,C)$ pairs provide strong explanatory power across a range of settings. The sharpness perturbation scale substantially affects this success, and the complexity representation influences its robustness. The remaining failures leave open whether more refined measurements would suffice or additional factors are needed.

Machine Learning (cs.LG)

arXiv:2606.29043 [cs.LG]

(or arXiv:2606.29043v2 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2606.29043 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.