aiqbench analysis
Sharpness and complexity together explain generalization better than either alone, but the paper is candid about where that breaks down. Results depend heavily on the sharpness perturbation scale and on how complexity is represented, so the "two-factor" conclusion is conditional rather than general. For ML practitioners, the useful takeaway is that generalization metric choices can change conclusions substantially, and the unexplained failures remain open.
How Far Can Sharpness and Complexity Jointly Explain Generalization?
arXiv:2606.29043v2 Announce Type: replace Abstract: Large-scale empirical studies have revealed that many widely used generalization measures correlate only weakly with generalization, suggesting that relying on a single factor is insufficient to explain it. In this work, we examine sharpness and complexity jointly, placing the reach of the two-factor view at the center: how far can they jointly explain generalization, what determines when they succeed, and where do they fall short? We investigate these questions quantitatively, using sixteen alternative metric pairs as probes and Pareto analysis and linear regression as evaluation tools. Our results show that suitably defined $(S,C)$ pairs provide strong explanatory power across a range of settings. The sharpness perturbation scale substantially affects this success, and the complexity representation influences its robustness. The remaining failures leave open whether more refined measurements would suffice or additional factors are needed.
Title:How Far Can Sharpness and Complexity Jointly Explain Generalization?
Abstract:Large-scale empirical studies have revealed that many widely used generalization measures correlate only weakly with generalization, suggesting that relying on a single factor is insufficient to explain it. In this work, we examine sharpness and complexity jointly, placing the reach of the two-factor view at the center: how far can they jointly explain generalization, what determines when they succeed, and where do they fall short? We investigate these questions quantitatively, using sixteen alternative metric pairs as probes and Pareto analysis and linear regression as evaluation tools. Our results show that suitably defined $(S,C)$ pairs provide strong explanatory power across a range of settings. The sharpness perturbation scale substantially affects this success, and the complexity representation influences its robustness. The remaining failures leave open whether more refined measurements would suffice or additional factors are needed.
Machine Learning (cs.LG)
arXiv:2606.29043 [cs.LG]
(or arXiv:2606.29043v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.29043 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 235 words
Related coverage
An Empirical Study of Data Scale, Model Complexity, and Input Modalities in Visual Generalization
arXiv · Jun 5, 2026
An Empirical Study of Data Scale, Model Complexity, and Input Modalities in Visual Generalization
arXiv · Jun 9, 2026
Dimension-Free Scaling Laws for Size Generalization: An Exponential Separation
arXiv · Oct 15, 2026
Generalization and infinite width
LessWrong · Aug 1, 2026