aiqbench analysis
This arXiv preprint asks when algorithmic stability explains why overparameterized models generalize despite interpolating their training data, and proposes a contractive-sensitivity certificate as a test. Its strongest claim, that stability fails to explain some low-risk interpolating regimes, is a useful counterpoint to stability-based accounts. However, the accompanying note says Theorem 5.2's necessity direction and sharp boundary lack a complete proof, so readers should treat the boundary result as unverified and the empirical link between certificate growth and generalization as the more reliable takeaway.
A Function-Space Stability Boundary for Generalization in Interpolating Learning Systems
arXiv:2602.03514v3 Announce Type: replace Abstract: Modern learning systems often interpolate training data while still generalizing well, yet it remains unclear when algorithmic stability explains this behavior. We model training as a function-space trajectory and measure sensitivity to single-sample perturbations along this trajectory. We propose a contractive propagation condition and a stability certificate obtained by unrolling the resulting recursion. A small certificate implies stability-based generalization, while we also prove that there exist interpolating regimes with small risk where such contractive sensitivity cannot hold, showing that stability is not a universal explanation. Experiments confirm that certificate growth predicts generalization differences across optimizers, step sizes, and dataset perturbations. The framework therefore identifies regimes where stability explains generalization and where alternative mechanisms must account for success.
Abstract:Modern learning systems often interpolate training data while still generalizing well, yet it remains unclear when algorithmic stability explains this behavior. We model training as a function-space trajectory and measure sensitivity to single-sample perturbations along this trajectory. We propose a contractive propagation condition and a stability certificate obtained by unrolling the resulting recursion. A small certificate implies stability-based generalization, while we also prove that there exist interpolating regimes with small risk where such contractive sensitivity cannot hold, showing that stability is not a universal explanation. Experiments confirm that certificate growth predicts generalization differences across optimizers, step sizes, and dataset perturbations. The framework therefore identifies regimes where stability explains generalization and where alternative mechanisms must account for success.
Theorem 5.2 does not establish the required quantitative neighbouring-dataset instability from its cited benign-overfitting setting. The claimed necessity direction and sharp stability boundary therefore lack a complete proof
Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)
arXiv:2602.03514 [cs.LG]
(or arXiv:2602.03514v3 [cs.LG] for this version)
Full text under the publisher's open licence · 163 words