analiza aiqbench
This research explicitly links the Law of Robustness to robust generalization for arbitrary data distributions, clarifying how Lipschitz bounds behave under global versus local statistical learning conditions. The study demonstrates that while global Rademacher complexity preserves the order of these bounds, local scales introduce dependencies on perturbation radius and concentration terms. These findings provide a rigorous theoretical foundation for understanding the relationship between overparameterization and model stability in non-standard data regimes.
Does Order Matter : Connecting The Law of Robustness to Robust Generalization
arXiv:2602.20971v3 Announce Type: replace Abstract: Bubeck and Selke (2021) propose the connection between the Law of Robustness and robust generalization error as an open problem. The Law of Robustness states that overparameterization is necessary for models to interpolate robustly, i.e., the interpolating function is required to be Lipschitz. Wu et al. (2023) extend this law to arbitrary data distributions, proving that the Lipschitz constant satisfies $L = \Omega(n^{1/d})$. Robust generalization, on the other hand, asks whether small robust training loss implies small robust test loss. This can be studied using statistical learning techniques such as Rademacher complexities, where a bound on the Rademacher complexity of the robust loss class implies a bound on the Lipschitzness of the function class. We use this connection to explicitly link the two for arbitrary data distributions. (i) We prove that the order of the Lipschitz bound remains the same when considering the global Rademacher complexity of robust loss classes. (ii) At the local scale, i.e., for subsets of functions with small empirical error, the order of the Lipschitz bound changes with the perturbation radius $\rho$ and the localized concentration term $\sqrt{r/n}$.
Abstract:Bubeck and Selke (2021) propose the connection between the Law of Robustness and robust generalization error as an open problem. The Law of Robustness states that overparameterization is necessary for models to interpolate robustly, i.e., the interpolating function is required to be Lipschitz. Wu et al. (2023) extend this law to arbitrary data distributions, proving that the Lipschitz constant satisfies $L = \Omega(n^{1/d})$. Robust generalization, on the other hand, asks whether small robust training loss implies small robust test loss. This can be studied using statistical learning techniques such as Rademacher complexities, where a bound on the Rademacher complexity of the robust loss class implies a bound on the Lipschitzness of the function class. We use this connection to explicitly link the two for arbitrary data distributions. (i) We prove that the order of the Lipschitz bound remains the same when considering the global Rademacher complexity of robust loss classes. (ii) At the local scale, i.e., for subsets of functions with small empirical error, the order of the Lipschitz bound changes with the perturbation radius $\rho$ and the localized concentration term $\sqrt{r/n}$.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2602.20971 [cs.LG]
(or arXiv:2602.20971v3 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 196 cuvinte
Subiecte conexe
Deconfusing Corrigibility
AI Alignment Forum · 17 oct. 2026
How Would We Know? Reflections on trying to make AI go well amid deep uncertainty
LessWrong · 5 oct. 2026
The current bottleneck is political will, not research
AI Alignment Forum · 11 iul. 2026
Does preservation make sense before we know how to revive?
LessWrong · 15 iun. 2026