analiza aiqbench
This research advances the theoretical understanding of deep learning by establishing explicit max-information bounds for differentially private algorithms, bridging the gap between privacy guarantees and generalization performance. By leveraging internal randomization structures in DP-SGD and output perturbation methods, the study enables non-vacuous PAC-Bayes bounds where priors are learned directly from data rather than chosen independently. These findings provide critical insights for deploying robust machine learning models that maintain both high utility and strict privacy standards in sensitive applications.
From Privacy to Generalization: Linear Max-Information Bounds for Differentially Private Learning Algorithms
arXiv:2605.26222v2 Announce Type: replace Abstract: Understanding the relationship between generalization and privacy remains a challenge in modern machine learning theory, particularly for deep networks that are trained by variants of differentially private stochastic gradient descent (DP- SGD). In this work we make progress on this persistent open problem. First, we derive explicit upper bounds on the approximate max-information of any algorithm that fulfills $(\epsilon, \delta)$-differential privacy or R\'enyi differential privacy, thereby going beyond the classical results for pure $\epsilon$-differential privacy. Subsequently, we show even stronger guarantees for two common private learning algorithms, output perturbation with the Gaussian mechanism, and streaming DP-SGD, by exploiting the structure of their internal randomization. As an application of our results, we demonstrate how to obtain non-vacuous PAC-Bayes generalization bounds for deep networks, in which the prior distribution is learned by DP-SGD instead of the classical way of choosing it in a data-independent way.
Abstract:Understanding the relationship between generalization and privacy remains a challenge in modern machine learning theory, particularly for deep networks that are trained by variants of differentially private stochastic gradient descent (DP- SGD). In this work we make progress on this persistent open problem. First, we derive explicit upper bounds on the approximate max-information of any algorithm that fulfills $(\epsilon, \delta)$-differential privacy or Rényi differential privacy, thereby going beyond the classical results for pure $\epsilon$-differential privacy. Subsequently, we show even stronger guarantees for two common private learning algorithms, output perturbation with the Gaussian mechanism, and streaming DP-SGD, by exploiting the structure of their internal randomization. As an application of our results, we demonstrate how to obtain non-vacuous PAC-Bayes generalization bounds for deep networks, in which the prior distribution is learned by DP-SGD instead of the classical way of choosing it in a data-independent way.
Machine Learning (cs.LG); Machine Learning (stat.ML)
arXiv:2605.26222 [cs.LG]
(or arXiv:2605.26222v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2605.26222 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 238 cuvinte
Subiecte conexe
LiveLaw Supreme Court Half-Yearly Digest: January - June 2026 [Part 2]
livelaw.in · 13 sept. 2026
How Entanglement Reshapes the Geometry of Quantum Differential Privacy
arXiv · 27 iul. 2026
The Binary Tree Mechanism is Optimal for Differentially Private Continual Counting
arXiv · 21 sept. 2026
Import AI 470: No rights for machines; automating environment generation with SPADE; and building better GPU kernels with Hawkeye
Import AI · 24 aug. 2026