aiqbench analysis
This research establishes a unified Rényi divergence framework to rigorously quantify the inherent differential privacy guarantees provided by sampling from Bayesian posteriors without injecting external noise. By analyzing posterior exponential moments across various model structures, the authors derive tighter privacy loss bounds and broaden the applicability of these "free" privacy mechanisms to diverse statistical settings. The work advances theoretical machine learning by formally linking Bayesian inference procedures to robust privacy metrics under different tail regimes.
On Inherent Privacy of Posterior Sampling: A Unified R\'enyi-Divergence Framework
arXiv:2610.19573v1 Announce Type: cross Abstract: Differential Privacy (DP) provides a rigorous framework for quantifying privacy guarantees. While most methods achieve DP by injecting calibrated random noise, the intrinsic randomness of certain procedures such as sampling can yield DP guarantees ``for free''. In this work, we develop a unified R\'enyi divergence framework to characterize the inherent DP guarantees of sampling from a Bayesian posterior distribution. We show that the privacy loss incurred by One-Posterior-Sampling (OPS) is governed by posterior exponential moments of the single-record log-likelihood ratio, and establish explicit DP guarantees under uniformly bounded, sub-Gaussian, or sub-exponential tail regimes. We apply the framework to representative Bayesian models, including categorical likelihoods with arbitrary priors, Gaussian-Gaussian conjugate pair, generalized linear models from the exponential family and linear regression with Gaussian likelihood, both with Gaussian priors on the regression coefficients. Our results elucidate sample size, model structure, neighboring relations in DP (bounded or unbounded), and how prior if applicable, jointly determine the inherent DP guarantees of OPS. Furthermore, our results recover the existing DP guarantees for OPS as special cases -- often with tighter privacy loss bounds -- and substantially broaden the class of Bayesian models for which the inherent DP guarantees of OPS can be rigorously characterized.
Abstract:Differential Privacy (DP) provides a rigorous framework for quantifying privacy guarantees. While most methods achieve DP by injecting calibrated random noise, the intrinsic randomness of certain procedures such as sampling can yield DP guarantees ``for free''. In this work, we develop a unified Rényi divergence framework to characterize the inherent DP guarantees of sampling from a Bayesian posterior distribution. We show that the privacy loss incurred by One-Posterior-Sampling (OPS) is governed by posterior exponential moments of the single-record log-likelihood ratio, and establish explicit DP guarantees under uniformly bounded, sub-Gaussian, or sub-exponential tail regimes. We apply the framework to representative Bayesian models, including categorical likelihoods with arbitrary priors, Gaussian-Gaussian conjugate pair, generalized linear models from the exponential family and linear regression with Gaussian likelihood, both with Gaussian priors on the regression coefficients. Our results elucidate sample size, model structure, neighboring relations in DP (bounded or unbounded), and how prior if applicable, jointly determine the inherent DP guarantees of OPS. Furthermore, our results recover the existing DP guarantees for OPS as special cases -- often with tighter privacy loss bounds -- and substantially broaden the class of Bayesian models for which the inherent DP guarantees of OPS can be rigorously characterized.
Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)
arXiv:2610.19573 [stat.ML]
(or arXiv:2610.19573v1 [stat.ML] for this version)
https://doi.org/10.48550/arXiv.2610.19573 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 299 words
Related coverage
The Tilted Sampling Problem: Optimism, Posterior Sampling, and Logarithmic Regret
arXiv · Oct 16, 2026
Bernoulli Flow Models: Self-Consistent Generative Modeling for Binary Data
arXiv · Oct 9, 2026
Not all Jensen-Shannon Divergence Estimators are Equal
arXiv · Jun 16, 2026
TabMosaic: Unified In-Context Diffusion for Tabular Generative Tasks
arXiv · Oct 15, 2026