aiqbench analysis
This research advances black-box privacy auditing for differentially private models by optimizing the "canary" dataset used in membership inference attacks via metagradient descent. The authors demonstrate that this optimization significantly tightens lower bounds on privacy parameters, offering a more rigorous assessment of DP-SGD mechanisms without requiring access to training internals. By proving the method is agnostic to specific DP-SGD implementations and efficient across model scales, the work provides a critical tool for evaluating the real-world security guarantees of private machine learning systems.
Optimizing Canaries for Privacy Auditing with Metagradient Descent
arXiv:2507.15836v2 Announce Type: replace Abstract: In this work we study black-box privacy auditing, where the goal is to lower bound the privacy parameter of a differentially private learning algorithm using only the algorithm's outputs (i.e., final trained model). For DP-SGD (the most successful method for training differentially private deep learning models), the canonical auditing approach uses membership inference - an auditor comes with a small set of special "canary" examples, inserts a random subset of them into the training set, and then tries to discern which of their canaries were included in the training set (typically via a membership inference attack). The auditor's success rate then provides a lower bound on the privacy parameters of the learning algorithm. Our main contribution is a method for optimizing the auditor's canary set to improve privacy auditing, leveraging recent work on metagradient optimization (Engstrom et al., 2025). Our empirical evaluation demonstrates that in certain instances, using such optimized canaries can improve empirical lower bounds for differentially private image classification models by several times when compared to canaries proposed in prior work. Furthermore, we demonstrate that our method is DP-SGD agnostic and efficient: canaries optimized for non-private SGD with a small model architecture remain effective when auditing larger models trained with DP-SGD.
Abstract:In this work we study black-box privacy auditing, where the goal is to lower bound the privacy parameter of a differentially private learning algorithm using only the algorithm's outputs (i.e., final trained model). For DP-SGD (the most successful method for training differentially private deep learning models), the canonical auditing approach uses membership inference - an auditor comes with a small set of special "canary" examples, inserts a random subset of them into the training set, and then tries to discern which of their canaries were included in the training set (typically via a membership inference attack). The auditor's success rate then provides a lower bound on the privacy parameters of the learning algorithm. Our main contribution is a method for optimizing the auditor's canary set to improve privacy auditing, leveraging recent work on metagradient optimization (Engstrom et al., 2025). Our empirical evaluation demonstrates that in certain instances, using such optimized canaries can improve empirical lower bounds for differentially private image classification models by several times when compared to canaries proposed in prior work. Furthermore, we demonstrate that our method is DP-SGD agnostic and efficient: canaries optimized for non-private SGD with a small model architecture remain effective when auditing larger models trained with DP-SGD.
Machine Learning (cs.LG); Cryptography and Security (cs.CR)
arXiv:2507.15836 [cs.LG]
(or arXiv:2507.15836v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2507.15836 Focus to learn more arXiv-issued DOI via DataCite
International Conference on Learning Representations (ICLR) 2026, pp. 92994-93013
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 308 words
Related coverage
Detectability without Interference: Efficient Canary Design for Privacy Auditing in One Run
arXiv · Oct 16, 2026
Research Note: Filtering Subversion-Relevant Information From Pretraining Data Is Feasible
LessWrong · Oct 5, 2026
How Smartsheet built a remote MCP server on AWS
AWS · Jul 17, 2026
MotherTree: Meta-learning on synthetic data improves decision tree training
arXiv · Oct 9, 2026