analiza aiqbench
This research exposes a critical vulnerability in current membership inference attacks when auditors lack access to representative harmful training data. By introducing the "unseen class" setting, the study demonstrates that quantile regression methods significantly outperform standard shadow models for detecting such risks without prior distributional knowledge. Practitioners must update their AI safety toolkits with these new techniques to effectively audit black-box models under real-world legal and ethical constraints.
Membership Inference Attacks for Unseen Classes
arXiv:2506.06488v3 Announce Type: replace Abstract: A key tool in developing safe AI models is \emph{data auditing}, i.e., using statistical tools to determine whether harmful content may have been used in the training data of a black-box model. Unfortunately, most \emph{membership inference attacks} (MIAs) used to perform this type of auditing themselves assume \emph{access} to examples of harmful content from the same distribution as the query data. In real-world auditing scenarios, auditors often face legal and ethical restrictions preventing them from accessing a representative set of samples of harmful content to train MIA models effectively. We abstract and formalize this setting into a new data access model, the ``unseen class'' setting, and show that the state of the art MIAs fail due to the lack of access to the full target distribution. We show in this setting, \emph{quantile regression attacks} outperform approaches typically considered to be SoTA. We demonstrate this both empirically and theoretically, showing that quantile regression attacks achieve up to \textbf{11$\times$ the TPR} of shadow model-based approaches in practice, and providing a theoretical model that outlines the generalization properties required for this approach to succeed. Our work identifies an important failure mode in existing MIAs and provides a cautionary tale for practitioners that aim to directly use existing tools for real-world applications of AI safety.
Abstract:A key tool in developing safe AI models is \emph{data auditing}, i.e., using statistical tools to determine whether harmful content may have been used in the training data of a black-box model. Unfortunately, most \emph{membership inference attacks} (MIAs) used to perform this type of auditing themselves assume \emph{access} to examples of harmful content from the same distribution as the query data. In real-world auditing scenarios, auditors often face legal and ethical restrictions preventing them from accessing a representative set of samples of harmful content to train MIA models effectively. We abstract and formalize this setting into a new data access model, the ``unseen class'' setting, and show that the state of the art MIAs fail due to the lack of access to the full target distribution. We show in this setting, \emph{quantile regression attacks} outperform approaches typically considered to be SoTA. We demonstrate this both empirically and theoretically, showing that quantile regression attacks achieve up to \textbf{11$\times$ the TPR} of shadow model-based approaches in practice, and providing a theoretical model that outlines the generalization properties required for this approach to succeed. Our work identifies an important failure mode in existing MIAs and provides a cautionary tale for practitioners that aim to directly use existing tools for real-world applications of AI safety.
Machine Learning (cs.LG); Cryptography and Security (cs.CR); Machine Learning (stat.ML)
arXiv:2506.06488 [cs.LG]
(or arXiv:2506.06488v3 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 229 cuvinte
Subiecte conexe
SoK: Rethinking Membership Inference in Tabular Machine Learning
arXiv · 14 oct. 2026
Membership Inference Attacks on Recommender System: A Survey
arXiv · 14 sept. 2026
Structured Driving-State Narratives for Small Language Model-Based GNSS Spoofing Detection
arXiv · 19 aug. 2026
Causal Evaluation of Membership Inference Attacks
arXiv · 8 iun. 2026