aiqbench analysis
This research establishes the first provable vulnerabilities in 2-layer ReLU neural networks, demonstrating how implicit bias can be exploited for membership inference and data reconstruction attacks. The findings are critical for understanding the theoretical limits of privacy guarantees in shallow models within high-dimensional or univariate settings.
Provable Privacy Attacks on Trained Shallow Neural Networks
arXiv:2410.07632v3 Announce Type: replace Abstract: We study what provable privacy attacks can be shown for trained 2-layer ReLU neural networks, focusing on two types of attacks: membership inference and data reconstruction. We prove that theoretical results on the implicit bias of 2-layer neural networks can be used to provably identify with high probability whether a given point was used in the training set in a high-dimensional, nearly orthogonal setting, and can also be used to construct a finite set of which at least a constant fraction are training points in a univariate setting. To the best of our knowledge, our work is the first to show provable vulnerabilities in this implicit-bias-driven setting.
Abstract:We study what provable privacy attacks can be shown for trained 2-layer ReLU neural networks, focusing on two types of attacks: membership inference and data reconstruction. We prove that theoretical results on the implicit bias of 2-layer neural networks can be used to provably identify with high probability whether a given point was used in the training set in a high-dimensional, nearly orthogonal setting, and can also be used to construct a finite set of which at least a constant fraction are training points in a univariate setting. To the best of our knowledge, our work is the first to show provable vulnerabilities in this implicit-bias-driven setting.
Machine Learning (cs.LG); Cryptography and Security (cs.CR)
arXiv:2410.07632 [cs.LG]
(or arXiv:2410.07632v3 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2410.07632 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 203 words
Related coverage
Private and interpretable clinical prediction with quantum-inspired tensor train models
arXiv · Aug 28, 2026
Are LLM-Enhanced GNNs Privacy-Safe?
arXiv · Aug 27, 2026
Provable one-poison backdoor attacks on linear models and ReLU neural networks
arXiv · Aug 28, 2026
CertDW: Towards Certified Dataset Ownership Verification via Conformal Calibration
arXiv · Sep 12, 2026