aiqbench analysis
PRIVET introduces a novel algorithm that uses extreme value statistics to assign individual privacy leak scores to synthetic data samples, addressing gaps in existing global or uncalibrated proximity-based methods. This approach effectively detects memorization and subtle leakage across diverse modalities, including high-dimensional genetic data, even when models are globally underfit.
PRIVET: PRoximIty leakage detection Via Extreme value Theory
arXiv:2510.24233v2 Announce Type: replace Abstract: Deep generative models are often trained on sensitive data, such as genetic sequences, health data, or more broadly, any copyrighted, licensed or protected content. This raises critical concerns around privacy-preserving synthetic data, and more specifically around privacy leakage, an issue closely tied to overfitting. Existing proximity-based methods mostly assess privacy risk through global criteria, which quantify a model's overall behaviour but cannot attribute risk to an individual record. Sample-level outputs do exist but they are either uncalibrated, discontinuous, or blind to leakage occurring while the model is globally underfit, which limits their practical use. Using extreme value statistics on nearest-neighbor distances, we propose PRIVET, a generic sample-based, modality-agnostic algorithm that assigns an individual proximity leak score to each synthetic sample. These are evaluated under a chosen representation and distance, each synthetic sample being assigned a continuous score measuring how improbable its proximity to the training set is under a no-leakage model. We empirically demonstrate that PRIVET detects memorization and more subtle forms of proximity-based data leakage across diverse data modalities, including settings with very high dimensionality and limited sample sizes such as genetic data, and in underfitting regimes that overfitting-based diagnostics cannot reach by construction. Our analysis further shows that the representation bounds what any distance-based evaluation can detect, existing computer vision embeddings failing to yield perceptually meaningful distances for near-duplicate samples. Accordingly, a low score is evidence of leakage in the chosen metric, while its absence is not a certificate of privacy.
Abstract:Deep generative models are often trained on sensitive data, such as genetic sequences, health data, or more broadly, any copyrighted, licensed or protected content. This raises critical concerns around privacy-preserving synthetic data, and more specifically around privacy leakage, an issue closely tied to overfitting. Existing proximity-based methods mostly assess privacy risk through global criteria, which quantify a model's overall behaviour but cannot attribute risk to an individual record. Sample-level outputs do exist but they are either uncalibrated, discontinuous, or blind to leakage occurring while the model is globally underfit, which limits their practical use. Using extreme value statistics on nearest-neighbor distances, we propose PRIVET, a generic sample-based, modality-agnostic algorithm that assigns an individual proximity leak score to each synthetic sample. These are evaluated under a chosen representation and distance, each synthetic sample being assigned a continuous score measuring how improbable its proximity to the training set is under a no-leakage model. We empirically demonstrate that PRIVET detects memorization and more subtle forms of proximity-based data leakage across diverse data modalities, including settings with very high dimensionality and limited sample sizes such as genetic data, and in underfitting regimes that overfitting-based diagnostics cannot reach by construction. Our analysis further shows that the representation bounds what any distance-based evaluation can detect, existing computer vision embeddings failing to yield perceptually meaningful distances for near-duplicate samples. Accordingly, a low score is evidence of leakage in the chosen metric, while its absence is not a certificate of privacy.
Machine Learning (cs.LG)
arXiv:2510.24233 [cs.LG]
(or arXiv:2510.24233v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2510.24233 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 335 words
Related coverage
Handling Imbalanced Classification: What Works Better Than SMOTE
Analytics Vidhya · Jul 12, 2026
Research Note: Filtering Subversion-Relevant Information From Pretraining Data Is Feasible
LessWrong · Oct 5, 2026
LiveLaw Supreme Court Half-Yearly Digest: January - June 2026 [Part 2]
livelaw.in · Sep 13, 2026
How might continual learning affect safety and alignment?
LessWrong · Jun 13, 2026