aiqbench analysis
This study reveals a critical trade-off in synthetic data generation where GANs and DDPMs maintain superior fidelity and utility under privacy constraints, whereas VAEs degrade rapidly. The findings underscore the necessity for multidimensional evaluation frameworks that jointly assess fidelity, privacy, and utility, particularly when operating under data scarcity. For practitioners, this highlights that model selection must account for specific differential privacy mechanisms, as their impact varies significantly across architectures.
No Free Lunch for Synthetic Images under Data Scarcity Conditions
arXiv:2606.07640v1 Announce Type: cross Abstract: This study investigates the trade-offs between fidelity, privacy, and utility in synthetic data generation under conditions of data scarcity and privacy sensitivity. We propose an evaluation framework that jointly assesses these three dimensions and apply it to three widely used generative models, VAE, GAN, and DDPM. The evaluation spans three image datasets, MNIST, OCTMNIST, and OrganAMNIST, encompassing both general-purpose and medical imaging domains. Notable differences arise between the three models in their behaviour when differential privacy mechanisms are introduced during training. GAN and DDPM demonstrate greater robustness, maintaining higher fidelity and downstream utility across a range of noise levels, while VAE degrades more rapidly as privacy constraints increase. This study highlights the importance of a multidimensional evaluation of deep generative models, also noting that their behaviour significantly differs when privacy techniques are applied.
Abstract:This study investigates the trade-offs between fidelity, privacy, and utility in synthetic data generation under conditions of data scarcity and privacy sensitivity. We propose an evaluation framework that jointly assesses these three dimensions and apply it to three widely used generative models, VAE, GAN, and DDPM. The evaluation spans three image datasets, MNIST, OCTMNIST, and OrganAMNIST, encompassing both general-purpose and medical imaging domains. Notable differences arise between the three models in their behaviour when differential privacy mechanisms are introduced during training. GAN and DDPM demonstrate greater robustness, maintaining higher fidelity and downstream utility across a range of noise levels, while VAE degrades more rapidly as privacy constraints increase. This study highlights the importance of a multidimensional evaluation of deep generative models, also noting that their behaviour significantly differs when privacy techniques are applied.
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2606.07640 [cs.CV]
(or arXiv:2606.07640v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2606.07640 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 234 words
Related coverage
HELICS: Biobank-scale Conditional Synthetic Genome Generation via Latent Flow Matching
arXiv · Oct 28, 2026
Synthetic Data Augmentation for Functional Data: Wasserstein Risk Bounds and Diffusion-Based Conditional Generation
arXiv · Oct 27, 2026
How Synthetic Labels Improve Conformal Prediction: A Perspective on Conditional Coverage
arXiv · Sep 30, 2026
Is spectral predictability enough? Testing {\Omega}-matching for synthetic pretraining under distribution shift
arXiv · Oct 23, 2026