analiza aiqbench
The paper introduces a benchmark of 216k adversarial images, built from four text-to-image generators and attacks optimized against an ensemble of seven detectors, to test whether AI-generated image detectors hold up against evasion. Its main finding is that adversarial examples transfer to detectors outside the attack ensemble, so the benchmark works as a worst-case robustness screen. Readers evaluating detection tools for fraud or disinformation should treat accuracy claims without adversarial testing as incomplete.
RAID: A Dataset for Testing the Adversarial Robustness of AI-Generated Image Detectors
arXiv:2506.03988v4 Announce Type: replace-cross Abstract: AI-generated images (AIGIs) have reached a level of quality at which humans are unable to reliably distinguish them from real images. To counter the inherent risks of fraud and disinformation, their detection is a pressing challenge and an active research topic. While many of the presented methods claim high detection accuracy, they are often evaluated without considering their adversarial robustness. In real-world scenarios, malicious actors can modify AIGIs, e.g., through imperceptible perturbations, to evade detection systems. In this work, we provide a dataset to assess the robustness of AI-generated image detectors. We present RAID (Robust evaluation of AI-generated image Detectors), a benchmark dataset of 216k diverse and transferable adversarial images. The proposed dataset is created starting from images generated by four different text-to-image models (to promote diversity) and optimizing attacks against an ensemble of seven state-of-the-art detectors (to promote transferability). Extensive experiments show that our methodology generates adversarial images that transfer to unseen detectors, degrading the performance of every detector outside the attack ensemble that we evaluate, and provides a practical, worst-case robustness screen: failure on RAID is a strong indicator that a detector is not adversarially robust. We release our dataset at https://huggingface.co/datasets/aimagelab/RAID and evaluation code at https://github.com/pralab/RAID.
Abstract:AI-generated images (AIGIs) have reached a level of quality at which humans are unable to reliably distinguish them from real images. To counter the inherent risks of fraud and disinformation, their detection is a pressing challenge and an active research topic. While many of the presented methods claim high detection accuracy, they are often evaluated without considering their adversarial robustness. In real-world scenarios, malicious actors can modify AIGIs, e.g., through imperceptible perturbations, to evade detection systems. In this work, we provide a dataset to assess the robustness of AI-generated image detectors. We present RAID (Robust evaluation of AI-generated image Detectors), a benchmark dataset of 216k diverse and transferable adversarial images. The proposed dataset is created starting from images generated by four different text-to-image models (to promote diversity) and optimizing attacks against an ensemble of seven state-of-the-art detectors (to promote transferability). Extensive experiments show that our methodology generates adversarial images that transfer to unseen detectors, degrading the performance of every detector outside the attack ensemble that we evaluate, and provides a practical, worst-case robustness screen: failure on RAID is a strong indicator that a detector is not adversarially robust. We release our dataset at this https URL and evaluation code at this https URL.
24 pages, 5 figures. Updated version: RAID dataset now contains 216k transferable adversarial images (PGD, APGD, and CWA attack splits), and evaluations on five additional unseen detectors
Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
arXiv:2506.03988 [cs.CV]
(or arXiv:2506.03988v4 [cs.CV] for this version)
Text integral sub licența deschisă a publicației · 247 cuvinte