aiqbench analysis
This research introduces a novel, attack-agnostic robustness metric grounded in the spectral norm of the Fisher Information Matrix, offering a theoretically grounded alternative to traditional adversarial testing. By establishing closed-form bounds for major architectures like Transformers and ResNets, the work provides an interpretable diagnostic tool that correlates strongly with actual adversarial vulnerability. The proposed scalable algorithms enable practical evaluation across diverse domains, from standard benchmarks to medical imaging, to guide the design of safer deep learning models.
Measuring Model Robustness via Fisher Information: Spectral Bounds, Theoretical Guarantees, and Practical Algorithms
arXiv:2606.04767v1 Announce Type: new Abstract: The robustness of deep neural networks is crucial for safety-critical deployments, yet existing evaluation methods are often attack-dependent and lack interpretability. We propose a principled, attack-agnostic robustness metric based on the spectral norm of the Fisher Information Matrix (FIM), which quantifies the worst-case sensitivity of the model's output distribution to input perturbations. Theoretically, we establish that the FIM equals the variance of the input Jacobian and derive closed-form spectral bounds for common architectures, including VGG, ResNet, DenseNet, and Transformer, providing the first theoretical robustness ranking. To enable scalable evaluation, we develop efficient algorithms, including power iteration and Hutchinson-based estimation, that support both white-box and black-box settings. Extensive experiments across multiple datasets, including CIFAR, ImageNet, and medical images, and across multiple architectures show a strong correlation between our metric and adversarial vulnerability. Our framework serves as an interpretable diagnostic tool that complements attack-based evaluations, offering insights into architectural sensitivity and guiding the design of more robust models. Code is available at: https://github.com/franz-chang/SRP/.
Abstract:The robustness of deep neural networks is crucial for safety-critical deployments, yet existing evaluation methods are often attack-dependent and lack interpretability. We propose a principled, attack-agnostic robustness metric based on the spectral norm of the Fisher Information Matrix (FIM), which quantifies the worst-case sensitivity of the model's output distribution to input perturbations. Theoretically, we establish that the FIM equals the variance of the input Jacobian and derive closed-form spectral bounds for common architectures, including VGG, ResNet, DenseNet, and Transformer, providing the first theoretical robustness ranking. To enable scalable evaluation, we develop efficient algorithms, including power iteration and Hutchinson-based estimation, that support both white-box and black-box settings. Extensive experiments across multiple datasets, including CIFAR, ImageNet, and medical images, and across multiple architectures show a strong correlation between our metric and adversarial vulnerability. Our framework serves as an interpretable diagnostic tool that complements attack-based evaluations, offering insights into architectural sensitivity and guiding the design of more robust models. Code is available at: this https URL.
Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2606.04767 [cs.LG]
(or arXiv:2606.04767v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.04767 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 263 words
Related coverage
Symmetry in quantum systems tied to hidden information
Quantum Zeitgeist · Oct 13, 2026
Does preservation make sense before we know how to revive?
LessWrong · Jun 15, 2026
How can LLM RL Work Despite Information-Theoretic Inefficiency
LessWrong · Oct 16, 2026
When Does Text Inform? Benchmarking Information-Theoretic Metrics for Multimodal Time-Series Forecasting
arXiv · Sep 12, 2026