DeepDefense: Robust Learning via Layer-Wise Gradient-Feature Alignment

arXiv:2511.13749v2 Announce Type: replace-cross Abstract: Deep neural networks are known to be vulnerable to adversarial perturbations, which are small, carefully crafted inputs that lead to incorrect predictions. In this paper, we propose DeepDefense, a novel defense framework that applies Gradient-Feature Alignment (GFA) regularization across multiple layers to suppress adversarial vulnerability. By aligning input gradients with internal feature representations, DeepDefense promotes a smoother loss landscape in tangential space, also known as feature space, thereby reducing the model's sensitivity to adversarial noise. We provide insights into how adversarial perturbations can be decomposed into radial and tangential components and demonstrate that alignment suppresses loss variation in tangential space, where most attacks are effective. Empirically, our method achieves significant improvements in robustness against both gradient-based and optimization-based attacks. For example, on CIFAR-10, CNN models trained with DeepDefense outperform standard adversarial training by up to 15.2 percent under APGD attacks and 24.7 percent under FGSM attacks. Against optimization-based attacks such as DeepFool and EADEN, DeepDefense requires perturbation magnitudes that are 20 to 30 times larger to cause misclassification, indicating stronger decision boundaries and a flatter loss landscape. Our approach is architecture-agnostic, simple to implement, and highly effective, offering a promising direction for improving the adversarial robustness of deep learning models.

Abstract:Deep neural networks are known to be vulnerable to adversarial perturbations, which are small, carefully crafted inputs that lead to incorrect predictions. In this paper, we propose DeepDefense, a novel defense framework that applies Gradient-Feature Alignment (GFA) regularization across multiple layers to suppress adversarial vulnerability. By aligning input gradients with internal feature representations, DeepDefense promotes a smoother loss landscape in tangential space, also known as feature space, thereby reducing the model's sensitivity to adversarial noise. We provide insights into how adversarial perturbations can be decomposed into radial and tangential components and demonstrate that alignment suppresses loss variation in tangential space, where most attacks are effective. Empirically, our method achieves significant improvements in robustness against both gradient-based and optimization-based attacks. For example, on CIFAR-10, CNN models trained with DeepDefense outperform standard adversarial training by up to 15.2 percent under APGD attacks and 24.7 percent under FGSM attacks. Against optimization-based attacks such as DeepFool and EADEN, DeepDefense requires perturbation magnitudes that are 20 to 30 times larger to cause misclassification, indicating stronger decision boundaries and a flatter loss landscape. Our approach is architecture-agnostic, simple to implement, and highly effective, offering a promising direction for improving the adversarial robustness of deep learning models.

Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

arXiv:2511.13749 [cs.LG]

(or arXiv:2511.13749v2 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2511.13749 Focus to learn more arXiv-issued DOI via DataCite

https://doi.org/10.1109/CCECE68150.2026.11610184 Focus to learn more DOI(s) linking to related resources

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

aiqbench analysis

DeepDefense: Robust Learning via Layer-Wise Gradient-Feature Alignment

Related coverage