Precision-Varying Prediction (PVP): Robustifying ASR systems against adversarial attacks

Abstract:With the increasing deployment of automated and agentic systems, ensuring the adversarial robustness of automatic speech recognition (ASR) models has become highly relevant. We observe that changing the precision of an ASR model during inference reduces the likelihood of adversarial attacks to succeed. We take advantage of this fact to make models more robust simply by randomly sampling the precision during prediction. Moreover, this insight can be turned into an adversarial example detection strategy by implementing a simple Gaussian classifier that thresholds the differences between outputs of models run with different precision. To further enhance security boundaries, we combine the approach with an existing uncertainty-based defense mechanism, which forces adaptive adversaries to introduce highly perceptible noise to bypass detection. An experimental analysis across various ASR models, languages, and attack types demonstrates a significant increase in adversarial robustness, competitive detection capabilities, and resistance to adaptive threats.

Machine Learning (cs.LG); Cryptography and Security (cs.CR); Audio and Speech Processing (eess.AS)

arXiv:2603.22590 [cs.LG]

(or arXiv:2603.22590v2 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2603.22590 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.