CASE: Contrastive Activation for Class-Sensitive Explanations

Abstract:Saliency methods are widely used to visualize which input features are deemed relevant to a model's prediction. However, their visual plausibility can obscure critical limitations. In this work, we propose a diagnostic test for class sensitivity: a method's ability to distinguish between competing class labels on the same input. Through extensive experiments, we show that many widely used saliency methods produce nearly identical explanations regardless of the class label, calling into question their reliability. We find that class-insensitive behavior persists across architectures and datasets, suggesting the failure mode is structural rather than model-specific. Motivated by these findings, we introduce CASE, a contrastive explanation method that isolates features uniquely discriminative for the predicted class. We evaluate CASE using the proposed diagnostic and a perturbation-based fidelity test, and show that it produces faithful and more class-specific explanations than existing methods.

Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

I.2.6; I.5.1; I.5.5; I.2.10

arXiv:2506.07327 [cs.CV]

(or arXiv:2506.07327v4 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2506.07327 Focus to learn more arXiv-issued DOI via DataCite

https://doi.org/10.1007/s10994-026-07169-w Focus to learn more DOI(s) linking to related resources

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.