analiza aiqbench
This paper proposes a post-hoc, model-agnostic way to explain multimodal classifiers by retrieving real training examples from the target class as counterfactuals. Its main technical claim is that intersecting per-modality nearest-neighbour sets (Multimodal Consensus Retrieval) gives more coherent explanations than attributing each modality separately. The evidence is from four datasets and three fusion strategies, with the authors noting that unimodal retrieval stays competitive when one modality dominates. Readers should weigh the benchmark scope and how well retrieved neighbours serve as explanations in practice, since plausibility here comes from grounding in the training data rather than from a formal guarantee.
CRANE: Post-Hoc Counterfactual Retrieval for Multimodal Classifiers
arXiv:2610.31594v1 Announce Type: new Abstract: Counterfactual explanation methods have been introduced for unimodal settings, with dedicated frameworks for tabular, timeseries, text, and image data, yet no post-hoc method has been designed for classifiers that learn from multimodal inputs. Existing explainability methods for multimodal classifiers apply modality-specific attribution independently to each input, which fails to yield coherent explanations and does not transfer to arbitrary pre-trained multimodal architectures. To address these shortcomings, we present CRANE (Counterfactual Retrieval via Agnostic Nearest-neighbour Explanations), the first post-hoc, model-agnostic framework for nearest-neighbour multimodal counterfactual generation that retrieves real training examples from the target class as candidate explanations, ensuring plausibility by construction. Across four heterogeneous datasets and three fusion strategies, we show that Multimodal Consensus Retrieval (MC-R), which intersects per-modality neighbour sets to identify instances jointly close to the query in every modality, emerges as the strongest multimodal strategy, achieving the best or joint-best outcome on three of four datasets while grounding every explanation in the joint training distribution. Unimodal strategies remain competitive when a single modality dominates the decision boundary.
Abstract:Counterfactual explanation methods have been introduced for unimodal settings, with dedicated frameworks for tabular, timeseries, text, and image data, yet no post-hoc method has been designed for classifiers that learn from multimodal inputs. Existing explainability methods for multimodal classifiers apply modality-specific attribution independently to each input, which fails to yield coherent explanations and does not transfer to arbitrary pre-trained multimodal architectures. To address these shortcomings, we present CRANE (Counterfactual Retrieval via Agnostic Nearest-neighbour Explanations), the first post-hoc, model-agnostic framework for nearest-neighbour multimodal counterfactual generation that retrieves real training examples from the target class as candidate explanations, ensuring plausibility by construction. Across four heterogeneous datasets and three fusion strategies, we show that Multimodal Consensus Retrieval (MC-R), which intersects per-modality neighbour sets to identify instances jointly close to the query in every modality, emerges as the strongest multimodal strategy, achieving the best or joint-best outcome on three of four datasets while grounding every explanation in the joint training distribution. Unimodal strategies remain competitive when a single modality dominates the decision boundary.
Machine Learning (cs.LG)
arXiv:2610.31594 [cs.LG]
(or arXiv:2610.31594v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.31594 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 263 cuvinte
Subiecte conexe
Counterfactual Explainability Framework With CycleGAN And Counterfactual-Classifier Alignnment Score for Retinal Disease Classification
arXiv · 24 iul. 2026
Towards Fast and Disentangled Counterfactuals for Visual Foundation Models
arXiv · 2 oct. 2026
Counterfactual Transition Graphs: Evaluating Cross-Class Transition Quality
arXiv · 25 aug. 2026
A Geometric View of Counterfactual Behavior: Interaction of Boundary Proximity and Local Support
arXiv · 5 iun. 2026