aiqbench analysis
This research establishes formal conditions under which Concept Bottleneck Models (CBMs) can provide both faithful and compact explanations, challenging the assumption that high-level concepts always yield concise interpretations. The authors demonstrate that without specific architectural constraints like probabilistic modeling and group lasso sparsification, faithful explanations often require including all bottleneck concepts, thereby negating interpretability benefits. These findings offer critical guidance for developers aiming to deploy reliable explainable AI systems in vision-language models and other complex architectures.
When Are Concept Bottleneck Model Explanations Faithful and Compact?
arXiv:2610.06285v1 Announce Type: cross Abstract: Concept bottleneck models (CBMs) are neural classifiers that allow to explain their decisions via high-level concepts, potentially enabling understanding, steering and debugging. However, their explanations are often derived heuristically. Building on formal explainability, we argue they should also be faithful, i.e., not misreport which concepts actually matter. We show that, for widespread CBM architectures, including recent VLM-based variants, faithful explanations must include all concepts in the bottleneck, compromising interpretability when this is large. This result applies to both heuristic and faithful-by-construction formal explanations. To encourage the existence of compact faithful explanations, we suggest i) modeling concepts probabilistically as binary or categorical random variables (rather than logits), and ii) employing per-concept training-time sparsification via group lasso (rather than regular elastic net). We also extend algorithms from formal explainability to CBMs, and show they outperform natural heuristics in terms of guarantees and explanation size. Overall, our work warns against naive interpretability claims and provides formal conditions and practical strategies for ensuring CBMs are as interpretable as advertised.
Title:When Are Concept Bottleneck Model Explanations Faithful and Compact?
Abstract:Concept bottleneck models (CBMs) are neural classifiers that allow to explain their decisions via high-level concepts, potentially enabling understanding, steering and debugging. However, their explanations are often derived heuristically. Building on formal explainability, we argue they should also be faithful, i.e., not misreport which concepts actually matter. We show that, for widespread CBM architectures, including recent VLM-based variants, faithful explanations must include all concepts in the bottleneck, compromising interpretability when this is large. This result applies to both heuristic and faithful-by-construction formal explanations. To encourage the existence of compact faithful explanations, we suggest i) modeling concepts probabilistically as binary or categorical random variables (rather than logits), and ii) employing per-concept training-time sparsification via group lasso (rather than regular elastic net). We also extend algorithms from formal explainability to CBMs, and show they outperform natural heuristics in terms of guarantees and explanation size. Overall, our work warns against naive interpretability claims and provides formal conditions and practical strategies for ensuring CBMs are as interpretable as advertised.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
arXiv:2610.06285 [cs.LG]
(or arXiv:2610.06285v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.06285 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 274 words
Related coverage
Latent reasoning architectures would likely undermine CoT, our strongest oversight tool
AI Alignment Forum · Sep 23, 2026
[AINews] not much happened today
Latent Space · Sep 18, 2026
Spatially Grounded Concept Bottleneck Models for Trustworthy Breast Ultrasound Diagnosis
arXiv · Jul 24, 2026
The current bottleneck is political will, not research
AI Alignment Forum · Jul 11, 2026