aiqbench analysis
The study reports that in a 1,511-participant pre-registered experiment, showing users how an AI model reasons did little to raise their trust, while outcome feedback had a larger, more reliable effect. Neither factor produced large gains in task performance, so higher trust did not reliably translate into better human-AI collaboration. For readers building or deploying AI, this suggests that explainability features should be tested for their actual effect on trust and performance rather than assumed to help.
Impact of Model Interpretability and Outcome Feedback on Trust in AI
arXiv:2111.08222v2 Announce Type: replace Abstract: This paper bridges the gap in Human-Computer Interaction (HCI) research by comparatively assessing the effects of interpretability and outcome feedback on user trust and collaborative performance with AI. Through novel pre-registered experiments (N=1,511 total participants) using an interactive prediction task, we analyzed how interpretability and outcome feedback influence users' task performance and trust in AI. The results counter the widespread belief that interpretability drives trust, showing that interpretability led to no robust improvements in trust and that outcome feedback had a significantly greater and more reliable effect. However, both factors had modest effects on participants' task performance. These findings suggest that (1) interpretability may be less effective at increasing trust than factors like outcome feedback, and (2) augmenting human performance via AI systems may not be a simple matter of increasing trust in AI, as increased trust is not always associated with equally sizable performance improvements. Our exploratory analyses further delve into the mechanisms underlying this trust-performance paradox. These findings present an opportunity for research to focus not only on methods for generating interpretations but also on techniques that ensure interpretations impact trust and performance in practice.
Abstract:This paper bridges the gap in Human-Computer Interaction (HCI) research by comparatively assessing the effects of interpretability and outcome feedback on user trust and collaborative performance with AI. Through novel pre-registered experiments (N=1,511 total participants) using an interactive prediction task, we analyzed how interpretability and outcome feedback influence users' task performance and trust in AI. The results counter the widespread belief that interpretability drives trust, showing that interpretability led to no robust improvements in trust and that outcome feedback had a significantly greater and more reliable effect. However, both factors had modest effects on participants' task performance. These findings suggest that (1) interpretability may be less effective at increasing trust than factors like outcome feedback, and (2) augmenting human performance via AI systems may not be a simple matter of increasing trust in AI, as increased trust is not always associated with equally sizable performance improvements. Our exploratory analyses further delve into the mechanisms underlying this trust-performance paradox. These findings present an opportunity for research to focus not only on methods for generating interpretations but also on techniques that ensure interpretations impact trust and performance in practice.
25 pages. Published version, CHI 2024
Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
arXiv:2111.08222 [cs.AI]
(or arXiv:2111.08222v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2111.08222 Focus to learn more arXiv-issued DOI via DataCite
https://doi.org/10.1145/3613904.3642780 Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 298 words
Related coverage
Beyond Score Prediction: LLM-Based Essay Scoring and Feedback Generation via Reinforcement Learning with Rubric Rewards
arXiv · Jul 23, 2026
Hamiltonian and double-bracket flow formulations of quantum measurements
arXiv · Oct 23, 2026
EPOCH: Reliable Discovery through Evidence-Governed Search
arXiv · Oct 7, 2026
Jiuge-Tuiqiao: An Interpretable Human-AI System for Classical Chinese Poetry Refinement
arXiv · Aug 25, 2026