Skip to content
aiqbench takeAI Safety

aiqbench analysis

This paper introduces a formal mental model framework designed to estimate and manipulate human trust in AI agents, addressing a critical requirement for safe human-AI interaction. By validating the model through human subject studies, it establishes a foundation for building decision-making systems that can dynamically adjust user reliance based on multidimensional trust perceptions. This work advances AI safety and alignment by providing a structured approach to managing the psychological dynamics between users and autonomous systems.
Sources coveringarXiv, AWS
What this isA model's reading of the article and related coverage, written for aiqbench. Not the publisher's words.

A Mental Model Based Framework of Trust

arXiv2 min read

arXiv:2301.12569v2 Announce Type: replace Abstract: Handling trust is a core requirement of effective interaction between people and AI agents. Thus, any decision-making framework designed to work with people must be able to estimate human trust. In this paper, we propose a mental model-based framework of trust that captures multidimensional aspects of trust and can be used to infer human trust. This framework can also be used as a foundation for any trust-aware decision-making framework. Through our mental model-based framework of trust, we propose a formal framework that captures intuitions about trust present in extant literature. We show how our framework captures various dimensions of trust perception and then use the framework to define trust evolution, human reliance, and decision-making. Additionally, we also propose a formalization of the appropriate level of trust in the agent. Using human subject studies, we evaluate if (1) changes in trust can be achieved by adjusting human beliefs about the agent according to predictions made by our mental model framework and (2) adjustments in human belief lead to corresponding modifications in trust perceptions; performance, process, and purpose, by controlling different aspects of the model.

Abstract:Handling trust is a core requirement of effective interaction between people and AI agents. Thus, any decision-making framework designed to work with people must be able to estimate human trust. In this paper, we propose a mental model-based framework of trust that captures multidimensional aspects of trust and can be used to infer human trust. This framework can also be used as a foundation for any trust-aware decision-making framework. Through our mental model-based framework of trust, we propose a formal framework that captures intuitions about trust present in extant literature. We show how our framework captures various dimensions of trust perception and then use the framework to define trust evolution, human reliance, and decision-making. Additionally, we also propose a formalization of the appropriate level of trust in the agent. Using human subject studies, we evaluate if (1) changes in trust can be achieved by adjusting human beliefs about the agent according to predictions made by our mental model framework and (2) adjustments in human belief lead to corresponding modifications in trust perceptions; performance, process, and purpose, by controlling different aspects of the model.

Artificial Intelligence (cs.AI)

arXiv:2301.12569 [cs.AI]

(or arXiv:2301.12569v2 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2301.12569 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Full text under the publisher's open licence · 275 words

Related coverage