aiqbench analysis
This study evaluates how existing design guidelines for trustworthy AI translate to the specific challenges of LLM-based companions focused on emotional connection. The research identifies nine key principle areas where ethical considerations and user experience requirements are deeply intertwined, revealing that standard recommendations cannot be directly applied without context-sensitive adaptation. These findings provide a critical framework for developers navigating the complex design tensions inherent in building relational artificial intelligence systems.
Assessing the Applicability of Existing Design Recommendations to AI Companion Design: A Multi-Method Study
arXiv:2609.14236v1 Announce Type: cross Abstract: With the rapid proliferation of large language model (LLM)-based systems, AI companions have emerged as conversational agents designed to cultivate emotional connection rather than primarily to support humans in instrumental tasks. Because engagement with AI companions involves relational, emotional, and potentially long-term interactions, their design is consequential. Prior work has offered guidance for designing trustworthy and relational AI systems and has begun to examine design for AI companionship. However, while such work provides insights into possible design solutions, less is known about what makes AI companion design difficult as a design problem. To examine this challenge, we assessed the applicability of existing design recommendations from adjacent domains in the context of AI companion design. Our multi-method investigation unfolded across four phases: literature review, practitioner co-analysis, internal heuristic evaluation, and external expert assessment. Throughout this process, we synthesized nine design principle areas that surfaced tensions in the applicability of existing recommendations to AI companion design. Our findings show that ethical and UX-oriented considerations are deeply intertwined and often require context-sensitive application. We document a systematic, multi-method problem analysis that uses these principle areas as an analytic artifact to examine why existing recommendations cannot be directly transferred to AI companion contexts.
Abstract:With the rapid proliferation of large language model (LLM)-based systems, AI companions have emerged as conversational agents designed to cultivate emotional connection rather than primarily to support humans in instrumental tasks. Because engagement with AI companions involves relational, emotional, and potentially long-term interactions, their design is consequential. Prior work has offered guidance for designing trustworthy and relational AI systems and has begun to examine design for AI companionship. However, while such work provides insights into possible design solutions, less is known about what makes AI companion design difficult as a design problem. To examine this challenge, we assessed the applicability of existing design recommendations from adjacent domains in the context of AI companion design. Our multi-method investigation unfolded across four phases: literature review, practitioner co-analysis, internal heuristic evaluation, and external expert assessment. Throughout this process, we synthesized nine design principle areas that surfaced tensions in the applicability of existing recommendations to AI companion design. Our findings show that ethical and UX-oriented considerations are deeply intertwined and often require context-sensitive application. We document a systematic, multi-method problem analysis that uses these principle areas as an analytic artifact to examine why existing recommendations cannot be directly transferred to AI companion contexts.
Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI)
arXiv:2609.14236 [cs.HC]
(or arXiv:2609.14236v1 [cs.HC] for this version)
https://doi.org/10.48550/arXiv.2609.14236 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
https://doi.org/10.1016/j.ijhcs.2026.103940 Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 306 words
Related coverage
State of the Field: AI for Epistemics and Coordination
LessWrong · Oct 2, 2026

Vertical Advantage: Transforming Industries with Lakebase and Agentic AI
Databricks · Aug 27, 2026

Top 10 Best Identity Threat Detection and Response (ITDR) Solutions in 2026
Cybersecurity News · Jul 17, 2026

10 Ways AI Helps You Build What Your Business Needs
AppDirect · May 15, 2026