aiqbench analysis
This research investigates the strategic dynamics between human advisors and personal AI, modeling how optimal recommendations must adapt to the probability of AI consultation and user trust levels. The study finds that advisors should often counteract predictable AI signals, a strategy whose effectiveness varies non-linearly with both consultation frequency and the degree of user reliance on the assistant. These insights are critical for understanding the evolving human-AI interaction loop in personal advisory contexts.
Strategic Advice in the Age of Personal AI
arXiv:2603.02055v2 Announce Type: replace Abstract: Personal AI assistants are changing how individuals use advice. We study how an advisor should design its recommendation in anticipation of stochastic consultation with personal AI whose recommendation is predictable. Personal AI enters through two dimensions: consultation probability and relative trust, which captures the relative influence personal AI receives when consulted. In the baseline model, the advisor optimally counteracts the personal AI signal. Counteraction increases with consultation probability but is hump-shaped in relative trust. The advisor's minimized loss is hump-shaped in consultation probability, vanishing when personal AI is never or always consulted. Greater relative trust in personal AI increases the irreducible loss arising from stochastic consultation. We extend the analysis to partial predictability and costly recommendation adjustment, characterizing their effects on optimal recommendations and minimized loss. The framework also accommodates richer information structures, including settings in which personal AI is perceived as having access to private information relevant to the task. We introduce an online forecasting experiment that examines how participants obtain personal AI advice and combine it with an advisor's recommendation and their initial judgments. Participants place weight on all three inputs. When access requires an additional action, some participants do not seek personal AI advice, while some others attempt to obtain it without success. Together, these findings highlight two distinct aspects of personal AI use: whether advice is obtained and how much weight it receives when available.
Abstract:Personal AI assistants are changing how individuals use advice. We study how an advisor should design its recommendation in anticipation of stochastic consultation with personal AI whose recommendation is predictable. Personal AI enters through two dimensions: consultation probability and relative trust, which captures the relative influence personal AI receives when consulted. In the baseline model, the advisor optimally counteracts the personal AI signal. Counteraction increases with consultation probability but is hump-shaped in relative trust. The advisor's minimized loss is hump-shaped in consultation probability, vanishing when personal AI is never or always consulted. Greater relative trust in personal AI increases the irreducible loss arising from stochastic consultation. We extend the analysis to partial predictability and costly recommendation adjustment, characterizing their effects on optimal recommendations and minimized loss. The framework also accommodates richer information structures, including settings in which personal AI is perceived as having access to private information relevant to the task. We introduce an online forecasting experiment that examines how participants obtain personal AI advice and combine it with an advisor's recommendation and their initial judgments. Participants place weight on all three inputs. When access requires an additional action, some participants do not seek personal AI advice, while some others attempt to obtain it without success. Together, these findings highlight two distinct aspects of personal AI use: whether advice is obtained and how much weight it receives when available.
Machine Learning (cs.LG); Computer Science and Game Theory (cs.GT); Human-Computer Interaction (cs.HC)
arXiv:2603.02055 [cs.LG]
(or arXiv:2603.02055v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2603.02055 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 330 words
Related coverage

Game-Theoretic Disempowerment: Loss of Control to Agencyless AI
LessWrong · Sep 27, 2026
State of the Field: AI for Epistemics and Coordination
LessWrong · Oct 2, 2026
Podcast with Cierra Lunde and Mike Kilroy from HKA Marketing Communications
Quantum Computing Report · Aug 1, 2026

Georgia Tech AI Safety Initiative Retrospective 2025-2026
LessWrong · Jul 24, 2026