aiqbench analysis
This research investigates the reliability of Large Language Models in making subjective consumer decisions, such as travel bookings, by estimating their implied willingness to pay against human benchmarks. The study reveals that while larger models can derive meaningful valuations, they systematically overestimate costs unless conditioned on specific user preferences or persona constraints. These findings are critical for developers deploying LLMs in economic decision-support roles where accurate value inference is essential.
Would a Large Language Model Pay Extra for a View? Inferring Willingness to Pay from Subjective Choices
arXiv:2602.09802v2 Announce Type: replace Abstract: As Large Language Models (LLMs) are increasingly deployed in applications such as travel assistance and purchasing support, they are often required to make subjective choices on behalf of users in settings where no objectively correct answer exists. We study LLM decision-making in a travel-assistant context by presenting models with choice dilemmas and analyzing their responses using multinomial logit models to derive implied willingness to pay (WTP) estimates. These WTP values are subsequently compared to human benchmark values from the economics literature. In addition to a baseline setting, we examine how model behavior changes under more realistic conditions, including the provision of information about users' past choices and persona-based prompting. Our results show that while meaningful WTP values can be derived for larger LLMs, they also display systematic deviations at the attribute level. Additionally, they tend to overestimate human WTP overall, particularly when expensive options or business-oriented personas are introduced. Conditioning models on prior preferences for cheaper options yields valuations that are closer to human benchmarks. Overall, our findings highlight both the potential and the limitations of using LLMs for subjective decision support and underscore the importance of careful model selection, prompt design, and user representation when deploying such systems in practice.
Title:Would a Large Language Model Pay Extra for a View? Inferring Willingness to Pay from Subjective Choices
Abstract:As Large Language Models (LLMs) are increasingly deployed in applications such as travel assistance and purchasing support, they are often required to make subjective choices on behalf of users in settings where no objectively correct answer exists. We study LLM decision-making in a travel-assistant context by presenting models with choice dilemmas and analyzing their responses using multinomial logit models to derive implied willingness to pay (WTP) estimates. These WTP values are subsequently compared to human benchmark values from the economics literature. In addition to a baseline setting, we examine how model behavior changes under more realistic conditions, including the provision of information about users' past choices and persona-based prompting. Our results show that while meaningful WTP values can be derived for larger LLMs, they also display systematic deviations at the attribute level. Additionally, they tend to overestimate human WTP overall, particularly when expensive options or business-oriented personas are introduced. Conditioning models on prior preferences for cheaper options yields valuations that are closer to human benchmarks. Overall, our findings highlight both the potential and the limitations of using LLMs for subjective decision support and underscore the importance of careful model selection, prompt design, and user representation when deploying such systems in practice.
Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2602.09802 [cs.AI]
(or arXiv:2602.09802v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2602.09802 Focus to learn more arXiv-issued DOI via DataCite
https://doi.org/10.1016/j.eswa.2026.133279 Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 324 words
Related coverage
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · Oct 8, 2026

Deconfusing Corrigibility
AI Alignment Forum · Oct 17, 2026

[AINews] Opus 5.5 is good at explainer videos
Latent Space · Sep 29, 2026
Humans Are Not Chinchillas: Revisiting “How Quick and Big Would A Software Intelligence Explosion Be?”
LessWrong · Oct 5, 2026