Do LLMs Choose Like Humans? Using Cognitive Theory to Evaluate LLM Decision-Making

Title:Do LLMs Choose Like Humans? Using Cognitive Theory to Evaluate LLM Decision-Making

Abstract:Large language models (LLMs) exhibit a range of human-like decision-making behaviors, but whether these reflect similar underlying mechanisms or surface-level mimicry remains unclear. We evaluate whether LLM context sensitivity aligns with a cognitive economic theory that explains human behavior through problem categorization and attention allocation. Across 12 open-source and commercial LLMs on a novel 140,000-trial product choice benchmark, context induces human-like shifts in choice and problem categorization, but does not reliably reweight attention between features like price and quality. Neither scale nor chain-of-thought reasoning reliably attenuates context sensitivity or generates human-like behavior. These results suggest that LLM decision mechanisms are distinct from human ones.

Computation and Language (cs.CL); Machine Learning (cs.LG)

arXiv:2609.22225 [cs.CL]

(or arXiv:2609.22225v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2609.22225 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.