analiza aiqbench
This research introduces a novel cognitive-ecological framework to explain how Large Language Models develop specific bias patterns, termed "metacognitive myopia," due to skewed training data. The authors propose technical mechanisms like hidden parallel reasoning histories that could enable models to monitor and control their own inference risks before responding. This work is significant for advancing AI safety by offering a theoretical basis for mitigating harmful stereotypes in agentic systems used within high-stakes organizational environments.
Metacognitive Myopia in Large Language Models
arXiv:2408.05568v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) exhibit potentially harmful biases that reinforce culturally embedded stereotypes, influence moral judgments, or amplify positive evaluations of majority groups. We propose metacognitive myopia as a cognitive-ecological framework accounting for a conglomerate of established and emerging LLM biases. Our theoretical framework posits that biased samples in the information environment cause five symptoms of metacognitive myopia in LLMs: integration of invalid embeddings, susceptibility to redundant information, neglect of base rates in conditional computation, decision rules based on frequency, and inappropriate higher-order statistical inference for nested data structures. Moreover, it posits that the two main components of metacognition, monitoring and control, could account for these five symptoms. Accordingly, we further outline how monitoring and control could be approximated technically, for instance, through hidden parallel reasoning histories that allow interactive LLMs to evaluate risks of myopic inference before generating overt responses. Our theoretical framework provides a novel perspective on flawed human-machine interactions and agentic AI and raises significant ethical concerns regarding the implementation of LLMs in organizational structures and high-stakes decisions.
Abstract:Large Language Models (LLMs) exhibit potentially harmful biases that reinforce culturally embedded stereotypes, influence moral judgments, or amplify positive evaluations of majority groups. We propose metacognitive myopia as a cognitive-ecological framework accounting for a conglomerate of established and emerging LLM biases. Our theoretical framework posits that biased samples in the information environment cause five symptoms of metacognitive myopia in LLMs: integration of invalid embeddings, susceptibility to redundant information, neglect of base rates in conditional computation, decision rules based on frequency, and inappropriate higher-order statistical inference for nested data structures. Moreover, it posits that the two main components of metacognition, monitoring and control, could account for these five symptoms. Accordingly, we further outline how monitoring and control could be approximated technically, for instance, through hidden parallel reasoning histories that allow interactive LLMs to evaluate risks of myopic inference before generating overt responses. Our theoretical framework provides a novel perspective on flawed human-machine interactions and agentic AI and raises significant ethical concerns regarding the implementation of LLMs in organizational structures and high-stakes decisions.
Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computers and Society (cs.CY); Applications (stat.AP)
arXiv:2408.05568 [cs.AI]
(or arXiv:2408.05568v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2408.05568 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 273 cuvinte
Subiecte conexe
Cognitive Enhancement: Rethinking the Necessity of Role-Playing for Large Language Models
arXiv · 1 oct. 2026
Human Cognition in Machines: A Unified Perspective of World Models
arXiv · 16 iun. 2026
Test-Time Adaptation of Reasoning Strategies with Bayesian Nonparametric Memory
arXiv · 6 oct. 2026
LLMs learn different forms of metacognition when trained to predict their own accuracy
arXiv · 29 sept. 2026