aiqbench analysis
This study investigates the limits of Large Language Model agents in prediction markets, revealing that their ability to aggregate dispersed private information degrades significantly as task complexity increases. The research highlights a critical parallel between AI and human cognition, showing that strategic prompting or feedback loops fail to overcome these reasoning bottlenecks regarding others' knowledge states.
Information Aggregation with AI Agents
arXiv:2604.20050v3 Announce Type: replace-cross Abstract: Can Large Language Models (AI agents) aggregate dispersed private information through trading and reason about the knowledge of others by observing price movements? We conduct a controlled experiment where AI agents trade in a prediction market after receiving private signals, measuring information aggregation by the log error of the last price. We find that although the median market is effective at aggregating information in the easy information structures, performance deteriorates in the harder structures, suggesting that AI agents may suffer from similar limitations as humans when reasoning about others. Consistent with our theoretical predictions, market accuracy does not improve from allowing cheap talk communication, changing the duration of the market, or strategic prompting; initial price has little average effect but matters in the very hard structure. We also find that "smarter" AI agents perform better at aggregation and are more profitable. Surprisingly, giving them feedback about past performance does not improve aggregation.
Abstract:Can Large Language Models (AI agents) aggregate dispersed private information through trading and reason about the knowledge of others by observing price movements? We conduct a controlled experiment where AI agents trade in a prediction market after receiving private signals, measuring information aggregation by the log error of the last price. We find that although the median market is effective at aggregating information in the easy information structures, performance deteriorates in the harder structures, suggesting that AI agents may suffer from similar limitations as humans when reasoning about others. Consistent with our theoretical predictions, market accuracy does not improve from allowing cheap talk communication, changing the duration of the market, or strategic prompting; initial price has little average effect but matters in the very hard structure. We also find that "smarter" AI agents perform better at aggregation and are more profitable. Surprisingly, giving them feedback about past performance does not improve aggregation.
General Economics (econ.GN); Artificial Intelligence (cs.AI); Computer Science and Game Theory (cs.GT)
arXiv:2604.20050 [econ.GN]
(or arXiv:2604.20050v3 [econ.GN] for this version)
Full text under the publisher's open licence · 172 words
Related coverage

Would We See It Coming? Preference Falsification Cascades in Multi-Agent Systems
LessWrong · Aug 4, 2026

Always ask your agent to be honest
LessWrong · Sep 16, 2026
DI Series: Serving Kimi-K3 MXFP4 Wide-EP Disaggregated on AMD Instinct MI300X and MI325X
AMD · Oct 8, 2026
Thread Trace Part 1: ROCprof Compute Viewer
AMD · Sep 15, 2026