analiza aiqbench
This research investigates how reinforcement learning shapes the market behaviors of LLM agents, specifically revealing the conditions under which they can effectively enforce collusion versus merely mimicking it. By introducing a novel audit method, the study distinguishes between agents that learn to punish cheaters and those that rely on static rules, offering critical insights into the stability of AI-driven economic systems.
Is Anyone Enforcing the Cartel? What Reinforcement Learning Teaches LLM Agents in Markets
De la arXiv
Text integralarXiv:2610.28787v1 Announce Type: cross Abstract: Evaluations of large language model (LLM) agents report that pricing agents collude at others' expense. Yet collusion is defined by enforcement: a cartel, in which competitors share monopoly profits, holds only if a member who cheats is punished. We introduce an audit that tests enforcement by forcing a member to cheat in counterfactual copies of a market, and use it to study what reinforcement learning (RL) teaches agents in pricing and trading.
Small LLMs trained by RL learn what their training rivals reward, but only what they readily try. In self-play, they learn fixed rules ignoring their rivals; against a punishing rival, to be deterred; against a cheating rival, to match its cuts, never the costly price war that would deter it, which they rarely try. A war forced into training is learned, but mostly only where it was forced. Against their own copies, deterred and matching agents can earn the same cartel profits, but only matching ones can enforce them, and then thinly: as a by-product and only against large cuts.
Q-learners enforce collusion only in pricing. Prompted LLMs reach cartel-like outcomes mostly through rules that ignore the market. Outcome measures cannot tell these cases apart; the audit can.
Title:Is Anyone Enforcing the Cartel? What Reinforcement Learning Teaches LLM Agents in Markets
Abstract:Evaluations of large language model (LLM) agents report that pricing agents collude at others' expense. Yet collusion is defined by enforcement: a cartel, in which competitors share monopoly profits, holds only if a member who cheats is punished. We introduce an audit that tests enforcement by forcing a member to cheat in counterfactual copies of a market, and use it to study what reinforcement learning (RL) teaches agents in pricing and trading.
Small LLMs trained by RL learn what their training rivals reward, but only what they readily try. In self-play, they learn fixed rules ignoring their rivals; against a punishing rival, to be deterred; against a cheating rival, to match its cuts, never the costly price war that would deter it, which they rarely try. A war forced into training is learned, but mostly only where it was forced. Against their own copies, deterred and matching agents can earn the same cartel profits, but only matching ones can enforce them, and then thinly: as a by-product and only against large cuts.
Q-learners enforce collusion only in pricing. Prompted LLMs reach cartel-like outcomes mostly through rules that ignore the market. Outcome measures cannot tell these cases apart; the audit can.
Computation and Language (cs.CL); Computer Science and Game Theory (cs.GT); Machine Learning (cs.LG)
arXiv:2610.28787 [cs.CL]
(or arXiv:2610.28787v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.28787 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
deviation_tests.csv
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 318 cuvinte
Subiecte conexe
Adaptive GR(1) Specification Repair for Liveness-Preserving Shielding in Reinforcement Learning
arXiv · 27 aug. 2026
Structural Dual Adaptation for Safe Reinforcement Learning
arXiv · 3 nov. 2026
On the Clock: Towards Punctual and Productive Time-Budgeted AI Agents
arXiv · 10 oct. 2026
KnitTrans: Coordinated Trajectory Transfer for Effective and Efficient LLM Reinforcement Learning
arXiv · 27 oct. 2026