Sari la conținut

Română · beta. Traducerea a fost făcută automat și este încă în curs de revizuire. Citește pagina în engleză

perspectiva aiqbenchCercetare AI

analiza aiqbench

This research investigates how reinforcement learning shapes the market behaviors of LLM agents, specifically revealing the conditions under which they can effectively enforce collusion versus merely mimicking it. By introducing a novel audit method, the study distinguishes between agents that learn to punish cheaters and those that rely on static rules, offering critical insights into the stability of AI-driven economic systems.
Sursă care acoperăarXiv

Is Anyone Enforcing the Cartel? What Reinforcement Learning Teaches LLM Agents in Markets

arXiv3 min de cititArticolul și perspectiva sunt în engleză.

De la arXiv

Text integral

arXiv:2610.28787v1 Announce Type: cross Abstract: Evaluations of large language model (LLM) agents report that pricing agents collude at others' expense. Yet collusion is defined by enforcement: a cartel, in which competitors share monopoly profits, holds only if a member who cheats is punished. We introduce an audit that tests enforcement by forcing a member to cheat in counterfactual copies of a market, and use it to study what reinforcement learning (RL) teaches agents in pricing and trading.

Small LLMs trained by RL learn what their training rivals reward, but only what they readily try. In self-play, they learn fixed rules ignoring their rivals; against a punishing rival, to be deterred; against a cheating rival, to match its cuts, never the costly price war that would deter it, which they rarely try. A war forced into training is learned, but mostly only where it was forced. Against their own copies, deterred and matching agents can earn the same cartel profits, but only matching ones can enforce them, and then thinly: as a by-product and only against large cuts.

Q-learners enforce collusion only in pricing. Prompted LLMs reach cartel-like outcomes mostly through rules that ignore the market. Outcome measures cannot tell these cases apart; the audit can.

Title:Is Anyone Enforcing the Cartel? What Reinforcement Learning Teaches LLM Agents in Markets

Abstract:Evaluations of large language model (LLM) agents report that pricing agents collude at others' expense. Yet collusion is defined by enforcement: a cartel, in which competitors share monopoly profits, holds only if a member who cheats is punished. We introduce an audit that tests enforcement by forcing a member to cheat in counterfactual copies of a market, and use it to study what reinforcement learning (RL) teaches agents in pricing and trading.

Small LLMs trained by RL learn what their training rivals reward, but only what they readily try. In self-play, they learn fixed rules ignoring their rivals; against a punishing rival, to be deterred; against a cheating rival, to match its cuts, never the costly price war that would deter it, which they rarely try. A war forced into training is learned, but mostly only where it was forced. Against their own copies, deterred and matching agents can earn the same cartel profits, but only matching ones can enforce them, and then thinly: as a by-product and only against large cuts.

Q-learners enforce collusion only in pricing. Prompted LLMs reach cartel-like outcomes mostly through rules that ignore the market. Outcome measures cannot tell these cases apart; the audit can.

Computation and Language (cs.CL); Computer Science and Game Theory (cs.GT); Machine Learning (cs.LG)

arXiv:2610.28787 [cs.CL]

(or arXiv:2610.28787v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2610.28787 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

deviation_tests.csv

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Text integral sub licența deschisă a publicației · 318 cuvinte

Subiecte conexe