aiqbench analysis
This paper gives formal conditions under which Q-learning pricing agents stay locked into supracompetitive prices once exploration stops. It also shows that the final outcome depends on the exploration path: agents can settle on collusive or competitive prices from the same starting point. For AI readers, the useful point is that learning-based pricing can sustain collusion without explicit coordination, and the result depends on exploration settings, not only on the algorithm's design.
Learning to Charge More: How Q-Learning Firms Lock In Supracompetitive Prices
arXiv:2505.22909v2 Announce Type: replace-cross Abstract: There is growing experimental evidence that $Q$-learning agents can learn to charge supracompetitive prices. We ask when such behavior persists, whether it emerges, and which strategies support it. In a repeated pricing game in which firms condition on the previous period's price profile, we give conditions on the learned action values under which supracompetitive prices become an absorbing state once experimentation ends. We then show that under softmax exploration these conditions are reached with positive and explicitly bounded probability, both from a random and from an all-zero initial $Q$-table. In the latter case firms lock in supracompetitive prices because they never learn that undercutting pays; the same mechanism can lock in the competitive price, so the long-run outcome is selected by the exploration path. Finally, we give sufficient conditions on the learned table under which the induced strategy is naive collusion, a grim trigger strategy or an increasing-price strategy; grim trigger strategies are subgame perfect under a standard discounting condition, whereas naive collusion is generally not an equilibrium.
Abstract:There is growing experimental evidence that $Q$-learning agents can learn to charge supracompetitive prices. We ask when such behavior persists, whether it emerges, and which strategies support it. In a repeated pricing game in which firms condition on the previous period's price profile, we give conditions on the learned action values under which supracompetitive prices become an absorbing state once experimentation ends. We then show that under softmax exploration these conditions are reached with positive and explicitly bounded probability, both from a random and from an all-zero initial $Q$-table. In the latter case firms lock in supracompetitive prices because they never learn that undercutting pays; the same mechanism can lock in the competitive price, so the long-run outcome is selected by the exploration path. Finally, we give sufficient conditions on the learned table under which the induced strategy is naive collusion, a grim trigger strategy or an increasing-price strategy; grim trigger strategies are subgame perfect under a standard discounting condition, whereas naive collusion is generally not an equilibrium.
General Economics (econ.GN); Artificial Intelligence (cs.AI); Computer Science and Game Theory (cs.GT)
arXiv:2505.22909 [econ.GN]
(or arXiv:2505.22909v2 [econ.GN] for this version)
https://doi.org/10.48550/arXiv.2505.22909 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 269 words