From Individual Learning to Market Equilibrium: Correcting Structural and Parametric Biases in RL Simulations of Economic Models

arXiv:2507.18229v3 Announce Type: replace-cross Abstract: Applying Reinforcement Learning (RL) to economic modeling reveals a fundamental tension: standard RL agents do not behave as competitive price-takers. While macroeconomic models typically assume agents take aggregate conditions as given, a naive single-agent RL algorithm learns to internalize its impact on the environment, effectively exerting market power. We illustrate this divergence in a search-and-matching model with concave production, showing that a standard RL agent converges to a monopsonistic allocation rather than the competitive equilibrium. Furthermore, we show that RL's approach to time discounting distorts standard intertemporal economic trade-offs. To resolve these issues, we propose a Calibrated Mean-Field Reinforcement Learning framework that restricts the agent to optimize against an exogenous aggregate state to restore price-taking behavior, and recalibrates the RL reward structure to reflect true economic opportunity costs. By iteratively updating the aggregate state, our algorithm converges to a stationary fixed point that recovers the competitive equilibrium. This approach provides a method for embedding learning algorithms into dynamic general equilibrium models.

Abstract:Applying Reinforcement Learning (RL) to economic modeling reveals a fundamental tension: standard RL agents do not behave as competitive price-takers. While macroeconomic models typically assume agents take aggregate conditions as given, a naive single-agent RL algorithm learns to internalize its impact on the environment, effectively exerting market power. We illustrate this divergence in a search-and-matching model with concave production, showing that a standard RL agent converges to a monopsonistic allocation rather than the competitive equilibrium. Furthermore, we show that RL's approach to time discounting distorts standard intertemporal economic trade-offs. To resolve these issues, we propose a Calibrated Mean-Field Reinforcement Learning framework that restricts the agent to optimize against an exogenous aggregate state to restore price-taking behavior, and recalibrates the RL reward structure to reflect true economic opportunity costs. By iteratively updating the aggregate state, our algorithm converges to a stationary fixed point that recovers the competitive equilibrium. This approach provides a method for embedding learning algorithms into dynamic general equilibrium models.

General Economics (econ.GN); Artificial Intelligence (cs.AI)

arXiv:2507.18229 [econ.GN]

(or arXiv:2507.18229v3 [econ.GN] for this version)

https://doi.org/10.48550/arXiv.2507.18229 Focus to learn more arXiv-issued DOI via DataCite

https://doi.org/10.1007/s10614-026-11464-y Focus to learn more DOI(s) linking to related resources

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

aiqbench analysis

From Individual Learning to Market Equilibrium: Correcting Structural and Parametric Biases in RL Simulations of Economic Models

Related coverage