Delphos: A reinforcement learning framework for assisting discrete choice model specification

Abstract:We introduce Delphos, a deep reinforcement learning framework for assisting discrete choice model specification process. Delphos aims to support the modeller by providing automated, data-driven suggestions for model specifications, thereby reducing the effort required to develop and refine utility functions. Delphos conceptualises model specification as a sequential decision-making problem, inspired by the way human choice modellers iteratively construct models through a series of reasoned specification decisions. In this setting, an agent learns to specify candidate model specifications by choosing a sequence of modelling actions, such as adding alternative specific constants, accommodating both generic and alternative-specific taste parameters, applying non-linear transformations to attributes, and including interactions with covariates. Each resulting candidate model is estimated and evaluated using a reward function defined by the modeller, which can reflect statistical model fit as well as behavioural expectations. Specifically, Delphos uses a Deep Q-Network to learn how individual specification decisions contribute to the eventual quality of the resulting model and, in turn, which sequences of modelling decisions tend to produce well-performing candidates. We evaluate Delphos on both simulated and empirical datasets using alternative reward functions. In simulated cases, learning curves, Q-value patterns, and performance metrics show that Delphos learns effective specification strategies while exploring only a small fraction of the feasible modelling space. We further apply the framework to two empirical datasets to benchmark and demonstrate its practical use. These experiments illustrate the ability of Delphos to generate competitive, behaviourally plausible models and highlight the potential of this adaptive, learning-based framework to assist the model specification process.

General Economics (econ.GN); Machine Learning (cs.LG)

arXiv:2506.06410 [econ.GN]

(or arXiv:2506.06410v4 [econ.GN] for this version)

https://doi.org/10.48550/arXiv.2506.06410 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.