Learning Controlled Stochastic Differential Equations

Abstract:We study the problem of learning controlled stochastic differential equations (SDEs) \[ dX_t = b(t,X_t,u_t)\,dt + \sigma(t,X_t,u_t)\,dW_t, \] whose drift and diffusion depend nonlinearly on time, state, and control values. From trajectory data, we aim to estimate coefficients whose induced density flows reproduce those of the observed dynamics. The data consist of several controls sampled from a finite-dimensional family and, for each control, multiple independent trajectories observed at discrete times over a fixed horizon. The controls are observed inputs, not learner-selected decisions. We propose a kernel method for multidimensional nonlinear controlled SDEs. The method estimates the density flow for each sampled control, then fits the drift $b$ and diffusion matrix $a=\sigma\sigma^\top$ by matching the estimated flows through the Fokker-Planck equation. Under Sobolev regularity assumptions, we prove finite-sample bounds on the error between the density flows of the learned and true SDEs, measured in $L^2$ over controls, time, and state. The bounds quantify how the error decreases with the number $K$ of sampled controls, with rates determined by the state and control-parametrization dimensions and Sobolev regularity. We further derive uniform-in-control guarantees and CVaR-type bounds for tail-sensitive quantities. Numerical experiments illustrate the method, and an open-source Python implementation is provided.

Machine Learning (stat.ML); Machine Learning (cs.LG)

arXiv:2411.01982 [stat.ML]

(or arXiv:2411.01982v2 [stat.ML] for this version)

https://doi.org/10.48550/arXiv.2411.01982 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.