Neural Operators Which Certifiably Simultaneously Solve Infinitely Many Convex Programs using Logarithmic Rank and Linear Complexity

Abstract:Neural operators (NOs) are a class of deep learning models designed to simultaneously solve infinitely many related problems by casting them into an infinite-dimensional space, whereon these NOs operate. Such operator learning problems arise naturally in mathematical finance, for instance, when learning families of optimal portfolios or hedging strategies. Two significant gaps, however, remain between theory and practice: 1) most universal approximation guarantees do not describe practically implementable NOs or are pure existence statements; and 2) worst-case parameter bounds from universal approximation theorems suggest that constructed NOs may require an unrealistically large number of parameters to solve most operator learning problems, which stands in direct opposition to a slew of experimental evidence. This paper closes these gaps by exhibiting a short and simple training algorithm to construct a specific class of NOs, proximal equilibrium operators (PEOs), using (realistic) finite-dimensional deep equilibrium layers to solve families of convex optimization problems over a separable Hilbert space $X$. Here, the inputs are smooth, convex loss functions on $X$, and the outputs are the associated (approximate) solutions to the optimization problems defined by each input loss. Our main result is a certificate showing that, when the input losses lie in a suitable infinite-dimensional set, our PEOs trained with our algorithm do in fact uniformly approximate the corresponding solutions to arbitrary precision, with rank and width growing only logarithmically and the depth growing linearly in the reciprocal of the approximation error. We then validate both our theoretical results and the trainability of PEOs on three applications: non-linear PDEs, stochastic optimal control, and portfolio optimization and hedging problems in mathematical finance.

Machine Learning (cs.LG); Numerical Analysis (math.NA); Optimization and Control (math.OC); Computational Finance (q-fin.CP)

arXiv:2508.14995 [cs.LG]

(or arXiv:2508.14995v2 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2508.14995 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.