Stochastic Gradient Descent for Operator Learning in Hilbert Spaces: Convergence Rates and Minimax Lower Bounds

Abstract:This study investigates the use of stochastic gradient descent (SGD) to learn operators between general Hilbert spaces. We study weak and strong regularity conditions for the target operator that characterize its structure and complexity. Under these conditions, we establish upper bounds for convergence rates of the SGD algorithm and derive a minimax lower bound analysis, further illustrating that our convergence analysis and regularity conditions quantitatively characterize the statistical difficulty of operator estimation under these regularity conditions. The analysis extends to nonlinear regression targets under model misspecification, in which case SGD converges to the best linear approximation. Moreover, applying our analysis to operator learning problems based on vector-valued and scalar-valued reproducing kernel Hilbert spaces yields new convergence results, thereby refining the conclusions of existing literature.

Machine Learning (stat.ML); Machine Learning (cs.LG); Functional Analysis (math.FA); Statistics Theory (math.ST)

arXiv:2402.04691 [stat.ML]

(or arXiv:2402.04691v5 [stat.ML] for this version)

https://doi.org/10.48550/arXiv.2402.04691 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.