aiqbench analysis
This research advances the theoretical foundations of machine learning by establishing near-optimal convergence rates for regularized stochastic gradient descent in infinite-dimensional, operator-valued settings. The work provides crucial high-probability guarantees for solving ill-posed inverse problems and parametric elliptic PDEs without explicit dependence on output dimensionality. By bridging functional analysis with practical optimization algorithms, it offers a robust framework for structured prediction tasks where traditional finite-dimensional methods fall short.
Learning Operators by Regularized Stochastic Gradient Descent with Operator-valued Kernels
arXiv:2504.18184v5 Announce Type: replace-cross Abstract: We consider a class of statistical inverse problems involving the estimation of a regression operator from a Polish space to a separable Hilbert space, where the target lies in a vector-valued reproducing kernel Hilbert space induced by an operator-valued kernel. To address the associated ill-posedness, we analyze regularized stochastic gradient descent (SGD) algorithms in both online and finite-horizon settings. The former uses polynomially decaying step sizes and regularization parameters, while the latter adopts fixed values. Under suitable structural and distributional assumptions, we establish prediction and estimation error bounds with no explicit dependence on the dimension of the output space. The resulting convergence rates are near-optimal in expectation, and we also derive high-probability estimates that imply almost sure convergence. Our analysis introduces a general technique for obtaining high-probability guarantees in infinite-dimensional settings. We illustrate the scope of our framework through applications to structured prediction and a class of parametric elliptic PDEs. For the latter, we construct kernels for arcsine and uniform sampling, verify the assumptions of our high-probability results, and obtain bounds uniform over finite parameter truncations under suitable conditions on coefficient decay.
Abstract:We consider a class of statistical inverse problems involving the estimation of a regression operator from a Polish space to a separable Hilbert space, where the target lies in a vector-valued reproducing kernel Hilbert space induced by an operator-valued kernel. To address the associated ill-posedness, we analyze regularized stochastic gradient descent (SGD) algorithms in both online and finite-horizon settings. The former uses polynomially decaying step sizes and regularization parameters, while the latter adopts fixed values. Under suitable structural and distributional assumptions, we establish prediction and estimation error bounds with no explicit dependence on the dimension of the output space. The resulting convergence rates are near-optimal in expectation, and we also derive high-probability estimates that imply almost sure convergence. Our analysis introduces a general technique for obtaining high-probability guarantees in infinite-dimensional settings. We illustrate the scope of our framework through applications to structured prediction and a class of parametric elliptic PDEs. For the latter, we construct kernels for arcsine and uniform sampling, verify the assumptions of our high-probability results, and obtain bounds uniform over finite parameter truncations under suitable conditions on coefficient decay.
Machine Learning (stat.ML); Machine Learning (cs.LG); Functional Analysis (math.FA); Statistics Theory (math.ST)
arXiv:2504.18184 [stat.ML]
(or arXiv:2504.18184v5 [stat.ML] for this version)
https://doi.org/10.48550/arXiv.2504.18184 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 283 words
Related coverage
Stochastic Gradient Descent for Operator Learning in Hilbert Spaces: Convergence Rates and Minimax Lower Bounds
arXiv · Sep 15, 2026
Resolution-Independent Analysis of Encoder--Decoder Operator Learning via Limiting Kernels
arXiv · Sep 15, 2026
Sparse Data Augmentation for Optimization with Provable Guarantees
arXiv · Sep 9, 2026
Exploding variance of means of exponentials: least-squares to the rescue – Machine Learning Research Blog
francisbach.com · Sep 27, 2026