analiza aiqbench
Broximal Gradient Descent introduces a theoretical optimization framework that eliminates the need for explicit projection steps in constrained machine learning problems. By minimizing a linear function over an intersection of constraint sets and local balls, this method offers a new approach to solving complex optimization regimes relevant to large-scale AI training. While currently focused on mathematical convergence theory, it represents a significant potential advancement for algorithms handling high-dimensional constrained data.
Broximal Gradient Descent: A Projection-Free Sister of Projected Gradient Descent
arXiv:2605.08850v2 Announce Type: replace-cross Abstract: We propose Broximal Gradient Descent (BroxGD), a projection-free sister method to projected gradient descent for constrained optimization. Its forward--backward construction replaces the proximal backward operation by the broximal operation of Gruntkowska et al. (2025). Instead of projecting, each step minimizes a linear function over the intersection of the constraint set $\mathcal{X}$ and a ball $\mathbb{B}(x_k,t_k)$ centered at the current iterate $x_k$, of suitable radius $t_k>0$: \[ x_{k+1}\in\arg\min_{z\in\mathcal{X}\cap\mathbb{B}(x_k,t_k)}\langle\nabla f(x_k),z\rangle. \] We develop a comprehensive convergence theory spanning a wide range of optimization regimes and radius rules. We expect BroxGD to find many applications and inspire numerous extensions, much like projected gradient descent. Our contribution is theoretical; potential applications and toy experiments illustrate the method and suggest directions for future work.
Abstract:We propose Broximal Gradient Descent (BroxGD), a projection-free sister method to projected gradient descent for constrained optimization. Its forward--backward construction replaces the proximal backward operation by the broximal operation of Gruntkowska et al. (2025). Instead of projecting, each step minimizes a linear function over the intersection of the constraint set $\mathcal{X}$ and a ball $\mathbb{B}(x_k,t_k)$ centered at the current iterate $x_k$, of suitable radius $t_k>0$: \[ x_{k+1}\in\arg\min_{z\in\mathcal{X}\cap\mathbb{B}(x_k,t_k)}\langle\nabla f(x_k),z\rangle. \] We develop a comprehensive convergence theory spanning a wide range of optimization regimes and radius rules. We expect BroxGD to find many applications and inspire numerous extensions, much like projected gradient descent. Our contribution is theoretical; potential applications and toy experiments illustrate the method and suggest directions for future work.
121 pages. Substantially expanded revision with a new title, additional convergence results, practical radius rules, applications, and numerical experiments
Optimization and Control (math.OC); Machine Learning (cs.LG); Machine Learning (stat.ML)
arXiv:2605.08850 [math.OC]
(or arXiv:2605.08850v2 [math.OC] for this version)
Text integral sub licența deschisă a publicației · 156 cuvinte
Subiecte conexe
Challenge: Hand coding weights for efficient sequence memorisation
AI Alignment Forum · 23 iul. 2026
A lift for input-convex neural net training
arXiv · 24 sept. 2026
Projected Riemannian Gradient Descent for the Bures-Wasserstein Barycenter: Dimension-Independent Linear Convergence at Unit Step Size
arXiv · 4 sept. 2026
The Implicit Bias of Steepest Descent with Mini-batch Stochastic Gradient
arXiv · 17 iun. 2026