aiqbench analysis
This research introduces Distributionally Robust Multi-Objective Optimization (DR-MOO), enhancing convergence rates from $\mathcal{O}(\epsilon^{-8})$ to $\mathcal{O}(\epsilon^{-4})$ through a novel single-loop algorithm that eliminates double sampling. By leveraging gradient clipping and Lagrangian dual reformulation, the method provides provable guarantees for handling distributional shifts in nonconvex problems without relying on uniformly bounded gradients. These advances offer significant theoretical efficiency gains for machine learning systems requiring robust decision-making under multiple conflicting criteria.
From Dual Tracking to Clipping: Provably Faster Distributionally Robust Multi-Objective Optimization
arXiv:2605.05660v2 Announce Type: replace Abstract: Multi-objective optimization (MOO) has received growing attention in applications that require learning under multiple criteria. However, most existing MOO formulations do not explicitly account for distributional shifts in the data. We introduce distributionally robust multi-objective optimization (DR-MOO), which minimizes multiple objectives under their respective worst-case distributions. We propose Pareto-type solution concepts for DR-MOO and develop multi-gradient descent algorithms (MGDA) with provable guarantees. Leveraging a Lagrangian dual reformulation, we first design a double-loop MGDA that uses an inner loop to estimate dual variables and achieves a total sample complexity $\mathcal{O}(\epsilon^{-8})$ for reaching an $\epsilon$-Pareto-stationary point. To further improve convergence, we combine large-batch sampling with gradient clipping to accommodate generalized smoothness and control bias in stochastic preference updates, eliminating the need for double sampling. This yields a single-loop double-clip MGDA with substantially improved sample complexity $\mathcal{O}(\epsilon^{-4})$. Our theory applies to nonconvex problems without requiring uniformly bounded gradients of the dual objectives. Experiments demonstrate that our methods are competitive with state-of-the-art MGDA baselines.
Abstract:Multi-objective optimization (MOO) has received growing attention in applications that require learning under multiple criteria. However, most existing MOO formulations do not explicitly account for distributional shifts in the data. We introduce distributionally robust multi-objective optimization (DR-MOO), which minimizes multiple objectives under their respective worst-case distributions. We propose Pareto-type solution concepts for DR-MOO and develop multi-gradient descent algorithms (MGDA) with provable guarantees. Leveraging a Lagrangian dual reformulation, we first design a double-loop MGDA that uses an inner loop to estimate dual variables and achieves a total sample complexity $\mathcal{O}(\epsilon^{-8})$ for reaching an $\epsilon$-Pareto-stationary point. To further improve convergence, we combine large-batch sampling with gradient clipping to accommodate generalized smoothness and control bias in stochastic preference updates, eliminating the need for double sampling. This yields a single-loop double-clip MGDA with substantially improved sample complexity $\mathcal{O}(\epsilon^{-4})$. Our theory applies to nonconvex problems without requiring uniformly bounded gradients of the dual objectives. Experiments demonstrate that our methods are competitive with state-of-the-art MGDA baselines.
Machine Learning (cs.LG); Optimization and Control (math.OC)
arXiv:2605.05660 [cs.LG]
(or arXiv:2605.05660v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2605.05660 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 256 words
Related coverage
State of the Field: AI for Epistemics and Coordination
LessWrong · Oct 2, 2026
Lattice-Based Cryptography Explained
Quantum Zeitgeist · Aug 15, 2026
News
Quantum Computing Report · Jul 25, 2026
FlashMask-3: Efficient and Expressive Mask-Aware Distributed Attention
arXiv · Oct 14, 2026