aiqbench analysis
MOSEL introduces a novel framework for multi-objective optimization in deep neural networks, achieving the recovery of full Pareto stationary fronts at the computational cost of standard single-objective training. By reformulating the problem as a Stackelberg game, this approach efficiently uncovers diverse and robust representations in conflicting settings like fairness-accuracy trade-offs while outperforming existing multi-task learning methods.
Learning Pareto Stationary Fronts via Single-Pass Backpropagation
arXiv:2610.06397v1 Announce Type: new Abstract: We propose MOSEL (Multi-Objective Stackelberg Efficient Learning), a framework for a posteriori multi-objective optimization (MOO) in deep neural networks that recovers a full front of Pareto stationary solutions at the computational cost of standard single-objective training. MOSEL reformulates the problem as a bilevel optimization problem that leverages network modularity to decouple representation learning from objective-preference alignment. Casting the bilevel optimization problem as a Stackelberg game enables solving the original a posteriori MOO problem in a single forward-backward pass. As a result, MOSEL matches the time and memory efficiency of standard single-objective training while enabling scalable Pareto stationary front learning. Empirically, MOSEL uncovers diverse and optimal Pareto frontiers in strongly conflicting settings (e.g., fairness-accuracy). Remarkably, even in weakly conflicting regimes such as multi-task learning, it consistently converges to solutions closer to the utopia point, outperforming both standard single-objective training and specialized multi-task learning methods. These results highlight the broader potential of a posteriori MOO learning as a pathway to efficiently learn more diverse and robust representations, ultimately improving generalization.
Abstract:We propose MOSEL (Multi-Objective Stackelberg Efficient Learning), a framework for a posteriori multi-objective optimization (MOO) in deep neural networks that recovers a full front of Pareto stationary solutions at the computational cost of standard single-objective training. MOSEL reformulates the problem as a bilevel optimization problem that leverages network modularity to decouple representation learning from objective-preference alignment. Casting the bilevel optimization problem as a Stackelberg game enables solving the original a posteriori MOO problem in a single forward-backward pass. As a result, MOSEL matches the time and memory efficiency of standard single-objective training while enabling scalable Pareto stationary front learning. Empirically, MOSEL uncovers diverse and optimal Pareto frontiers in strongly conflicting settings (e.g., fairness-accuracy). Remarkably, even in weakly conflicting regimes such as multi-task learning, it consistently converges to solutions closer to the utopia point, outperforming both standard single-objective training and specialized multi-task learning methods. These results highlight the broader potential of a posteriori MOO learning as a pathway to efficiently learn more diverse and robust representations, ultimately improving generalization.
Machine Learning (cs.LG)
arXiv:2610.06397 [cs.LG]
(or arXiv:2610.06397v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.06397 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 262 words
Related coverage
Dropless MoE Training in JAX with Primus-Turbo
AMD · Jun 10, 2026
SHARP: Sleep-based Hierarchical Accelerated Replay for Long Range Non-Stationary Temporal Pattern Recognition
arXiv · Jun 5, 2026
Jev: System One models for Prod, not God — with Diogo Almeida, CEO, TypeSafe AI
Latent Space · Sep 21, 2026
Learning Fair Pareto-Optimal Policies in Multi-Objective Reinforcement Learning
arXiv · Jun 17, 2026