aiqbench analysis
This paper introduces the Stochastic Penalty-Barrier Method (SPBM), a novel optimization technique designed to improve constrained machine learning tasks such as fairness-aware training and physics-informed neural networks. By incorporating exponential averaging of dual variables and handling non-smoothness via the Moreau envelope, the method maintains computational efficiency comparable to standard regularized Adam even with thousands of constraints. The work provides theoretical analysis on mini-batching bias and empirical validation across multiple benchmarks, offering a practical advance for integrating domain knowledge into statistical models without prohibitive runtime costs.
Stochastic Penalty-Barrier Method for Constrained Machine Learning
arXiv:2605.18618v3 Announce Type: replace Abstract: Constrained Machine Learning (CML) enables fairness-aware training, physics-informed neural networks, and integration of symbolic domain knowledge into statistical models. In this work, we introduce the Stochastic Penalty-Barrier Method (SPBM) for CML problems. SPBM extends classical penalty and barrier methods by incorporating an exponential averaging of the dual variables, a stabilized penalty schedule, and the Moreau envelope to handle non-smoothness. We analyze the bias that mini-batching introduces in the barrier function and show that the feasible set of the resulting transformed problem is contained within the original one. We compare SPBM with CML baselines across multiple fairness and physics informed neural networks experiments. We find that SPBM is competitive with state-of-the-art methods. We also observe, on our fairness-based computational benchmark, that the per-epoch runtime of CML methods is largely independent of the number of constraints, and within $1.3\times$ of the per-epoch runtime of regularized Adam, for a number of constraints ranging from $90$ to $9900$.
Abstract:Constrained Machine Learning (CML) enables fairness-aware training, physics-informed neural networks, and integration of symbolic domain knowledge into statistical models. In this work, we introduce the Stochastic Penalty-Barrier Method (SPBM) for CML problems. SPBM extends classical penalty and barrier methods by incorporating an exponential averaging of the dual variables, a stabilized penalty schedule, and the Moreau envelope to handle non-smoothness. We analyze the bias that mini-batching introduces in the barrier function and show that the feasible set of the resulting transformed problem is contained within the original one. We compare SPBM with CML baselines across multiple fairness and physics informed neural networks experiments. We find that SPBM is competitive with state-of-the-art methods. We also observe, on our fairness-based computational benchmark, that the per-epoch runtime of CML methods is largely independent of the number of constraints, and within $1.3\times$ of the per-epoch runtime of regularized Adam, for a number of constraints ranging from $90$ to $9900$.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2605.18618 [cs.LG]
(or arXiv:2605.18618v3 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2605.18618 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 249 words
Related coverage
Learned Preconditioning for a Primal-Dual Interior-Point Method
arXiv · Sep 30, 2026
Kernel-based potential mean-field games with unbiased random Fourier $U$-statistics
arXiv · Sep 22, 2026
Transition Path Sampling Using Koopman Operators and Exit-Time Optimal Control
arXiv · Oct 8, 2026
IRENE: A Convolutional GRU Ensemble Model for Radar Precipitation Nowcasting over Italy
arXiv · Sep 16, 2026