analiza aiqbench
This paper introduces ORACLE, a novel framework for constrained learning that assesses constraint compatibility after the optimizer has already acted, rather than intervening earlier in the training loop. By evaluating steps within the optimizer's native geometry across diverse PDE benchmarks and optimization algorithms, it achieves superior or comparable performance in the vast majority of configurations without requiring gradient modifications. This approach represents a significant methodological advance in handling constraints for complex machine learning models involving structured preconditioning and adaptive scaling.
ORACLE: Optimizer-Relative Alignment for Constrained LEarning
arXiv:2610.09040v1 Announce Type: new Abstract: Constraint handling methods typically intervene before the optimizer acts, by modifying the objective or the gradient. Yet momentum, adaptive scaling, and structured preconditioning can substantially reshape that signal before it becomes a parameter update. We formulate optimizer relative constrained learning, where constraint compatibility is assessed on the post optimizer update. Building on this view, we introduce ORACLE, which evaluates the native optimizer's realized step through a joint endpoint linearization of heterogeneous constraint families, constructs the resulting alignment in the optimizer's own geometry, bounds its authority, and commits it only after validation. We evaluate ORACLE across eight Partial Differential Equation benchmarks and four optimizers spanning Euclidean, diagonal adaptive, and structured preconditioned geometries, where it improves or matches native optimizer in 94% of configurations. Cross model analysis shows the same behavior in 92% of configurations, while matched comparisons show improvements over alternative constraint-handling methods acting at the objective, gradient, and post-optimizer levels.
Abstract:Constraint handling methods typically intervene before the optimizer acts, by modifying the objective or the gradient. Yet momentum, adaptive scaling, and structured preconditioning can substantially reshape that signal before it becomes a parameter update. We formulate optimizer relative constrained learning, where constraint compatibility is assessed on the post optimizer update. Building on this view, we introduce ORACLE, which evaluates the native optimizer's realized step through a joint endpoint linearization of heterogeneous constraint families, constructs the resulting alignment in the optimizer's own geometry, bounds its authority, and commits it only after validation. We evaluate ORACLE across eight Partial Differential Equation benchmarks and four optimizers spanning Euclidean, diagonal adaptive, and structured preconditioned geometries, where it improves or matches native optimizer in 94% of configurations. Cross model analysis shows the same behavior in 92% of configurations, while matched comparisons show improvements over alternative constraint-handling methods acting at the objective, gradient, and post-optimizer levels.
Machine Learning (cs.LG)
arXiv:2610.09040 [cs.LG]
(or arXiv:2610.09040v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.09040 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 244 cuvinte
Subiecte conexe
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · 8 oct. 2026
A Zeroth-Order Paradigm for LLM Preference Alignment
arXiv · 17 sept. 2026
Task Specialization Fine-Tuning for Contextual Reinforcement Learning
arXiv · 20 aug. 2026
Lattice-Based Cryptography Explained
Quantum Zeitgeist · 15 aug. 2026