analiza aiqbench
This paper introduces RACER, a novel framework for robust adversarial reinforcement learning that enhances stability through risk-sensitive objectives and critic consistency regularization. By adaptively regulating perturbation strength and reducing disagreement between Q-value estimators, the method effectively mitigates brittleness under distributional shifts while maintaining informative exploration. The approach demonstrates superior performance and training stability on continuous control benchmarks compared to existing robust RL baselines.
Robust Adversarial Reinforcement Learning with Risk Sensitivity and Critic Consistency Regularization
arXiv:2609.27667v1 Announce Type: new Abstract: Reinforcement learning (RL) achieves strong performance in sequential decision-making but remains brittle under dynamic uncertainty and distributional shifts. Robust Adversarial Reinforcement Learning (RARL) improves robustness via worst-case perturbations, but existing approaches frequently suffer from unstable optimization and degraded value estimation. In particular, overly aggressive adversaries can drive the agent toward uninformative failure states, while adversarial perturbations amplify disagreement between double critics and introduce biased value targets. We propose a unified framework, RACER (Risk-sensitive robust Adversarial critic ConsistEncy-regularized Reinforcement learning), that revisits adversarial RL from a risk-sensitive perspective. First, we introduce a state-dependent adversarial objective that adaptively regulates perturbation strength, suppressing harmful disturbances while preserving informative exploration. Second, we propose critic consistency regularization to reduce disagreement between Q-value estimators and stabilize learning. Comprehensive experiments on challenging continuous control benchmarks demonstrate that RACER consistently improves performance, robustness, and training stability over strong robust RL baselines.
Abstract:Reinforcement learning (RL) achieves strong performance in sequential decision-making but remains brittle under dynamic uncertainty and distributional shifts. Robust Adversarial Reinforcement Learning (RARL) improves robustness via worst-case perturbations, but existing approaches frequently suffer from unstable optimization and degraded value estimation. In particular, overly aggressive adversaries can drive the agent toward uninformative failure states, while adversarial perturbations amplify disagreement between double critics and introduce biased value targets. We propose a unified framework, RACER (Risk-sensitive robust Adversarial critic ConsistEncy-regularized Reinforcement learning), that revisits adversarial RL from a risk-sensitive perspective. First, we introduce a state-dependent adversarial objective that adaptively regulates perturbation strength, suppressing harmful disturbances while preserving informative exploration. Second, we propose critic consistency regularization to reduce disagreement between Q-value estimators and stabilize learning. Comprehensive experiments on challenging continuous control benchmarks demonstrate that RACER consistently improves performance, robustness, and training stability over strong robust RL baselines.
Machine Learning (cs.LG)
arXiv:2609.27667 [cs.LG]
(or arXiv:2609.27667v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.27667 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 237 cuvinte
Subiecte conexe
Certifying Lower Bounds for Risk-Sensitive Reinforcement Learning under Adversarial State Perturbations
arXiv · 11 sept. 2026
Gemini 4 Argon: Built to solve your hardest problems
Google DeepMind · 13 oct. 2026
State of Pandemic Early Warning
LessWrong · 24 sept. 2026
An overview of frontier AI company monitoring practices in mid 2026
LessWrong · 7 oct. 2026