aiqbench analysis
This research introduces residual reinforcement learning as a method to decouple task performance from safety constraints in humanoid robots, addressing the critical challenge of preventing instability during complex interactions. By training a separate residual policy specifically for safety corrections, the approach achieves a superior Pareto trade-off without requiring manual tuning of competing reward terms within a single model. The work advances AI safety protocols essential for deploying autonomous humanoids in unstructured environments.
ResSafe: Learning Safety Filtering with Residual Reinforcement Learning for Humanoids
arXiv:2609.15988v1 Announce Type: new Abstract: Safe control of humanoid robots remains challenging due to their high-dimensional dynamics, contact-rich interactions, and sensitivity to disturbances. Although reinforcement learning has enabled effective locomotion and motion tracking, learned policies can still generate unsafe actions that lead to instability or falls. In this work, we propose residual reinforcement learning as an implicit safety-filtering mechanism for safe humanoid control. Instead of relying on a single nominal policy to simultaneously balance performance, safety, and robustness, we decouple performance and safety. The nominal policy focuses solely on task performance, while a residual policy learns safety corrections. This decoupling leads to a better performance--safety Pareto trade-off and avoids the need for careful tuning of multiple competing reward terms within a single policy training. We show that the residual policy can act as an implicit safety filter.
Abstract:Safe control of humanoid robots remains challenging due to their high-dimensional dynamics, contact-rich interactions, and sensitivity to disturbances. Although reinforcement learning has enabled effective locomotion and motion tracking, learned policies can still generate unsafe actions that lead to instability or falls. In this work, we propose residual reinforcement learning as an implicit safety-filtering mechanism for safe humanoid control. Instead of relying on a single nominal policy to simultaneously balance performance, safety, and robustness, we decouple performance and safety. The nominal policy focuses solely on task performance, while a residual policy learns safety corrections. This decoupling leads to a better performance--safety Pareto trade-off and avoids the need for careful tuning of multiple competing reward terms within a single policy training. We show that the residual policy can act as an implicit safety filter.
Robotics (cs.RO)
arXiv:2609.15988 [cs.RO]
(or arXiv:2609.15988v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2609.15988 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 225 words
Related coverage
Robust Koopman Control Barrier Filters for Safe Actor-Critic Reinforcement Learning
arXiv · Jun 9, 2026
A Risk-Sensitive and Uncertainty-Aware Decision-Making and Control Framework for Safe and Robust Autonomous Driving
arXiv · Sep 10, 2026
HetGPS: Scalable Graph Multi-Agent Reinforcement Learning with Physics-Anchored Adaptive Safety for EV Charging
arXiv · Aug 5, 2026
Complete Guide to Thinking Machines Inkling
Analytics Vidhya · Jul 20, 2026