ReBound: Reset-Free Reinforcement Learning for Agile Driving via Reset-Aware Semi-Markov Bootstrapping

Abstract:We present ReBound (reset-aware semi-Markov bootstrapping), a value-learning method for reset-free reinforcement learning (RL) that learns agile driving through real-world RL without human intervention after training starts. Reset-free RL explores by alternating between a forward policy that performs the task and a reset policy that restores a drivable state. Standard reset-free RL treats collisions as episode terminations in value-function bootstrapping, truncating the return at each collision. This conflicts with the reset-free objective, under which driving resumes after recovery, and acts as a hidden collision penalty absent from the reward function. ReBound instead applies semi-Markov bootstrapping: it treats the collision-causing action and the subsequent recovery as a single semi-Markov transition and bootstraps from the post-recovery value, discounted by the actual recovery time. We further apply reward centering to stabilize value learning in the resulting continuing task. In simulation and on a 1/10-scale vehicle, ReBound substantially outperforms conventional value-learning methods and model-based control.

Robotics (cs.RO)

arXiv:2604.07672 [cs.RO]

(or arXiv:2604.07672v2 [cs.RO] for this version)

https://doi.org/10.48550/arXiv.2604.07672 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.