analiza aiqbench
This research introduces GERS, a bilevel optimization framework that enhances reinforcement learning generalization using only scalar feedback when full trajectory data is unavailable. By leveraging evolutionary algorithms to shape rewards at the upper level, the method achieves performance comparable to domain randomization while operating under stricter privacy constraints. The approach offers a practical solution for deploying RL agents in restricted environments where detailed state observations cannot be shared or collected.
Evolutionary Bilevel Reward Shaping for Generalization in Reinforcement Learning
arXiv:2606.16236v1 Announce Type: new Abstract: Reinforcement learning (RL) often suffers from performance degradation when deployed in environments that differ from those encountered during training. Existing techniques such as domain randomization (DR) mitigate this, but require access to diverse training environments and full trajectory observability, assumptions that fail in privacy-preserving or restricted scenarios where only scalar performance metrics are available. We propose Generalization via Evolutionary Reward Shaping (GERS), a bilevel optimization approach to improve generalization on unseen test environments using only scalar feedback from validation environments. At the lower level, an RL agent guided via a reward function shaped by the upper level learns a policy on a limited set of training environments with accessible trajectory data; at the upper level, CMA-ES optimizes the reward shaping parameters to maximize the cumulative unshaped reward on separate validation environments for which trajectory access is unavailable. Results on continuous control tasks indicate that GERS outperforms the standard RL baseline on unseen test environments. GERS performance is comparable to DR, despite DR treating the combined set of training and validation environments of GERS as a single training set that requires trajectory access, whereas GERS cannot access validation trajectories. These results confirm that GERS effectively enhances generalization under restricted data access constraints.
Abstract:Reinforcement learning (RL) often suffers from performance degradation when deployed in environments that differ from those encountered during training. Existing techniques such as domain randomization (DR) mitigate this, but require access to diverse training environments and full trajectory observability, assumptions that fail in privacy-preserving or restricted scenarios where only scalar performance metrics are available. We propose Generalization via Evolutionary Reward Shaping (GERS), a bilevel optimization approach to improve generalization on unseen test environments using only scalar feedback from validation environments. At the lower level, an RL agent guided via a reward function shaped by the upper level learns a policy on a limited set of training environments with accessible trajectory data; at the upper level, CMA-ES optimizes the reward shaping parameters to maximize the cumulative unshaped reward on separate validation environments for which trajectory access is unavailable. Results on continuous control tasks indicate that GERS outperforms the standard RL baseline on unseen test environments. GERS performance is comparable to DR, despite DR treating the combined set of training and validation environments of GERS as a single training set that requires trajectory access, whereas GERS cannot access validation trajectories. These results confirm that GERS effectively enhances generalization under restricted data access constraints.
Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
arXiv:2606.16236 [cs.LG]
(or arXiv:2606.16236v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.16236 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 300 cuvinte
Subiecte conexe
Generalization Hacking: Models Can Game Reinforcement Learning by Preventing Behavioral Generalization
arXiv · 12 iun. 2026
Generalized Kalman filter based temporal difference reinforcement learning
arXiv · 23 iul. 2026
Athena-Brain Technical Report: An Efficient Robot Brain for General Intelligence and Embodied Interactio
arXiv · 23 iul. 2026
GRASP: Group-Adaptive Hybrid RL-SFT Post-Training for Medical VQA without Reasoning Annotations
arXiv · 27 oct. 2026