aiqbench analysis
Temporal Self-Imitation Learning (TSIL) addresses a critical bottleneck in reinforcement learning by converting discarded inefficient trajectories into self-supervisory signals, thereby enhancing sample efficiency and robustness across long-horizon robotic tasks. This method demonstrates that mining for temporal efficiency within existing training data can significantly improve policy convergence without requiring external reward models or massive new datasets.
Temporal Self-Imitation Learning
arXiv:2606.19752v3 Announce Type: replace Abstract: Long-horizon policies trained with reinforcement learning can still achieve high return through inefficient interactions, while rare efficient behaviors discovered during training may be forgotten. We argue that temporal efficiency itself provides a source of self-supervision for reinforcement learning. We introduce Temporal Self-Imitation Learning (TSIL), a reinforcement learning framework that mines temporally efficient successful trajectories generated during learning and converts them into reusable supervision for future policy improvement. TSIL progressively refines learning using configuration-conditioned adaptive temporal targets derived from fast successful trajectories, while preserving and replaying efficient behaviors through efficiency-weighted self-imitation learning. Across 30 long-horizon tasks spanning robot manipulation and interactive navigation, TSIL consistently improves task success rates, learning efficiency, behavioral efficiency, and robustness to unstable training conditions.
Abstract:Long-horizon policies trained with reinforcement learning can still achieve high return through inefficient interactions, while rare efficient behaviors discovered during training may be forgotten. We argue that temporal efficiency itself provides a source of self-supervision for reinforcement learning. We introduce Temporal Self-Imitation Learning (TSIL), a reinforcement learning framework that mines temporally efficient successful trajectories generated during learning and converts them into reusable supervision for future policy improvement. TSIL progressively refines learning using configuration-conditioned adaptive temporal targets derived from fast successful trajectories, while preserving and replaying efficient behaviors through efficiency-weighted self-imitation learning. Across 30 long-horizon tasks spanning robot manipulation and interactive navigation, TSIL consistently improves task success rates, learning efficiency, behavioral efficiency, and robustness to unstable training conditions.
Robotics (cs.RO); Artificial Intelligence (cs.AI)
arXiv:2606.19752 [cs.RO]
(or arXiv:2606.19752v3 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2606.19752 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 211 words
Related coverage
SeedPolicy: Horizon Scaling via Self-Evolving Diffusion Policy for Robot Manipulation
arXiv · Jul 31, 2026
What's Continual Learning, and Why Might We Expect To See It In Advanced LLM Agents?
LessWrong · Jun 12, 2026
Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories
arXiv · Jun 3, 2026
Robot Learning to Communicate through Projected Visual Abstractions
arXiv · Jul 28, 2026