aiqbench analysis
This research introduces TRACC, a novel framework that transforms failed human demonstrations into valuable training data for humanoid robots by extracting usable motion prefixes and inferring intended outcomes. By leveraging these partial trajectories, the method enables skill acquisition without requiring perfect, successful reference videos, addressing a significant bottleneck in current robot learning pipelines. This approach significantly expands the scope of available supervision signals, making it easier to train robust policies from real-world, imperfect video sources.
Track-and-Complete: Learning Humanoid Skills from a Single Failed Human Video
arXiv:2609.36924v1 Announce Type: new Abstract: Learning humanoid skills from videos typically requires a successful human demonstration, which often demands custom data collection. Although failures have traditionally been treated only as negative examples in robot learning, they can still reveal a usable trajectory prefix before the task fails, as well as the intended outcome. To leverage this information from a failed-attempt video, we propose TRACC, a pipeline that imitates the useful portion of the motion trajectory and then completes the task based on the inferred task outcome. The usable motion prefix serves as prior knowledge until the failure occurs, after which the task-completion reward guides the policy to learn the intended task goal without requiring a successful task trajectory. We evaluate our method on six in-the-wild failed human tasks from the Oops! dataset. Our experimental results demonstrate the effectiveness of the proposed approach for learning from failed attempts when no successful demonstration is available. Thus, these findings establish failed human videos as a viable source of supervision for humanoid skill learning.
Abstract:Learning humanoid skills from videos typically requires a successful human demonstration, which often demands custom data collection. Although failures have traditionally been treated only as negative examples in robot learning, they can still reveal a usable trajectory prefix before the task fails, as well as the intended outcome. To leverage this information from a failed-attempt video, we propose TRACC, a pipeline that imitates the useful portion of the motion trajectory and then completes the task based on the inferred task outcome. The usable motion prefix serves as prior knowledge until the failure occurs, after which the task-completion reward guides the policy to learn the intended task goal without requiring a successful task trajectory. We evaluate our method on six in-the-wild failed human tasks from the Oops! dataset. Our experimental results demonstrate the effectiveness of the proposed approach for learning from failed attempts when no successful demonstration is available. Thus, these findings establish failed human videos as a viable source of supervision for humanoid skill learning.
8 pages, 6 figures, 6 tables. Project website: this https URL
Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2609.36924 [cs.RO]
(or arXiv:2609.36924v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2609.36924 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 275 words
Related coverage
Links #5: 2026/07
LessWrong · Jul 31, 2026
Synthesis Superintelligence: from Semiconductors to Superconductors — Periodic Labs’ Liam Fedus and Ekin Dogus Cubuk
Latent Space · Oct 8, 2026
How to Evaluate General-Purpose Robot Policies for Real-World Deployment
NVIDIA · Jul 12, 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026