analiza aiqbench
This paper proposes DiTaR, a method for learning dense robot-manipulation rewards from videos by modeling task dynamics in a latent space that filters out visual distractors. Its evaluation covers simulated benchmarks (MetaWorld-v3, ManiSkill) and offline reward relabeling on real-world data, so the claims rest on policy-learning gains over video-reward baselines rather than real-world deployment. Readers working on reward learning or robot RL should check how well the latent disentanglement generalizes beyond these benchmarks.
Disentangling Task-Relevant Latent Dynamics for Reward Modeling
arXiv:2610.14159v1 Announce Type: new Abstract: Reward design remains a major bottleneck for reinforcement learning in robotic manipulation. Handcrafted dense rewards can compensate for the limited guidance provided by sparse rewards, but require substantial task-specific reward engineering. Learning rewards from videos provides a scalable, data-driven alternative to manual reward design. However, existing video-based rewards are often computed in entangled visual spaces where task-relevant state changes are confounded with extraneous visual variation. As a result, the learned reward may respond to visual distractors instead of actual task execution. To address this limitation, we propose DiTaR (Disentangling Task-Relevant Latent Dynamics for Reward Modeling), a framework for learning dense rewards from heterogeneous videos. DiTaR learns a continuous task-relevant representation from multi-frame video latents through an asymmetric two-stage procedure that suppresses task-irrelevant visual variation. In the task-relevant latent space, we train a history- and language-conditioned rectified-flow model to capture task dynamics and derive a directional dense reward from temporal changes in dynamics compatibility. Together, these components enable reward modeling directly over task-relevant dynamics rather than generic visual variation. DiTaR substantially improves policy learning on MetaWorld-v3 and ManiSkill over video-based reward baselines, and transfers to real-world offline RL through reward relabeling. Reward analyses further show that DiTaR provides more reliable rewards.
Abstract:Reward design remains a major bottleneck for reinforcement learning in robotic manipulation. Handcrafted dense rewards can compensate for the limited guidance provided by sparse rewards, but require substantial task-specific reward engineering. Learning rewards from videos provides a scalable, data-driven alternative to manual reward design. However, existing video-based rewards are often computed in entangled visual spaces where task-relevant state changes are confounded with extraneous visual variation. As a result, the learned reward may respond to visual distractors instead of actual task execution. To address this limitation, we propose DiTaR (Disentangling Task-Relevant Latent Dynamics for Reward Modeling), a framework for learning dense rewards from heterogeneous videos. DiTaR learns a continuous task-relevant representation from multi-frame video latents through an asymmetric two-stage procedure that suppresses task-irrelevant visual variation. In the task-relevant latent space, we train a history- and language-conditioned rectified-flow model to capture task dynamics and derive a directional dense reward from temporal changes in dynamics compatibility. Together, these components enable reward modeling directly over task-relevant dynamics rather than generic visual variation. DiTaR substantially improves policy learning on MetaWorld-v3 and ManiSkill over video-based reward baselines, and transfers to real-world offline RL through reward relabeling. Reward analyses further show that DiTaR provides more reliable rewards.
Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2610.14159 [cs.RO]
(or arXiv:2610.14159v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2610.14159 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 299 cuvinte
Subiecte conexe
CausalDreamer: Learning Predictive World Models with Latent Disentanglement
arXiv · 9 oct. 2026
Reflections on unlearning and inoculation
LessWrong · 20 sept. 2026
[AINews] not much happened today
Latent Space · 3 oct. 2026
PIRAMID: Progress and Plans
LessWrong · 27 iul. 2026