analiza aiqbench
DEXCAST advances dexterous robotics by enabling robots to learn complex hand-object manipulation skills directly from human videos without costly teleoperation. The framework addresses high-dimensional motion modeling through a novel perception module that captures temporal dependencies to condition diffusion policies for future action generation. This research significantly lowers the barrier for deploying advanced robotic hands in real-world scenarios by eliminating manual demonstration requirements.
DexCast: Learning Real-World Dexterous Control from Human Videos
arXiv:2610.14691v1 Announce Type: cross Abstract: Recent advances in 3D hand-object interaction (HOI) perception have opened new opportunities for learning dexterous robotic skills from human videos. However, existing dexterous imitation learning methods often rely on costly and manually teleoperated robot demonstrations, requiring substantial human effort. Therefore, we build DEXCAST, a framework that learns dexterous robotic hand control directly from monocular RGB-D videos by leveraging joint HOI reconstructions, without requiring any teleoperation. Furthermore, how to model high-dimensional robotic hand motion and extract temporal context to inform future action generation remains a key challenge. To this end, we propose a DexHand Motion Perception Module (DMPM) to capture inter-joint coordination and temporal dependencies in compact latent representations that condition our diffusion policy. Extensive real-world experiments demonstrate that DEXCAST achieves exceptional performance across diverse dexterous manipulation tasks.
Abstract:Recent advances in 3D hand-object interaction (HOI) perception have opened new opportunities for learning dexterous robotic skills from human videos. However, existing dexterous imitation learning methods often rely on costly and manually teleoperated robot demonstrations, requiring substantial human effort. Therefore, we build DEXCAST, a framework that learns dexterous robotic hand control directly from monocular RGB-D videos by leveraging joint HOI reconstructions, without requiring any teleoperation. Furthermore, how to model high-dimensional robotic hand motion and extract temporal context to inform future action generation remains a key challenge. To this end, we propose a DexHand Motion Perception Module (DMPM) to capture inter-joint coordination and temporal dependencies in compact latent representations that condition our diffusion policy. Extensive real-world experiments demonstrate that DEXCAST achieves exceptional performance across diverse dexterous manipulation tasks.
Robotics (cs.RO); Artificial Intelligence (cs.AI)
arXiv:2610.14691 [cs.RO]
(or arXiv:2610.14691v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2610.14691 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 223 cuvinte
Subiecte conexe
Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy — Jason Ma, Dyna Robotics
Latent Space · 13 oct. 2026
The Machines that Make the Machines
NVIDIA · 7 oct. 2026
2026 BAIR Graduate Showcase
BAIR Blog · 1 iul. 2026
HiFun: a Hierarchical Framework for Efficient Functional Dexterous Manipulation Learning
arXiv · 15 oct. 2026