analiza aiqbench
RoboMirror addresses a critical bottleneck in humanoid robotics by using Vision-Language Models to extract motion intent directly from video, bypassing the need for explicit pose reconstruction or retargeting. This approach bridges the gap between visual understanding and physical control, enabling more responsive telepresence with significantly reduced latency compared to existing text-to-motion pipelines. The framework demonstrates that genuine semantic interpretation of raw footage yields superior task success rates over mechanical mimicry methods.
RoboMirror: Understand Before You Imitate for Video to Humanoid Locomotion
arXiv:2512.23649v4 Announce Type: replace Abstract: Humans learn locomotion through visual observation, interpreting visual content first before imitating actions. However, state-of-the-art humanoid locomotion systems rely on either curated motion capture trajectories or sparse text commands, leaving a critical gap between visual understanding and control. Text-to-motion methods suffer from semantic sparsity and staged pipeline errors, while video-based approaches only perform mechanical pose mimicry without genuine visual understanding. We propose RoboMirror, the first retargeting-free video-to-locomotion framework embodying "understand before you imitate". Leveraging VLMs, it distills raw egocentric/third-person videos into visual motion intents, which directly condition a diffusion-based policy to generate physically plausible, semantically aligned locomotion without explicit pose reconstruction or retargeting. Extensive experiments validate the effectiveness of RoboMirror, it enables telepresence via egocentric videos, drastically reduces third-person control latency by 80%, and achieves a 3.7% higher task success rate than baselines. By reframing humanoid control around video understanding, we bridge the visual understanding and action gap.
Authors:Zhe Li, Boan Zhu, Yangyang Wei, Shuanghao Bai, Yuheng Ji, Yibo Peng, Tao Huang, Pengwei Wang, Zhongyuan Wang, S.-H. Gary Chan, Chang Xu, Cheng Chi, Jianfei Yang, Shanghang Zhang
Abstract:Humans learn locomotion through visual observation, interpreting visual content first before imitating actions. However, state-of-the-art humanoid locomotion systems rely on either curated motion capture trajectories or sparse text commands, leaving a critical gap between visual understanding and control. Text-to-motion methods suffer from semantic sparsity and staged pipeline errors, while video-based approaches only perform mechanical pose mimicry without genuine visual understanding. We propose RoboMirror, the first retargeting-free video-to-locomotion framework embodying "understand before you imitate". Leveraging VLMs, it distills raw egocentric/third-person videos into visual motion intents, which directly condition a diffusion-based policy to generate physically plausible, semantically aligned locomotion without explicit pose reconstruction or retargeting. Extensive experiments validate the effectiveness of RoboMirror, it enables telepresence via egocentric videos, drastically reduces third-person control latency by 80%, and achieves a 3.7% higher task success rate than baselines. By reframing humanoid control around video understanding, we bridge the visual understanding and action gap.
Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2512.23649 [cs.RO]
(or arXiv:2512.23649v4 [cs.RO] for this version)
Text integral sub licența deschisă a publicației · 193 cuvinte
Subiecte conexe
PHUMA: Physically Reliable Humanoid Locomotion Dataset
arXiv · 5 iun. 2026
PHUMA: Physically Reliable Humanoid Locomotion Dataset
arXiv · 14 oct. 2026
NVIDIA GTC Taipei at COMPUTEX: Live Updates on What’s Next in AI
NVIDIA · 21 mai 2026
Towards a General Humanoid Loco-Manipulation Model via Egocentric Whole-Body Human Data Pretraining
arXiv · 2 oct. 2026