aiqbench analysis
ViLoMan represents a significant advancement in autonomous humanoid robotics by solving the critical challenge of integrating locomotion and manipulation through direct learning from onboard depth sensing and proprioception. The framework successfully bridges the simulation-to-reality gap, enabling a Unitree G1 to perform complex whole-body tasks like door-closing without relying on pre-recorded reference motions or intermediate commands. This work pushes the boundaries of scalable, adaptive control policies essential for next-generation humanoids operating in unstructured environments.
ViLoMan: Learning Visual-Proprioceptive Whole-Body Loco-Manipulation Skills for Humanoid Robots
arXiv:2609.19340v1 Announce Type: new Abstract: Humanoid loco-manipulation requires adaptive whole-body coordination to seamlessly integrate locomotion and physical interaction. Despite recent advances, learning autonomous loco-manipulation remains challenging due to the scarcity of diverse, physically executable robot-object interaction data and the difficulty of learning unified whole-body control directly from onboard observations. We present ViLoMan, a scalable framework for autonomous humanoid loco-manipulation. ViLoMan first transforms partial kinematic demonstrations of human-object interactions into complete, physically executable robot trajectories. It then leverages these trajectories within a teacher-student distillation framework to learn a unified policy that maps egocentric depth observations and proprioceptive measurements directly to joint-level whole-body actions. During deployment, the policy requires neither reference motions nor intermediate commands. We evaluate ViLoMan on door-closing tasks across diverse door configurations and robot initial conditions in both simulation and the real world. Experimental results demonstrate that a single policy enables a Unitree G1 humanoid to complete the full task using only onboard depth sensing and proprioception, while generalizing robustly across task variations and transferring effectively from simulation to reality. Project page: viloman-anonymous.pages.dev.
Abstract:Humanoid loco-manipulation requires adaptive whole-body coordination to seamlessly integrate locomotion and physical interaction. Despite recent advances, learning autonomous loco-manipulation remains challenging due to the scarcity of diverse, physically executable robot-object interaction data and the difficulty of learning unified whole-body control directly from onboard observations. We present ViLoMan, a scalable framework for autonomous humanoid loco-manipulation. ViLoMan first transforms partial kinematic demonstrations of human-object interactions into complete, physically executable robot trajectories. It then leverages these trajectories within a teacher-student distillation framework to learn a unified policy that maps egocentric depth observations and proprioceptive measurements directly to joint-level whole-body actions. During deployment, the policy requires neither reference motions nor intermediate commands. We evaluate ViLoMan on door-closing tasks across diverse door configurations and robot initial conditions in both simulation and the real world. Experimental results demonstrate that a single policy enables a Unitree G1 humanoid to complete the full task using only onboard depth sensing and proprioception, while generalizing robustly across task variations and transferring effectively from simulation to reality. Project page: this http URL.
10 pages, 7 figures. Project page: this https URL
Robotics (cs.RO)
arXiv:2609.19340 [cs.RO]
(or arXiv:2609.19340v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2609.19340 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 273 words
Related coverage
$\omega$-0: A Latent Predictive World Action Model for Concurrent Humanoid Loco-Manipulation
arXiv · Aug 7, 2026
Being-M0.7: A Latent World-Action Model for Humanoid Robots
arXiv · Oct 9, 2026
DreamMimic: Learning Visuomotor Whole-Body Loco-Manipulation via World Model
arXiv · Aug 25, 2026
Uni-VLaT: Whole-Body Tactile Adaptation of VLA Policies for Humanoid Loco-Manipulation
arXiv · Sep 29, 2026