aiqbench analysis
ZeroWBC addresses the high cost of teleoperation data by leveraging human egocentric videos to train whole-body humanoid interaction policies without direct robot demonstrations. The framework utilizes a generation-then-tracking architecture where a Vision-Language Model predicts motion tokens that are subsequently decoded and retargeted to the robot via an interactive tracking policy. This approach offers a scalable pathway for deploying diverse, scene-aware behaviors on humanoids like the Unitree G1, potentially accelerating the transition from research prototypes to practical applications.
ZeroWBC: Learning Natural Whole-Body Humanoid Interaction from Human Egocentric Data
arXiv:2603.09170v2 Announce Type: replace Abstract: Achieving versatile and natural whole-body humanoid interaction control remains challenging due to the high cost of whole-body teleoperation data. We present ZeroWBC, a teleoperation-free framework that learns humanoid whole-body interaction from human egocentric videos paired with synchronized whole-body motion and text annotations. ZeroWBC adopts a generation-then-tracking formulation to tackle the static scene whole-body interaction control problem. Given an initial egocentric image and a language instruction, a fine-tuned Vision-Language Model generates future human whole-body motion tokens, which are decoded into continuous motions and retargeted to the humanoid. The resulting reference motions, together with root and key body-part trajectories, are then executed by a general interactive motion tracking policy. To improve interaction performance, we introduce an interaction-oriented tracking reward that prioritizes global root and key body-part trajectory alignment while preserving natural whole-body motion. Experiments on the Unitree G1 humanoid robot show that ZeroWBC enables diverse scene-aware behaviors without robot teleoperation demonstrations. These results suggest a scalable paradigm for learning natural humanoid whole-body interaction from human egocentric data.
Abstract:Achieving versatile and natural whole-body humanoid interaction control remains challenging due to the high cost of whole-body teleoperation data. We present ZeroWBC, a teleoperation-free framework that learns humanoid whole-body interaction from human egocentric videos paired with synchronized whole-body motion and text annotations. ZeroWBC adopts a generation-then-tracking formulation to tackle the static scene whole-body interaction control problem. Given an initial egocentric image and a language instruction, a fine-tuned Vision-Language Model generates future human whole-body motion tokens, which are decoded into continuous motions and retargeted to the humanoid. The resulting reference motions, together with root and key body-part trajectories, are then executed by a general interactive motion tracking policy. To improve interaction performance, we introduce an interaction-oriented tracking reward that prioritizes global root and key body-part trajectory alignment while preserving natural whole-body motion. Experiments on the Unitree G1 humanoid robot show that ZeroWBC enables diverse scene-aware behaviors without robot teleoperation demonstrations. These results suggest a scalable paradigm for learning natural humanoid whole-body interaction from human egocentric data.
Robotics (cs.RO); Artificial Intelligence (cs.AI)
arXiv:2603.09170 [cs.RO]
(or arXiv:2603.09170v2 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2603.09170 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 259 words
Related coverage
iGPC: Generative Motion Priors for Object-Aware Humanoid Interaction
arXiv · Oct 7, 2026
Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy — Jason Ma, Dyna Robotics
Latent Space · Oct 13, 2026
WT-UMI: Tactile-based Whole-Body Manipulation via Force-Supervised Contact-Aware Planning
arXiv · Jun 12, 2026
LYRIC: Language-Driven Physics-Based Character Control for Contact-Rich Whole-Body Object Interaction
arXiv · Sep 18, 2026