analiza aiqbench
GRAIL addresses the scalability bottleneck in humanoid robot training by generating over 20,000 synthetic locomotion-manipulation sequences entirely within a virtual environment. By leveraging known 3D configurations and video foundation model priors, the pipeline reduces depth ambiguity and morphology mismatch to produce high-fidelity data for sim-to-real deployment. The approach demonstrates practical efficacy on a Unitree G1, achieving 84% success in object pick-up and 90% in stair-climbing without requiring physical teleoperation or motion capture setups.
GRAIL: Generating Humanoid Loco-Manipulation from 3D Assets and Video Priors
arXiv:2606.05160v1 Announce Type: new Abstract: Scaling humanoid loco-manipulation requires robot-compatible demonstrations across diverse objects, whole-body motions, and scene geometries, but teleoperation and motion capture are difficult to scale because each collection depends on physical setups, instrumented actors, and robot operation. We present GRAIL, a digital generation pipeline that remains fully virtual until deployment: it composes 3D assets, simulator-ready scenes, and priors from video foundation models (VFMs) to synthesize interactions without rebuilding physical environments or teleoperating the robot. Rather than reconstructing unconstrained in-the-wild videos, GRAIL starts from fully specified 3D configurations in which object geometry, camera parameters, metric scale, environment depth, and a robot-proportioned character are known before video generation and reused during reconstruction. This privileged setup better conditions 4D recovery, allowing model-based object tracking, human motion estimation, and interaction-aware optimization to reconstruct metric 4D human-object interaction (HOI) trajectories with reduced depth ambiguity and morphology mismatch. We retarget the recovered motions to a humanoid robot and train complementary task-general trackers: an object-aware latent adaptor for manipulation and a scene-aware tracker for terrain traversal. GRAIL produces over 20,000 sequences spanning pick-up, object manipulation, sitting, and terrain traversal. Using only GRAIL-generated data, we train egocentric visual policies through a sim-to-real pipeline and deploy them on a Unitree G1 humanoid, achieving 84\% real-world success on diverse object pick-up and 90\% success on stair-climbing.
Abstract:Scaling humanoid loco-manipulation requires robot-compatible demonstrations across diverse objects, whole-body motions, and scene geometries, but teleoperation and motion capture are difficult to scale because each collection depends on physical setups, instrumented actors, and robot operation. We present GRAIL, a digital generation pipeline that remains fully virtual until deployment: it composes 3D assets, simulator-ready scenes, and priors from video foundation models (VFMs) to synthesize interactions without rebuilding physical environments or teleoperating the robot. Rather than reconstructing unconstrained in-the-wild videos, GRAIL starts from fully specified 3D configurations in which object geometry, camera parameters, metric scale, environment depth, and a robot-proportioned character are known before video generation and reused during reconstruction. This privileged setup better conditions 4D recovery, allowing model-based object tracking, human motion estimation, and interaction-aware optimization to reconstruct metric 4D human-object interaction (HOI) trajectories with reduced depth ambiguity and morphology mismatch. We retarget the recovered motions to a humanoid robot and train complementary task-general trackers: an object-aware latent adaptor for manipulation and a scene-aware tracker for terrain traversal. GRAIL produces over 20,000 sequences spanning pick-up, object manipulation, sitting, and terrain traversal. Using only GRAIL-generated data, we train egocentric visual policies through a sim-to-real pipeline and deploy them on a Unitree G1 humanoid, achieving 84\% real-world success on diverse object pick-up and 90\% success on stair-climbing.
Robotics (cs.RO)
arXiv:2606.05160 [cs.RO]
(or arXiv:2606.05160v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2606.05160 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 231 cuvinte
Subiecte conexe
NVIDIA GTC Taipei at COMPUTEX: Live Updates on What’s Next in AI
NVIDIA · 21 mai 2026
KinoGen: Customizable Humanoid Loco-Manipulation References via Kinematically Grounded Video Generation
arXiv · 15 oct. 2026
OASIS: From Simulation Data Collection to Real-World Humanoid Loco-Manipulation
arXiv · 9 iun. 2026
$\omega$-0: A Latent Predictive World Action Model for Concurrent Humanoid Loco-Manipulation
arXiv · 7 aug. 2026