analiza aiqbench
EgoAERO addresses a critical bottleneck in robot learning by enabling dexterous manipulation policies to be derived from a single egocentric video without requiring pre-scanned object assets. The framework achieves performance comparable to CAD-based reconstructions by reconstructing contact-consistent trajectories and introducing an online quality assessment mechanism. Additionally, the release of the EgoDex-R dataset, containing 4.3M RGB-D frames, provides a significant new resource for training and validating dexterous robot policies.
EgoAERO: Learning Dexterous Manipulation from a Single Egocentric Video without Object Assets
arXiv:2606.08057v1 Announce Type: new Abstract: Egocentric RGB-D videos offer a natural source of human dexterous manipulation demonstrations, but existing data is difficult to use for robot learning because object pose, geometry, and contact information are often missing or require pre-scanned object assets. We present EgoAERO, the first framework that learns dexterous manipulation from a single egocentric RGB-D human demonstration without object assets. EgoAERO reconstructs contact-consistent hand-object trajectories through asset-free object tracking and reconstruction, ego motion compensation, and adaptive contact optimization, then converts them into robot policies using two-stage residual learning. We further introduce an online quality assessment mechanism and construct EgoDex-R, a large-scale egocentric dataset with 4.3M RGB-D frames for dexterous policy learning. Simulation and real-world experiments show that EgoAERO enables single-demonstration dexterous manipulation and achieves downstream performance close to CAD-based reconstructions on HOI4D.
Abstract:Egocentric RGB-D videos offer a natural source of human dexterous manipulation demonstrations, but existing data is difficult to use for robot learning because object pose, geometry, and contact information are often missing or require pre-scanned object assets. We present EgoAERO, the first framework that learns dexterous manipulation from a single egocentric RGB-D human demonstration without object assets. EgoAERO reconstructs contact-consistent hand-object trajectories through asset-free object tracking and reconstruction, ego motion compensation, and adaptive contact optimization, then converts them into robot policies using two-stage residual learning. We further introduce an online quality assessment mechanism and construct EgoDex-R, a large-scale egocentric dataset with 4.3M RGB-D frames for dexterous policy learning. Simulation and real-world experiments show that EgoAERO enables single-demonstration dexterous manipulation and achieves downstream performance close to CAD-based reconstructions on HOI4D.
Robotics (cs.RO); Artificial Intelligence (cs.AI)
arXiv:2606.08057 [cs.RO]
(or arXiv:2606.08057v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2606.08057 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 149 cuvinte
Subiecte conexe
Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy — Jason Ma, Dyna Robotics
Latent Space · 13 oct. 2026
NVIDIA GTC Taipei at COMPUTEX: Live Updates on What’s Next in AI
NVIDIA · 21 mai 2026
RoboReact: Agentic Skill Distillation from Generated Egocentric Videos for Generalizable Whole-Body Manipulation
arXiv · 5 aug. 2026
RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation
arXiv · 30 sept. 2026