analiza aiqbench
The paper is a controlled comparison of what a robot policy should predict when trained on in-the-wild egocentric human video. Holding the architecture and data mix fixed, it finds that predicting DINO features or 3D motion flow transfers better than predicting raw pixels, with reported gains of up to 4x on out-of-distribution generalization and 20-30% in-domain. These are results from three real-world bimanual tasks, so readers should treat them as promising evidence rather than settled findings, but the design isolates the variable well.
EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data
arXiv:2607.08436v2 Announce Type: replace-cross Abstract: Egocentric human data offers scalable supervision for robot manipulation. However, behavior cloning entangles transferable content like objects, scenes, and task semantics, with non-transferable factors like human morphology, head motion, and behavioral style. We study whether World Action Models (WAMs) provide a better training signal by requiring policies to predict not only actions, but also how the scene evolves. The central question is what world representation best enables human-to-robot transfer. We hypothesize that an effective world target should abstract appearance, capture agent-invariant physical effects, and separate camera motion from environment change. We introduce EgoWAM, a controlled human-robot co-training framework that fixes the policy backbone, action head, and data mixture while varying only the world prediction target, comparing Pixel, DINO, and 3D motion flow. Across three real-world bimanual tasks, WAM co-training scales more effectively with in-the-wild egocentric human data than behavior cloning. Pixel-based prediction transfers weakly, while DINO and 3D flow yield substantial gains: DINO improves out-of-distribution object and scene generalization by up to 4x, and 3D flow improves in-domain performance by 20-30%. More details: https://gatech-rl2.github.io/egowam.github.io
Abstract:Egocentric human data offers scalable supervision for robot manipulation. However, behavior cloning entangles transferable content like objects, scenes, and task semantics, with non-transferable factors like human morphology, head motion, and behavioral style. We study whether World Action Models (WAMs) provide a better training signal by requiring policies to predict not only actions, but also how the scene evolves. The central question is what world representation best enables human-to-robot transfer. We hypothesize that an effective world target should abstract appearance, capture agent-invariant physical effects, and separate camera motion from environment change. We introduce EgoWAM, a controlled human-robot co-training framework that fixes the policy backbone, action head, and data mixture while varying only the world prediction target, comparing Pixel, DINO, and 3D motion flow. Across three real-world bimanual tasks, WAM co-training scales more effectively with in-the-wild egocentric human data than behavior cloning. Pixel-based prediction transfers weakly, while DINO and 3D flow yield substantial gains: DINO improves out-of-distribution object and scene generalization by up to 4x, and 3D flow improves in-domain performance by 20-30%. More details: this https URL
Robotics (cs.RO); Artificial Intelligence (cs.AI)
arXiv:2607.08436 [cs.RO]
(or arXiv:2607.08436v2 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2607.08436 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 270 cuvinte
Subiecte conexe
Game Design Creativity evals
LessWrong · 12 oct. 2026
Echoverse: Deep, evolving environments for computer-use agents
Microsoft Research · 30 iul. 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · 25 sept. 2026
World Models and Yann LeCun’s Bet Against Large Language Models
Quantum Zeitgeist · 17 iul. 2026