aiqbench analysis
GeoLAM advances robotics by extracting geometry-grounded latent actions from unlabeled human videos, effectively disentangling manipulation motion from camera artifacts and appearance changes. This framework enables robots to learn transferable skills from passive observation without requiring explicit pose or trajectory annotations, bridging the gap between video understanding and physical execution. The approach demonstrates that structural priors derived from 4D geometry can significantly enhance robotic manipulation performance in complex environments.
GeoLAM: Learning Geometry-Grounded Latent Actions from Unlabeled Human Videos
arXiv:2609.17099v1 Announce Type: cross Abstract: Human videos provide rich manipulation experience, but extracting action representations that preserve useful motion remains challenging. Visual reconstruction alone can entangle manipulation-related motion with appearance changes and camera movement. We present GeoLAM, a framework for learning geometry-grounded latent actions from action-free human videos. GeoLAM combines future-frame reconstruction through a frozen geometric feature hierarchy with motion supervision from a training-only 4D geometry teacher. The geometric representation provides a structural prior, while the teacher's predictions yield spatially pooled targets capturing 3D displacement, residual image-plane motion, and surface-orientation changes. Visibility and confidence weighting reduces the contribution of unreliable estimates, encouraging continuous latent actions to retain geometric motion without explicit hand-pose or hand-trajectory annotations. After video pretraining without action labels, the learned representation provides transition targets for a world-action model trained on action-labeled robot demonstrations. The model jointly denoises latent actions and executable action chunks, with future-video prediction used only as an auxiliary training task. Deployment therefore requires neither the geometry teacher nor future-video generation. Evaluations on a latent-action benchmark and robotic manipulation tasks demonstrate the strong performance of GeoLAM.
Abstract:Human videos provide rich manipulation experience, but extracting action representations that preserve useful motion remains challenging. Visual reconstruction alone can entangle manipulation-related motion with appearance changes and camera movement. We present GeoLAM, a framework for learning geometry-grounded latent actions from action-free human videos. GeoLAM combines future-frame reconstruction through a frozen geometric feature hierarchy with motion supervision from a training-only 4D geometry teacher. The geometric representation provides a structural prior, while the teacher's predictions yield spatially pooled targets capturing 3D displacement, residual image-plane motion, and surface-orientation changes. Visibility and confidence weighting reduces the contribution of unreliable estimates, encouraging continuous latent actions to retain geometric motion without explicit hand-pose or hand-trajectory annotations. After video pretraining without action labels, the learned representation provides transition targets for a world-action model trained on action-labeled robot demonstrations. The model jointly denoises latent actions and executable action chunks, with future-video prediction used only as an auxiliary training task. Deployment therefore requires neither the geometry teacher nor future-video generation. Evaluations on a latent-action benchmark and robotic manipulation tasks demonstrate the strong performance of GeoLAM.
Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
arXiv:2609.17099 [cs.CV]
(or arXiv:2609.17099v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2609.17099 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 275 words
Related coverage
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · Oct 8, 2026
SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space
arXiv · Aug 4, 2026
CLAM: Continuous Latent Action Models for Robot Learning from Unlabeled Demonstrations
arXiv · Jul 31, 2026
GeoWorldAD: Geometry World Action Model for Autonomous Driving
arXiv · Jul 21, 2026