aiqbench analysis
KeyGen advances robotic manipulation by learning canonicalized 3D keypoints from point clouds, enabling policies to generalize across unseen object instances without overfitting to specific geometries. By conditioning a visuomotor diffusion model on these structured representations, the framework achieves robust performance in both simulation and real-world settings under varying poses and scales. This approach addresses a critical bottleneck in embodied AI, allowing systems to handle diverse object categories more effectively than traditional behavior cloning methods.
KeyGen: Unsupervised Keypoint based Object-Centric Representations for Category-Level Policy Generalization
arXiv:2609.28818v1 Announce Type: new Abstract: Generalization in robotic manipulation requires policies to perform tasks across diverse unseen object instances that vary in shape, size, and pose. However, conventional behavior cloning (BC) methods often overfit to instance-specific geometry and appearance, limiting transfer to novel objects. We introduce KeyGen, a framework that learns canonicalized semantic 3D keypoints from point clouds and uses them as structured object-centric representations for policy learning. A visuomotor diffusion policy conditions on these keypoints together with object-centric geometry to predict full manipulation trajectories, enabling consistent geometric correspondence across object instances. To evaluate category-level generalization, we construct a photorealistic simulation benchmark with three manipulation tasks and a planning-driven data generation pipeline that produces expert trajectories across diverse object instances. Experiments show that KeyGen significantly outperforms prior methods on both seen and unseen objects under pose variation, scales effectively with additional demonstrations per object, maintains robustness to object rescaling, and achieves strong performance in both simulation and real-world manipulation.
Abstract:Generalization in robotic manipulation requires policies to perform tasks across diverse unseen object instances that vary in shape, size, and pose. However, conventional behavior cloning (BC) methods often overfit to instance-specific geometry and appearance, limiting transfer to novel objects. We introduce KeyGen, a framework that learns canonicalized semantic 3D keypoints from point clouds and uses them as structured object-centric representations for policy learning. A visuomotor diffusion policy conditions on these keypoints together with object-centric geometry to predict full manipulation trajectories, enabling consistent geometric correspondence across object instances. To evaluate category-level generalization, we construct a photorealistic simulation benchmark with three manipulation tasks and a planning-driven data generation pipeline that produces expert trajectories across diverse object instances. Experiments show that KeyGen significantly outperforms prior methods on both seen and unseen objects under pose variation, scales effectively with additional demonstrations per object, maintains robustness to object rescaling, and achieves strong performance in both simulation and real-world manipulation.
Robotics (cs.RO); Artificial Intelligence (cs.AI)
arXiv:2609.28818 [cs.RO]
(or arXiv:2609.28818v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2609.28818 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 250 words
Related coverage
Sparse Meets Dense: Correspondence Guided Robotic Manipulation with Rigid-Deformable Interactions
arXiv · Aug 4, 2026
Action-Slot: Structured Action-Centric Representation Learning for Multi-Agent Atomic Activity Understanding
arXiv · Sep 22, 2026
PIRAMID: Progress and Plans
LessWrong · Jul 27, 2026
Deconfusing Corrigibility
AI Alignment Forum · Oct 17, 2026