analiza aiqbench
LIFD introduces a rectified-flow-based diffusion framework that enables robots to maintain persistent 3D scene memory despite partial observability, significantly improving manipulation success rates on standard benchmarks. By combining multi-view agreement with geometric supervision, the method allows policy learning from a single RGB camera while effectively inferring missing spatial information. This advancement addresses a critical gap in robotic autonomy where current views are insufficient for complex tasks involving moving objects or changing perspectives.
LIFD: Anchored Diffusion for 3D-Aware Scene Memory in Robotic Manipulation
arXiv:2609.19796v1 Announce Type: new Abstract: Robotic manipulation under partial observability requires spatial information that extends beyond the current view. Geometry-aware RGB features describe visible structure, but previously observed regions may disappear as the robot or scene moves. Maintaining a useful scene representation therefore requires retaining observation history while inferring missing content without losing its connection to visible evidence. We introduce LIFD (Look, Imagine, Focus, and Do), a framework for persistent, 3D-aware scene memory. LIFD learns a scene-token representation from multi-view agreement and completes it from a single RGB view and recurrent memory. A rectified-flow model generates the tokens while Anchor-Guided Cross-Attention conditions completion on current geometric features. Compact slot features connect this representation to a manipulation policy. Multi-view and geometric supervision are used during representation learning; deployment requires one RGB camera, proprioception, and a task instruction. LIFD (Staged) reaches 91.6% average success on LIBERO and 79.8% on MetaWorld, improving LIBERO average success by 3.1 percentage points over Joint training. On four UR5e task families with ten demonstrations per family, it achieves 56.0% mean success, compared with 40.5% for OpenVLA-7B.
Abstract:Robotic manipulation under partial observability requires spatial information that extends beyond the current view. Geometry-aware RGB features describe visible structure, but previously observed regions may disappear as the robot or scene moves. Maintaining a useful scene representation therefore requires retaining observation history while inferring missing content without losing its connection to visible evidence. We introduce LIFD (Look, Imagine, Focus, and Do), a framework for persistent, 3D-aware scene memory. LIFD learns a scene-token representation from multi-view agreement and completes it from a single RGB view and recurrent memory. A rectified-flow model generates the tokens while Anchor-Guided Cross-Attention conditions completion on current geometric features. Compact slot features connect this representation to a manipulation policy. Multi-view and geometric supervision are used during representation learning; deployment requires one RGB camera, proprioception, and a task instruction. LIFD (Staged) reaches 91.6% average success on LIBERO and 79.8% on MetaWorld, improving LIBERO average success by 3.1 percentage points over Joint training. On four UR5e task families with ten demonstrations per family, it achieves 56.0% mean success, compared with 40.5% for OpenVLA-7B.
8 pages, 4 figures. Submitted to ICRA 2027
Robotics (cs.RO)
arXiv:2609.19796 [cs.RO]
(or arXiv:2609.19796v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2609.19796 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 275 cuvinte
Subiecte conexe
Action-Effect Memory Pretraining for Robot Manipulation
arXiv · 12 iun. 2026
Mem-World: Memory-Augmented Action-Conditioned World Models for Persistent Robot Manipulation
arXiv · 16 sept. 2026
Action-Effect Memory Pretraining for Robot Manipulation
arXiv · 19 aug. 2026
Build a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod
AWS · 4 sept. 2026