analiza aiqbench
The paper proposes a learned spatiotemporal map of the environment and robot body, updated online from camera and proprioceptive data, and feeds its tokens into a vision-language-action policy. The authors report gains over image-only baselines on BEHAVIOR-1K, including faster subgoal completion and better robustness to scene changes. These are simulation benchmark results, so the practical value depends on how well the approach transfers to real robots and to environments outside the benchmark.
SERF: Spatiotemporal Environment and Robot Feature Map for Long-Horizon Mobile Manipulation
arXiv:2606.12956v2 Announce Type: replace Abstract: Long-horizon robot mobile manipulation requires continual reasoning about localization, environment changes, and task progress, all of which are challenging to infer from image observations alone. In this paper, we show that conditioning a mobile manipulation policy on a spatiotemporal feature map improves reasoning over long horizons. The map represents the environment and the articulated robot body as neural points in a shared latent space and is updated online from egocentric observations and proprioceptive state. We update the environment neural points using object-level rigid tracking and the robot neural points using forward kinematics. We use our spatiotemporal environment and robot feature (SERF) map as a state input to a vision-language-action (VLA) model by extracting map tokens from multiple reference frames and spatial scales, providing the policy with both local and global context. We demonstrate SERF on BEHAVIOR-1K, a benchmark for long-horizon mobile manipulation in household environments. Experiments show that the SERF VLA policy outperforms image-only baselines, reaches subgoals faster by following more direct trajectories, improves robustness to scene-configuration shifts, and recovers from object-drop failures.
Abstract:Long-horizon robot mobile manipulation requires continual reasoning about localization, environment changes, and task progress, all of which are challenging to infer from image observations alone. In this paper, we show that conditioning a mobile manipulation policy on a spatiotemporal feature map improves reasoning over long horizons. The map represents the environment and the articulated robot body as neural points in a shared latent space and is updated online from egocentric observations and proprioceptive state. We update the environment neural points using object-level rigid tracking and the robot neural points using forward kinematics. We use our spatiotemporal environment and robot feature (SERF) map as a state input to a vision-language-action (VLA) model by extracting map tokens from multiple reference frames and spatial scales, providing the policy with both local and global context. We demonstrate SERF on BEHAVIOR-1K, a benchmark for long-horizon mobile manipulation in household environments. Experiments show that the SERF VLA policy outperforms image-only baselines, reaches subgoals faster by following more direct trajectories, improves robustness to scene-configuration shifts, and recovers from object-drop failures.
Robotics (cs.RO)
arXiv:2606.12956 [cs.RO]
(or arXiv:2606.12956v2 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2606.12956 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 264 cuvinte
Subiecte conexe
SERF: Spatiotemporal Environment and Robot Feature Map for Long-Horizon Mobile Manipulation
arXiv · 12 iun. 2026
Worlds in One Demo: A Synthetic Data Engine for Learning Open-World Mobile Manipulation
arXiv · 20 iul. 2026
From Imitation to Alignment: Human-Preference Flow Policies for Long-Horizon Sidewalk Navigation
arXiv · 12 iun. 2026
Socially Consistent Multi-Robot Navigation Using Decoupled Planning and Trajectory Coordination
arXiv · 24 iul. 2026