analiza aiqbench
This paper addresses a critical bottleneck in Vision-Language-Action (VLA) models by proposing HARP, a framework that aligns human and robot visual representations to enable effective co-training from large-scale human video data. By bridging cross-embodiment discrepancies through paired demonstrations and unpaired dynamics supervision, the method achieves significant improvements in real-world manipulation success rates without relying on direct visual feature transfer. This work advances the state of generalizable robotic learning, moving closer to systems that can leverage vast human-generated content for robot policy synthesis.
HARP-VLA: Human-Robot Aligned Representation Learning for Vision-Language-Action Model
arXiv:2605.31234v2 Announce Type: replace Abstract: Learning generalizable vision-language-action (VLA) models from large-scale human videos is promising but challenging due to cross-embodiment discrepancies in both visual observations and executable actions. While latent action models reduce the action execution gap by learning action abstractions, they still rely on visual features. Thus, misaligned human and robot visual representations can lead to inconsistencies in policy inputs and induce domain-dependent latent actions, hindering effective co-training with human videos.To address this, we propose HARP, a human-robot aligned representation learning framework for more effective VLA pretraining from human videos. Specifically, HARP uses paired human-robot demonstrations as cross-embodiment bridges and abundant unpaired human and robot videos as a scalable dynamics supervision data source. It trains a robot-adapted visual encoder and a latent action model with manipulation-centric auxiliary cues and a source-relative pair-discriminative alignment loss, which adapts robot representations toward human semantics while preserving pair-level discrimination. The learned aligned vision encoder and latent action model provide a unified vision and action representation for VLA-style policy learning, where human and robot videos provide vision-language-to-latent-action supervision and a lightweight robot action head grounds latent actions into executable commands. Experiments on feature visualization, simulation, and real-world manipulation show improved human-robot alignment and downstream policy performance, achieving 4.481 average length on CALVIN ABC$\rightarrow$D and a 7.1 percentage-point improvement in real-world success rate over the strongest evaluated baseline. Code, videos, and demos are available at https://puzhenyuan.github.io/HARP-VLA-website/.
Abstract:Learning generalizable vision-language-action (VLA) models from large-scale human videos is promising but challenging due to cross-embodiment discrepancies in both visual observations and executable actions. While latent action models reduce the action execution gap by learning action abstractions, they still rely on visual features. Thus, misaligned human and robot visual representations can lead to inconsistencies in policy inputs and induce domain-dependent latent actions, hindering effective co-training with human this http URL address this, we propose HARP, a human-robot aligned representation learning framework for more effective VLA pretraining from human videos. Specifically, HARP uses paired human-robot demonstrations as cross-embodiment bridges and abundant unpaired human and robot videos as a scalable dynamics supervision data source. It trains a robot-adapted visual encoder and a latent action model with manipulation-centric auxiliary cues and a source-relative pair-discriminative alignment loss, which adapts robot representations toward human semantics while preserving pair-level discrimination. The learned aligned vision encoder and latent action model provide a unified vision and action representation for VLA-style policy learning, where human and robot videos provide vision-language-to-latent-action supervision and a lightweight robot action head grounds latent actions into executable commands. Experiments on feature visualization, simulation, and real-world manipulation show improved human-robot alignment and downstream policy performance, achieving 4.481 average length on CALVIN ABC$\rightarrow$D and a 7.1 percentage-point improvement in real-world success rate over the strongest evaluated baseline. Code, videos, and demos are available at this https URL.
Robotics (cs.RO)
arXiv:2605.31234 [cs.RO]
(or arXiv:2605.31234v2 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2605.31234 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 322 cuvinte
Subiecte conexe
LARA: Latent Action Representation Alignment for Vision-Language-Action Models
arXiv · 8 iun. 2026
Building a Foundation Stack for General-Purpose Robots
IEEE Spectrum · 13 iul. 2026
Links #5: 2026/07
LessWrong · 31 iul. 2026
UMI-Bridge: Action-Anchored Latent Alignment across Human and Robot Manipulation Data
arXiv · 17 sept. 2026