analiza aiqbench
This research introduces the ISS Policy, a 3D visuomotor diffusion model that enhances robotic manipulation by incorporating implicit scene supervision directly from expert demonstrations. By leveraging a DiT backbone to predict how actions alter the surrounding geometry, the framework achieves state-of-the-art performance in both simulated and real-world dual-arm tasks without requiring additional inference-time annotations. This advancement represents a significant step forward in scaling vision-based imitation learning for complex robotic control through improved geometric understanding.
ISS Policy: Scalable Diffusion Policy with Implicit Scene Supervision
arXiv:2512.15020v3 Announce Type: replace Abstract: Vision-based imitation learning has enabled impressive robotic manipulation skills, but action imitation alone provides limited supervision of the geometric consequences of robot behavior. To address this limitation, we introduce **Implicit Scene Supervision (ISS) Policy**, a 3D visuomotor diffusion policy with a DiT backbone that predicts continuous action sequences from point-cloud observations. ISS augments action diffusion with a supervised robot motion predictor that maps generated actions and robot-state context to end-effector motion, and then uses the predicted motion together with gripper intent to forecast future point-cloud representations. By explicitly modeling the intermediate transition from action to robot motion, ISS encourages the policy to capture how its actions affect the surrounding 3D scene. We further introduce asymmetric gradient routing to separate direct motion regression from scene-level policy supervision, together with a change-balanced objective that accounts for variations in scene-change magnitude. These auxiliary objectives provide dynamics-aware geometric supervision using only expert demonstrations, without requiring additional annotations or auxiliary modules at inference time. ISS Policy achieves state-of-the-art performance on single-arm manipulation tasks in MetaWorld and dexterous manipulation tasks in Adroit, while real-world dual-arm experiments further demonstrate its effectiveness on physical robotic manipulation. The resulting framework preserves the scalable DiT backbone and standard diffusion-policy control interface. Code and videos will be released.
Abstract:Vision-based imitation learning has enabled impressive robotic manipulation skills, but action imitation alone provides limited supervision of the geometric consequences of robot behavior. To address this limitation, we introduce **Implicit Scene Supervision (ISS) Policy**, a 3D visuomotor diffusion policy with a DiT backbone that predicts continuous action sequences from point-cloud observations. ISS augments action diffusion with a supervised robot motion predictor that maps generated actions and robot-state context to end-effector motion, and then uses the predicted motion together with gripper intent to forecast future point-cloud representations. By explicitly modeling the intermediate transition from action to robot motion, ISS encourages the policy to capture how its actions affect the surrounding 3D scene. We further introduce asymmetric gradient routing to separate direct motion regression from scene-level policy supervision, together with a change-balanced objective that accounts for variations in scene-change magnitude. These auxiliary objectives provide dynamics-aware geometric supervision using only expert demonstrations, without requiring additional annotations or auxiliary modules at inference time. ISS Policy achieves state-of-the-art performance on single-arm manipulation tasks in MetaWorld and dexterous manipulation tasks in Adroit, while real-world dual-arm experiments further demonstrate its effectiveness on physical robotic manipulation. The resulting framework preserves the scalable DiT backbone and standard diffusion-policy control interface. Code and videos will be released.
Robotics (cs.RO)
arXiv:2512.15020 [cs.RO]
(or arXiv:2512.15020v3 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2512.15020 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 298 cuvinte
Subiecte conexe
Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding
arXiv · 20 iul. 2026
From Video to Control: A Survey of Learning Manipulation Interfaces from Temporal Visual Data
arXiv · 4 iun. 2026
From Video to Control: A Survey of Learning Manipulation Interfaces from Temporal Visual Data
arXiv · 3 iun. 2026
Triton-Based Optimization of Video Sparse Attention on ROCm
AMD · 13 iul. 2026