analiza aiqbench
TLC-DiT introduces a plug-in architecture for Diffusion Transformer robot policies that explicitly injects task-aligned local visual features, significantly improving robustness against environmental perturbations. By modulating frozen vision encoder outputs without altering the core diffusion objective, this approach enhances both success rates in benchmark tasks and interpretability through direct visualization of attended regions. The method demonstrates clear efficacy in real-world bimanual manipulation, offering a practical pathway to make language-conditioned robot policies more reliable and inspectable.
TLC-DiT: Task-Aligned Local Visual Conditioning for Robust Multitask Robot Manipulation
arXiv:2609.34297v1 Announce Type: new Abstract: Language-conditioned robot policies have made clear progress in multitask manipulation, but task-relevant local visual evidence usually stays hidden inside a visual backbone or attention layers. This leaves the policy difficult to inspect and fragile under visual change, two symptoms of a missing explicit, task-aligned local visual channel. We present TLC-DiT, a plug-in extension of the Multitask Diffusion Transformer (DiT) policy that adds explicit task-guided local visual feature maps without changing the diffusion objective or the action-generation process. For each camera view, frozen DINOv2 patch features are modulated by the CLIP task embedding through FiLM and refined by a lightweight CoordConv CNN adapter into smooth spatial maps, which are concatenated with the original global image, language, joint-state, and timestep conditions. On LIBERO, TLC-DiT reaches a 93.5% average success rate, compared with 86.5% for Multitask DiT and 79.25% for SmolVLA. On LIBERO-plus, the total success rate improves from 54.07% to 57.24%, with larger gains under camera, background, and sensor-noise changes. In real-world bimanual tasks, TLC-DiT raises Teabag Putting completion from 44% to 89% while maintaining comparable Match Box Opening performance. Feature-map visualizations confirm that the model attends to task-relevant regions across views and perturbations, providing a direct way to inspect the visual evidence.
Abstract:Language-conditioned robot policies have made clear progress in multitask manipulation, but task-relevant local visual evidence usually stays hidden inside a visual backbone or attention layers. This leaves the policy difficult to inspect and fragile under visual change, two symptoms of a missing explicit, task-aligned local visual channel. We present TLC-DiT, a plug-in extension of the Multitask Diffusion Transformer (DiT) policy that adds explicit task-guided local visual feature maps without changing the diffusion objective or the action-generation process. For each camera view, frozen DINOv2 patch features are modulated by the CLIP task embedding through FiLM and refined by a lightweight CoordConv CNN adapter into smooth spatial maps, which are concatenated with the original global image, language, joint-state, and timestep conditions. On LIBERO, TLC-DiT reaches a 93.5% average success rate, compared with 86.5% for Multitask DiT and 79.25% for SmolVLA. On LIBERO-plus, the total success rate improves from 54.07% to 57.24%, with larger gains under camera, background, and sensor-noise changes. In real-world bimanual tasks, TLC-DiT raises Teabag Putting completion from 44% to 89% while maintaining comparable Match Box Opening performance. Feature-map visualizations confirm that the model attends to task-relevant regions across views and perturbations, providing a direct way to inspect the visual evidence.
Robotics (cs.RO)
arXiv:2609.34297 [cs.RO]
(or arXiv:2609.34297v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2609.34297 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 294 cuvinte
Subiecte conexe
How to Evaluate General-Purpose Robot Policies for Real-World Deployment
NVIDIA · 12 iul. 2026
CoRE: Learning Collaboration-Role Experts for Decentralized Collaborative Manipulation with One Policy
arXiv · 7 oct. 2026
News
Quantum Computing Report · 7 iun. 2026
Humanity’s Last Invention — Richard Socher of Recursive
Latent Space · 14 sept. 2026