aiqbench analysis
This research demonstrates that diffusion policy backbones for robotic manipulation are significantly over-parameterized compared to image synthesis tasks, allowing a frozen 5M-parameter model to match larger architectures when task adaptation occurs solely through conditioning pathways. The findings suggest that architectures inherited from vision and language domains may be a poor inductive bias for low-dimensional action targets, offering a more efficient training recipe for Vision-Language-Action models.
Freeze, Share, Shrink: Rethinking the Action Backbone in Diffusion Policies
arXiv:2511.12101v3 Announce Type: replace-cross Abstract: Many recent Vision-Language-Action models employ diffusion or flow-matching backbones with hundreds of millions of parameters for action generation. However, unlike image synthesis where the output spans millions of diverse pixels, a manipulation policy generates only short sequences of low-dimensional, physically correlated action values, a far simpler target that may not require such capacity. We confirm this intuition and show that, in modulation-conditioned diffusion policies, task adaptation can be routed entirely through the conditioning pathway, leaving a frozen, observation-free backbone that serves as a reusable trajectory prior. To establish this, we introduce a decoupled training recipe: a general-purpose action head is first pretrained on observation-free forward-kinematics data, then frozen while only the conditioning pathway is trained for downstream tasks. Using Diffusion Policy as a testbed, we show that on both MimicGen and LIBERO, a single frozen backbone shared across all tasks matches normally trained counterparts. In our ablations, this succeeds with modulation-based conditioning, while attention-based conditioning is embedded in the backbone's own weights and collapses once they are frozen. Ablations show the pretraining signal (joint positions, end-effector poses, or none) has little effect, while a randomly initialized backbone fails entirely, indicating that pretraining need only supply a general trajectory prior. Finally, a 5M-parameter MLP backbone matches or exceeds both the 244M U-Net and the transformer on these benchmarks, suggesting the action backbone is over-parameterized and that architectures inherited from image and language generation are a poor inductive-bias fit for the low-dimensional action target, with implications for action-backbone design in VLA models and other policies with an action backbone.
Abstract:Many recent Vision-Language-Action models employ diffusion or flow-matching backbones with hundreds of millions of parameters for action generation. However, unlike image synthesis where the output spans millions of diverse pixels, a manipulation policy generates only short sequences of low-dimensional, physically correlated action values, a far simpler target that may not require such capacity. We confirm this intuition and show that, in modulation-conditioned diffusion policies, task adaptation can be routed entirely through the conditioning pathway, leaving a frozen, observation-free backbone that serves as a reusable trajectory prior. To establish this, we introduce a decoupled training recipe: a general-purpose action head is first pretrained on observation-free forward-kinematics data, then frozen while only the conditioning pathway is trained for downstream tasks. Using Diffusion Policy as a testbed, we show that on both MimicGen and LIBERO, a single frozen backbone shared across all tasks matches normally trained counterparts. In our ablations, this succeeds with modulation-based conditioning, while attention-based conditioning is embedded in the backbone's own weights and collapses once they are frozen. Ablations show the pretraining signal (joint positions, end-effector poses, or none) has little effect, while a randomly initialized backbone fails entirely, indicating that pretraining need only supply a general trajectory prior. Finally, a 5M-parameter MLP backbone matches or exceeds both the 244M U-Net and the transformer on these benchmarks, suggesting the action backbone is over-parameterized and that architectures inherited from image and language generation are a poor inductive-bias fit for the low-dimensional action target, with implications for action-backbone design in VLA models and other policies with an action backbone.
Robotics (cs.RO); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2511.12101 [cs.RO]
(or arXiv:2511.12101v3 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2511.12101 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 354 words
Related coverage
Build a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod
AWS · Sep 4, 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026
Little Venice, Little Sparta: The UAE Statecraft Meets The War On Iran - Analysis
eurasiareview.com · Sep 24, 2026
SUAVE: Unified Video-Action Models via Masked Diffusion
arXiv · Oct 6, 2026