PerFACT: Motion Policy with LLM-Powered Dataset Synthesis and Fusion Action-Chunking Transformers

Abstract:Deep learning methods have significantly enhanced motion planning for robotic manipulators by leveraging prior experiences within planning datasets. However, state-of-the-art neural motion planners are primarily trained on small datasets collected in manually generated workspaces, limiting their deployment in various everyday scenarios. Additionally, these planners often rely on monolithic network architectures that struggle to encode critical planning information. To address these challenges, we introduce Motion Policy with Dataset Synthesis powered by large language models (LLMs) and Fusion Action-Chunking Transformers (PerFACT), which incorporates two key components. Firstly, a novel workspace generation method, PerFACT, enables large-scale planning data collection by leveraging procedural primitive generation, and LLM-powered primitive suggestion and placement. Secondly, we introduce Fusion Motion Policy Networks (M$\pi$NetsFusion), an end-to-end, open-loop neural motion planner that uses a fusion action-chunking transformer to better encode planning signals and attend to multiple feature modalities. Leveraging PerFACT, we collect a dataset of 3.5M trajectories to train and evaluate M$\pi$NetsFusion against state-of-the-art planners. Results show that M$\pi$NetsFusion achieves consistently low planning time with sub-second inference, while maintaining competitive performance compared to both sampling-based and end-to-end neural benchmark planners. Project website: \href{this https URL}{this https URL}

Robotics (cs.RO); Systems and Control (eess.SY)

arXiv:2512.03444 [cs.RO]

(or arXiv:2512.03444v2 [cs.RO] for this version)

https://doi.org/10.48550/arXiv.2512.03444 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.