aiqbench analysis
This research addresses the critical challenge of sparse rewards in long-horizon robotic manipulation by introducing a Bayesian non-parametric skill prior based on Dirichlet Process Mixtures. By replacing fixed parametric assumptions with adaptive latent space modeling, the approach significantly improves exploration efficiency and task success rates across multiple benchmarks compared to standard reinforcement learning baselines. The method effectively enables robots to discover diverse, multi-modal skills without predefining component counts, offering a robust solution for complex, real-world automation tasks.
Robotic Long-Horizon Manipulation with Bayesian Non-parametric Skill Priors
arXiv:2503.21975v2 Announce Type: replace-cross Abstract: Long-horizon manipulation under sparse rewards remains challenging for reinforcement learning due to delayed feedback and inefficient exploration. Existing skill-based approaches often assume a fixed parametric prior (e.g., a single Gaussian), limiting their ability to capture diverse and multi-modal skill structures required for complex tasks. We propose a Bayesian non-parametric skill prior that models temporally extended skills in a structured latent space using a Dirichlet Process Mixture, enabling adaptive skill discovery without predefining the number of components. Integrated into a hierarchical RL framework, the learned prior guides high-level skill selection while a pretrained decoder generates temporally abstracted actions, improving exploration efficiency in sparse-reward settings. Experiments on Franka Kitchen, LIBERO-Long, Meta-World, and a real robot demonstrate consistent gains in long-horizon manipulation, achieving over 0.8 success rate within 1.5M steps, whereas SAC fails to converge even after 5M steps ($<$0.1). Compared to a single-Gaussian prior baseline, our model yields an average improvement of 21.8\%.
Abstract:Long-horizon manipulation under sparse rewards remains challenging for reinforcement learning due to delayed feedback and inefficient exploration. Existing skill-based approaches often assume a fixed parametric prior (e.g., a single Gaussian), limiting their ability to capture diverse and multi-modal skill structures required for complex tasks. We propose a Bayesian non-parametric skill prior that models temporally extended skills in a structured latent space using a Dirichlet Process Mixture, enabling adaptive skill discovery without predefining the number of components. Integrated into a hierarchical RL framework, the learned prior guides high-level skill selection while a pretrained decoder generates temporally abstracted actions, improving exploration efficiency in sparse-reward settings. Experiments on Franka Kitchen, LIBERO-Long, Meta-World, and a real robot demonstrate consistent gains in long-horizon manipulation, achieving over 0.8 success rate within 1.5M steps, whereas SAC fails to converge even after 5M steps ($<$0.1). Compared to a single-Gaussian prior baseline, our model yields an average improvement of 21.8\%.
Robotics (cs.RO); Artificial Intelligence (cs.AI)
arXiv:2503.21975 [cs.RO]
(or arXiv:2503.21975v2 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2503.21975 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 245 words
Related coverage
DROM: A Language-Guided Diffusion Framework for Multi-Skill Robotic Manipulation
arXiv · Sep 30, 2026
ManiUnit: A Manipulation Skill Dataset and Benchmark for Long-Horizon Tasks
arXiv · Oct 9, 2026
LabDex: A Hierarchical Benchmark for Dexterous Manipulation in Laboratories
arXiv · Aug 20, 2026
MoMaStage: Skill-State Graph Guided Planning and Closed-Loop Execution for Long-Horizon Indoor Mobile Manipulation
arXiv · Sep 4, 2026