A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AI

Abstract:Navigation and manipulation are core capabilities in Embodied AI, but training agents to perform them directly in the real world is costly, time-consuming, and unsafe. Therefore, sim-to-real transfer has emerged as a key approach, yet the sim-to-real gap persists. This survey examines how physics simulators address this gap by analyzing properties that have received limited attention in prior surveys. We also analyze their features for navigation and manipulation tasks, as well as their hardware requirements. Additionally, we offer a resource with benchmark datasets, metrics, simulation platforms, and methods to help researchers select suitable tools while accounting for hardware constraints.

35 pages, 9 figures. Accepted to ACM Computing Surveys (CSUR)

Robotics (cs.RO); Artificial Intelligence (cs.AI)

arXiv:2505.01458 [cs.RO]

(or arXiv:2505.01458v3 [cs.RO] for this version)

https://doi.org/10.48550/arXiv.2505.01458 Focus to learn more arXiv-issued DOI via DataCite

https://doi.org/10.1145/3856802 Focus to learn more DOI(s) linking to related resources

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.