aiqbench analysis
This research addresses the critical bottleneck of sample inefficiency in Multi-Agent Reinforcement Learning by introducing a GPU-accelerated simulation environment that achieves a 30,000x speedup over standard tools like Gazebo. The proposed Transformer-based architecture (TransfMAPPO) successfully enables scalable fleet training with policies invariant to fleet size, allowing for effective curriculum learning on complex underwater tracking tasks. Ultimately, the study demonstrates a viable path to closing the sim-to-real gap for autonomous vehicle fleets, maintaining tracking errors below 5m even with multiple fast-moving targets.
Scaling Multi Agent Reinforcement Learning for Underwater Acoustic Tracking via Autonomous Vehicles
arXiv:2505.08222v3 Announce Type: replace-cross Abstract: Autonomous vehicles (AVs) offer a cost-effective solution for scientific missions such as underwater tracking. Reinforcement learning (RL) has emerged as a powerful method for controlling AVs, but scaling to fleets (essential for multi-target tracking or rapidly moving targets) is challenging. Multi-Agent RL (MARL) is notoriously sample-inefficient, and while high-fidelity simulators like Gazebo's LRAUV provide up to 100x faster-than-real-time single-robot simulations, they offer little speedup in multi-vehicle scenarios, making MARL training impractical. Yet, high-fidelity simulation is crucial to test complex policies and close the sim-to-real gap. To address these limitations, we develop a GPU-accelerated environment that achieves up to 30,000x speedup over Gazebo while preserving its dynamics. This enables fast, end-to-end GPU training and seamless transfer to Gazebo for evaluation. We also introduce a Transformer-based architecture (TransfMAPPO) that learns policies invariant to fleet size and number of targets, enabling curriculum learning to train larger fleets on increasingly complex scenarios. After large-scale GPU training, we perform extensive evaluations in Gazebo, showing our method maintains tracking errors below 5m even with multiple fast-moving targets.
Abstract:Autonomous vehicles (AVs) offer a cost-effective solution for scientific missions such as underwater tracking. Reinforcement learning (RL) has emerged as a powerful method for controlling AVs, but scaling to fleets (essential for multi-target tracking or rapidly moving targets) is challenging. Multi-Agent RL (MARL) is notoriously sample-inefficient, and while high-fidelity simulators like Gazebo's LRAUV provide up to 100x faster-than-real-time single-robot simulations, they offer little speedup in multi-vehicle scenarios, making MARL training impractical. Yet, high-fidelity simulation is crucial to test complex policies and close the sim-to-real gap. To address these limitations, we develop a GPU-accelerated environment that achieves up to 30,000x speedup over Gazebo while preserving its dynamics. This enables fast, end-to-end GPU training and seamless transfer to Gazebo for evaluation. We also introduce a Transformer-based architecture (TransfMAPPO) that learns policies invariant to fleet size and number of targets, enabling curriculum learning to train larger fleets on increasingly complex scenarios. After large-scale GPU training, we perform extensive evaluations in Gazebo, showing our method maintains tracking errors below 5m even with multiple fast-moving targets.
Robotics (cs.RO); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC); Performance (cs.PF)
arXiv:2505.08222 [cs.RO]
(or arXiv:2505.08222v3 [cs.RO] for this version)
Full text under the publisher's open licence · 192 words
Related coverage
News
Quantum Computing Report · Jul 25, 2026
AI and Quantum Computing Briefing
livetradingnews.com · Aug 4, 2026
Mastering Agentic Techniques: AI Agent Reinforcement Learning
NVIDIA · Jul 1, 2026
Fine-tune NVIDIA Nemotron 3 models with Amazon SageMaker AI serverless model customization
AWS · Jul 10, 2026