CoFlow: Coordinated Few-Step Flow for Offline Multi-Agent Decision Making

Abstract:How can generative offline multi-agent reinforcement learning achieve both fast joint trajectory generation and effective cooperation? Multi-agent diffusion models generate joint trajectories through iterative sampling but incur high inference latency. Distillation reduces sampling steps but may lose the teacher's cross-agent dependencies, reducing the success rate of cooperative tasks. A joint few-step generator can be trained directly to avoid losing these dependencies during distillation. However, insufficient modeling of cross-agent dependencies may still impair cooperation. In addition, its consistency training process introduces heavy computational and memory costs. Therefore, we propose Coordinated Few-Step Flow (CoFlow), which directly learns a joint averaged-velocity field for distillation-free one-step and few-step multi-agent trajectory generation. To support cooperation during generation, Coordinated Velocity Attention (CVA) incorporates teammate trajectory information into each agent's prediction. To reduce the consistency-training cost of this joint model, we approximate the correction term using finite differences. Across 48 configurations on MPE and SMAC, CoFlow supports centralized and decentralized execution. Under centralized execution, CoFlow outperforms our reproduced baseline by 11.2%, averaging relative gains equally across both benchmark suites, with a 12.93-fold model-sampling speedup. CVA improves normalized scores by 165.1% on average over the same model with cross-agent attention disabled. Compared with the exact-derivative implementation, finite-difference training achieves a 1.78-fold speedup and reduces peak GPU memory by 41.1%. Project page: this https URL.

27 pages. Substantially revised manuscript with updated methodology, experiments, theoretical analysis, figures, and references

Artificial Intelligence (cs.AI)

arXiv:2605.01457 [cs.AI]

(or arXiv:2605.01457v4 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2605.01457 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.