AgentMV: A State-Guided Multi-Agent Framework for Budget-Aware Music Video Generation

Abstract:Generating a complete music video from a song requires more than synthesizing visually plausible clips for individual lyric prompts. A practical system must maintain long-range visual consistency, coordinate recurring motifs, synchronize edits with musical structure, and manage the cumulative cost of video generation. Existing approaches typically generate segments independently or adopt fixed generation strategies, limiting their ability to perform global planning over an entire song. We present AgentMV, a state-guided multi-agent framework for budget-aware music video generation. AgentMV decomposes the production process into specialized agents for music perception, script planning, visual asset provision, segment realization, and final assembly, coordinated through a Structured Persistent State that enables information exchange and state tracking throughout the generation process. To optimize generation resources, we formulate motif-aware segment realization as a group-level Multiple-Choice Knapsack Problem solved via dynamic programming, considering segment importance, generation quality, cost, and motif reuse under a global budget constraint. Experiments on song benchmarks demonstrate that AgentMV improves quality-cost trade-offs over existing MV generation frameworks, highlighting the potential of state-guided multi-agent coordination and budget-aware planning for long-form music video generation with improved quality and efficiency.

Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multiagent Systems (cs.MA)

arXiv:2605.10723 [cs.CV]

(or arXiv:2605.10723v2 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2605.10723 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.