aiqbench analysis
This arXiv paper proposes an open-source metric that scores individual AI-generated music clips by training lightweight prediction heads on frozen MuQ-310M features, filling a gap left by distributional metrics like Fréchet Audio Distance and by closed-source learned scorers. The reported correlations with human ratings are strong on the MusicEval benchmark, but they come from one dataset with expert ratings, so generalization to other systems and listener populations remains to be shown. For readers, the most useful findings are that encoder choice matters more than architecture and that LoRA adaptation on about 150 clips can yield usable personalized evaluators.
MuQ-Eval: An Open-Source Per-Sample Quality Metric for AI Music Generation Evaluation
arXiv:2603.22677v2 Announce Type: replace Abstract: Distributional metrics such as Fr\'echet Audio Distance cannot score individual music clips and correlate poorly with human judgments, while the only per-sample learned metric achieving high human correlation is closed-source. We introduce MUQ-EVAL, an open-source per-sample quality metric for AIgenerated music built by training lightweight prediction heads on frozen MuQ-310M features using MusicEval, a dataset of generated clips from 31 text-to-music systems with expert quality ratings. Our simplest model, frozen features with attention pooling and a two-layer MLP, achieves system-level SRCC = 0.957 and utterance-level SRCC = 0.838 with human mean opinion scores. A systematic ablation over training objectives and adaptation strategies shows that no addition meaningfully improves the frozen baseline, indicating that frozen MuQ representations already capture quality-relevant information. Encoder choice is the dominant design factor, outweighing all architectural and training decisions. LoRA-adapted models trained on as few as 150 clips already achieve usable correlation, enabling personalized quality evaluators from individual listener annotations. A controlled degradation analysis reveals selective sensitivity to signal-level artifacts but insensitivity to musical-structural distortions. Our metric, MUQ-EVAL, is fully open-source, outperforms existing open per-sample metrics, and runs in real time on a single consumer GPU. Code, model weights, and evaluation scripts are available at https://github.com/dgtql/MuQ-Eval.
Abstract:Distributional metrics such as Fréchet Audio Distance cannot score individual music clips and correlate poorly with human judgments, while the only per-sample learned metric achieving high human correlation is closed-source. We introduce MUQ-EVAL, an open-source per-sample quality metric for AIgenerated music built by training lightweight prediction heads on frozen MuQ-310M features using MusicEval, a dataset of generated clips from 31 text-to-music systems with expert quality ratings. Our simplest model, frozen features with attention pooling and a two-layer MLP, achieves system-level SRCC = 0.957 and utterance-level SRCC = 0.838 with human mean opinion scores. A systematic ablation over training objectives and adaptation strategies shows that no addition meaningfully improves the frozen baseline, indicating that frozen MuQ representations already capture quality-relevant information. Encoder choice is the dominant design factor, outweighing all architectural and training decisions. LoRA-adapted models trained on as few as 150 clips already achieve usable correlation, enabling personalized quality evaluators from individual listener annotations. A controlled degradation analysis reveals selective sensitivity to signal-level artifacts but insensitivity to musical-structural distortions. Our metric, MUQ-EVAL, is fully open-source, outperforms existing open per-sample metrics, and runs in real time on a single consumer GPU. Code, model weights, and evaluation scripts are available at this https URL.
Artificial Intelligence (cs.AI); Sound (cs.SD)
I.2.6; J.5; H.5.5
arXiv:2603.22677 [cs.AI]
(or arXiv:2603.22677v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2603.22677 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 299 words
Related coverage
KVAE: Family of Tokenizers for Multimodal Generative Models
arXiv · Aug 7, 2026
ArguLens: An Open-Source System for Automated Essay Scoring and Label-Aware Feedback Generation
arXiv · Aug 19, 2026
LOADBench: A Large Open Analog Dataset and Benchmark for Machine Learning in IC Design
arXiv · Oct 17, 2026
Auditor-in-a-Box: Tools for Third-Party Auditing
LessWrong · Jul 28, 2026