aiqbench analysis
This article introduces MulTTiPop, a new dataset designed specifically to benchmark automatic music transcription models against multitrack MIDI recordings of pop songs spanning decades. By evaluating state-of-the-art machine learning systems on this data, the authors highlight significant performance gaps, particularly noting that current best models only achieve a 29% Onset F1 score. This work directly advances the field of audio signal processing and music AI by providing a rigorous standard for future model development in music transcription.
MulTTiPop: A Multitrack Transcription Dataset for Pop Music
arXiv:2607.08756v2 Announce Type: replace-cross Abstract: We present MulTTiPop, a dataset of pop music segments and their associated multitrack MIDI recordings for the evaluation of automatic music transcription models. MulTTiPop contains 572 segments of popular music totaling 3.5 hours of audio, and contains songs from diverse genres and decades from the 1930s to 2000s. To collect this dataset, we perform metadata-based matching on song segments from the Lakh MIDI and TheoryTab datasets, manually identify an anchor beat between the audio and MIDI, then use beat tracking on the audio and warp the MIDI to match its tempo and timing. We evaluate state-of-the-art automatic music transcription models on MulTTiPop and find substantial room for improvement, with the best model achieving 29% Onset F1. More details and sound examples of MulTTiPop are available at https://gclef-cmu.org/multtipop.
Abstract:We present MulTTiPop, a dataset of pop music segments and their associated multitrack MIDI recordings for the evaluation of automatic music transcription models. MulTTiPop contains 572 segments of popular music totaling 3.5 hours of audio, and contains songs from diverse genres and decades from the 1930s to 2000s. To collect this dataset, we perform metadata-based matching on song segments from the Lakh MIDI and TheoryTab datasets, manually identify an anchor beat between the audio and MIDI, then use beat tracking on the audio and warp the MIDI to match its tempo and timing. We evaluate state-of-the-art automatic music transcription models on MulTTiPop and find substantial room for improvement, with the best model achieving 29% Onset F1. More details and sound examples of MulTTiPop are available at this https URL.
8 pages, 4 figures. Associated web preview available at this https URL
Sound (cs.SD); Machine Learning (cs.LG)
arXiv:2607.08756 [cs.SD]
(or arXiv:2607.08756v2 [cs.SD] for this version)
https://doi.org/10.48550/arXiv.2607.08756 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 235 words
Related coverage
JazzSAMBA: A Synchronous and Asynchronous Multi-take Band Audio Dataset of Jazz Standards for Live Music Models
arXiv · Sep 30, 2026
TUTTI: Toward generalizable audio-to-score transcription via fully synthesized data
arXiv · Sep 2, 2026
Audio-FLAN: An Instruction-Following Dataset for Unified Audio Understanding and Generation of Speech, Music, and Sound
arXiv · Jun 9, 2026
Controllable-CoT leads to covert reasoning capabilities
LessWrong · Sep 22, 2026