AlignBeat: A Latent Variable Model for Multi-Class Beat Tracking from Partially Labeled Data

Abstract:Recent neural beat trackers predict beats and downbeats with two independent frame-wise binary classification heads, a convention adopted because a single multi-class head cannot fully exploit datasets that annotate only beats. The heads can disagree, so Beat This moves each predicted downbeat to the nearest predicted beat. We instead predict a sparse set of grid points, each with an event time and one distribution over downbeat, beat and no event, so a downbeat is a beat by construction and no reconciliation is needed. The alignment between grid points and annotated events is latent, and we fit the model by expectation-maximization. Where no downbeat labels were recorded the event class is latent too and is marginalized out. Over eight-fold cross-validation on eighteen datasets, retaining rather than discarding the beat-only labeled data raises beat CMLt by 4.6 points on the datasets that provide it; downbeat CMLt rises 3.6 points over Beat This, with both F-measures moving by less than one point and no post-processing at any stage.

5 pages, 1 figure, 4 tables. Submitted to ICASSP 2027

Sound (cs.SD); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

arXiv:2510.14391 [cs.SD]

(or arXiv:2510.14391v3 [cs.SD] for this version)

https://doi.org/10.48550/arXiv.2510.14391 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.