Encoding of musical structures in hidden units of restricted Boltzmann machines

Abstract:Restricted Boltzmann machines (RBMs) are energy-based models originating from statistical physics, in which hidden units mediate the probability distribution of high-dimensional visible configurations. In this study, we use symbolic music as a structured non-physical dataset and investigate how musical regularities are encoded in the hidden layer of a Bernoulli-Bernoulli RBM. Musical scores by J.~S.~Bach are converted into binary piano-roll representations and used to train the model in an unsupervised manner. We then analyze the visible-layer patterns induced by individual hidden units by activating hidden units separately and computing the corresponding expected visible configurations. The trained RBM reconstructs piano-roll-like inputs and assigns lower energies to piano-roll configurations than to most non-musical binary images, indicating that the learned energy function captures statistical features of the piano-roll dataset. The hidden units mainly encode local temporal and pitch-statistical structures, such as sparse piano-roll-like textures, rather than directly separable musical concepts such as melodies, chords, or keys. We also analyze hidden-layer representations using t-SNE and find that transposed versions of the same musical pieces are not necessarily mapped to nearby regions in the hidden space. This behavior indicates that the trained RBM does not robustly capture transposition equivalence, which is naturally explained by the lack of translational invariance in standard RBM architectures. Samples from the trained RBM show local pitch organization, whereas iterative continuation reveals limited long-range coherence. These results provide a statistical-physics case study of how a simple spin model represents structured creative data and clarify both the usefulness and limitations of standard RBMs as interpretable models of musical structure.

Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

arXiv:2509.04899 [cs.SD]

(or arXiv:2509.04899v4 [cs.SD] for this version)

https://doi.org/10.48550/arXiv.2509.04899 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.