aiqbench analysis
This research reveals that generative music models encode artist identity directly within their internal activations, allowing lyrics alone to trigger specific stylistic imitations without explicit prompts. The study demonstrates that these conditioning signals propagate from the text encoder to the diffusion backbone, exposing a gap in current safeguards designed only for prompt-side replication. By using latent-space analysis to audit training data, the work provides critical insights into the implicit learning mechanisms of audio generation systems.
Ghost in the Encoder: Decodable Artist Identity Representations in Lyrics-to-Song Generation
arXiv:2609.39552v1 Announce Type: cross Abstract: Text-to-song generation models can be prompted to imitate specific artists or regurgitate entire songs from their training data. Although these phenomena have been documented behaviorally on small datasets, little is known about the internal representations that may give rise to them. Prior interpretability work on generative audio has focused on locating semantic concepts such as genre or time signature within model activations. In this work, we show that a trained model can be probed for linearly decodable representations of artist identity from song lyrics alone, without any additional identifiers. Through a controlled case study of ACE-Step 1.5 spanning 2,000 songs across 100 artists, we demonstrate that the artist associated with a given set of lyrics can be identified within the model's internal activations, and that this conditioning signal propagates from the lyric encoder to the diffusion backbone during inference. These findings indicate that lyrics constitute an artist-level conditioning channel not addressed by prompt-side replication safeguards. More broadly, our work highlights how latent-space analysis can be used to audit what generative music models have implicitly learned from their training data.
Abstract:Text-to-song generation models can be prompted to imitate specific artists or regurgitate entire songs from their training data. Although these phenomena have been documented behaviorally on small datasets, little is known about the internal representations that may give rise to them. Prior interpretability work on generative audio has focused on locating semantic concepts such as genre or time signature within model activations. In this work, we show that a trained model can be probed for linearly decodable representations of artist identity from song lyrics alone, without any additional identifiers. Through a controlled case study of ACE-Step 1.5 spanning 2,000 songs across 100 artists, we demonstrate that the artist associated with a given set of lyrics can be identified within the model's internal activations, and that this conditioning signal propagates from the lyric encoder to the diffusion backbone during inference. These findings indicate that lyrics constitute an artist-level conditioning channel not addressed by prompt-side replication safeguards. More broadly, our work highlights how latent-space analysis can be used to audit what generative music models have implicitly learned from their training data.
8 pages, 3 figures. Accepted to ISMIR 2026
Sound (cs.SD); Machine Learning (cs.LG)
arXiv:2609.39552 [cs.SD]
(or arXiv:2609.39552v1 [cs.SD] for this version)
https://doi.org/10.48550/arXiv.2609.39552 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 283 words
Related coverage
Learning Jazz Pianist Style with Cross-Attention Conditioning
arXiv · Oct 5, 2026
Do LLMs feel pain?
LessWrong · Oct 4, 2026
The user in the LLM might be a moral patient too (a strange implication of the PSM and simulators)
LessWrong · Oct 15, 2026
[AINews] High Taste Testers Ep. 1
Latent Space · Oct 17, 2026