Deep neural networks (DNNs) are increasingly popular computational tools across different domains, including artificial music generation. The inner layers of trained DNN models encode structural knowledge abstracted from the training dataset, even in the absence of any explicit supervision: for example, Large Language Models encode syntactic relations between words (Manning et al., 2020). In the case of music, little to no research has addressed the music-theoretically relevant structural knowledge that generative DNNs acquire during training (Cosme-Clifford et al., 2023). In this study, we show evidence that core aspects of tonal harmony emerge unsupervised in generative DNNs.

An LSTM-based model was trained towards next-pitch prediction based on 100-note-long sequences drawn from a MIDI-like encoding of a large dataset of 1300 pieces of tonal music spanning from the 16th to the 20th centuries (Hentschel et al., 2025). The dataset comprised expert-curated harmonic annotations in the form of Roman Numeral labels, which were not used during training but exploited as ground-truth for the analyses.

In essence, the model can be thought of as mapping (“embedding”) each note of an input sequence into some location in a high-dimensional space. We analyzed the internal embedding spaces to determine whether the distances between the embeddings of different notes reflected features of the underlying harmony: chord identity and harmonic directedness.

Chord identity reflects the idea that each note in a tonal setting expressed an underlying harmony. Logistic-regression models reveal that embedding distance between two different notes is a predictor of whether those notes express functionally different chords. Harmonic directedness reflects the idea that some chord progressions have an asymmetric directionality as they establish a functional relation between a preparatory and a target harmony (e.g., dominants resolving to tonics). Results indicate that embedding distances mirror such functional asymmetries.

Overall, these results support that unsupervised training on raw musical surfaces is sufficient for the emergence of music-theoretically and cognitively relevant abstractions pertaining to latent harmonic structure, and show that such abstractions are encoded in terms of distances in the model's internal embedding spaces. As one of the first steps towards the music-theoretical interpretability of “black-box” AI models, this study paves the way for addressing several questions. From a technical perspective, what kind of music-theoretical knowledge is learnt by trained DNNs and how is it encoded? From a creative perspective, how can this knowledge be explicitly manipulated? From a cognitive perspective, what are the minimal conditions under which implicit learning of abstract musical structure is possible through mere exposure to a repertoire?

References

Cosme-Clifford, N., Symons, J., Kapoor, K., & White, C. Wm. (2023). Musicological Interpretability in Generative Transformers. 2023 4th International Symposium on the Internet of Sounds, 1–9. 

Hentschel, J., Rammos, Y., Neuwirth, M., & Rohrmeier, M. (2025). A corpus and a modular infrastructure for the empirical study of (an)notated music. Scientific Data, 12(1), 685. 

Manning, C. D., Clark, K., Hewitt, J., Khandelwal, U., & Levy, O. (2020). Emergent Linguistic Structure in Artificial Neural Networks Trained by Self-Supervision. Proceedings of the National Academy of Sciences, 117(48), 30046–30054.