Hierarchical Symbolic Pop Music Generation with Graph Neural Networks

Hierarchical Symbolic Pop Music Generation with Graph Neural Networks Wen Qing Lim1,JinhuaLiang 1, and Huan Zhang1 Queen Mary University of London, Centre for Digital Music, London, United Kingdom [email protected],{jinhua.liang, huan.zhang}@qmul.ac.uk Abstract. Music is inherently made up of complex structures, and representing them as graphs helps to capture multiple levels of relationships. While music generation has been explored using various deep generation techniques, research on graph-related music generation is sparse. Earlier graph-based music generation worked only on generating melodies, and recent works to generate polyphonic music do not account for longer-term structure. In this paper, we explore a multi-graph approach to represent both the rhythmic patterns and phrase structure of Chinese pop music. Consequently, we propose a two-step approach that aims to generate polyphonic music with coherent rhythm and long-term structure. We train two Variational Auto-Encoder networks: one on a MIDI dataset to generate 4-bar phrases, and another on song structure labels to generate full song structure. Our work shows that the models are able to learn most of the structural nuances in the training dataset, including chord and pitch frequency distributions, and phrase attributes. Keywords: Music generation ·Graph neural network ·Symbolic music generation. 1Introduction Music, an intricate tapestry of rhythm, melody, and harmony, is inherently structured in a way that lends itself well to graph representations. At a higher level, music can viewed as a progression of sections and phrases, while at a more granular level, it is made up of complex interplay of notes, chords, and rhythms. Both levels of music structure can intuitively be captured in graphs. Meanwhile, symbolic music formats (e.g. MIDI, XML) contains structured information about pitch, duration, onset time, and tracks [2]. This makes symbolic music an ideal format to be represented as graphs [28,17]. Music generation has seen rapid advancements, particularly with the increasing popularity of text-to-audio models [3,19,18] that create complete songs from text All rights remain with the authors under the Creative Commons Attribution 4.0 International License (CC BY 4.0). Proc. of the 17th Int. Symposium on Computer Music Multidisciplinary Research, London, United Kingdom, 2025 Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 985 W.Q. Lim et al. prompts. While raw audio generation has the advantage of producing high-fidelity music that can be used for creative purposes, it lacks the level of customization needed for music composition and arrangement. Symbolic music allows for more precise control over musical elements like melody, harmony, and rhythm. Symbolic music generation has made great progress in capturing inter-track relationships like chords and rhythm [14], but still struggle with long term structure. [25,20] While work like MELONS [29] and PopMNet [24] have found success in generating longer forms of music using graph representations, they work only on melodies instead of polyphonic music. Conversely, work like Polyphemus [4] have successfully generated structured polyphonic music using graph representations, but barely extends beyond a few bars. An intuitive question arises: can we derive amodelthatcreatesbothpolyphonicandlong-termmusic? In this paper, we aim to generate polyphonic symbolic music with long-term structure with a hierarchical two-stage framework. First, we train two separate Variational Auto-Encoder (VAE)s to encode and decode graph representations of song and phrase structures respectively. Each VAE encodes its input as an embedding in a latent space. Using the trained VAEs, we then generate new song structure and phrases by passing a random latent embedding through the model’s decoder. Combining the two models, the generated phrases are interpolated according to the song structure to form coherent phrases that make up a song. The contributions of the paper are: 1. We proposed a novel graph representation that captures song structure based on POP909 structural annotation, including the relationships between phrases, and the phrase attributes. 2. We explored the use of VAE graph network for song structure generation based on the proposed structure graph. We then employed a two-stage approach that combines 4-bar phrase prediction and structure prediction. 3. We conducted in-depth evaluation analysis regarding the structural aspect that follows existing literature, demonstrating that our generated songs have structure that is closer to human-composed songs. 2RelatedWork The problem we have laid out concerns three areas of research: (A) Symbolic music generation, (B) Graph representations of music, and (C) Graph generation. Each will be discussed in their respective sections below. 2.1 Symbolic Music Generation One of the challenges in symbolic music generation lies in capturing the intricacies and multi-faceted qualities [20,26, 27] of human-composed music. Bhandari et al. [1] compared different structure representations of symbolic music, and found that music generation still faces the challenges of “modelling nuanced development and variation of themes much like human compositions across extended periods”. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 986 Hierarchical Symbolic Pop Music Generation with Graph Neural Networks Dai et al. [6,8] found that generated music lacked structural properties that were in human-created music. These include phrase structure, chord and melody progressions, in which generated music failed to capture the varying patterns of entropy at different parts of the song. In a separate study [7], Dai analyzed the relationship between the structure of music and the elements of harmony, melody, and rhythm, and found that patterns in harmony and melody were related to hierarchical structure. Dai’s analysis metrics in both studies informs the evaluation methods in our paper. Prior work [11, 22] have attempted generating longer sequences of music with limited success. Mittal et al. [21] explored diffusion architecture to generate long-term melodies, and Zou et al. [29] explored a graphRNN approach. 2.2 Graph Representation of Music One of the core advantage of graph representation of music is the flexibility of design, which can be customized based on the needs of individual music information retrieval (MIR) tasks. The most common one is to construct notelevel nodes and construct edges based on note relationships [13,17,28]. There are also works on bar-level [4] and even phrase-level [29] node designs. Research exploring graph representations of music have shown the potential for graph models to perform well in tasks that benefit from learning structural information, such as music classification [9,28], modelling perceptual musical similarity [23], and modelling expressive music performances [13]. Furthermore, they provide explainability, and quantify relative importance of various features of the note [16]. 2.3 Music Graph Generation While there have been research [15, 10] on graph representations of music for various music tasks, few have explored using graphs for music generation. In particular, two key studies, PopMNet [24], and MELONS [29] have been pivotal to the study of music generation using graph neural networks (GNNs). PopMNet introduced a pioneering approach by utilizing a chord-progression and melody structure graph to generate pop music melodies [24]. It successfully captured musical dependencies, but was limited to single-track melodies only. MELONS built on PopMNet by incorporating a bar-level relationship graph and using GraphRNN with Transformer models to capture long-term structure in melodies [29]. This enabled the generation of coherent melodies over extended periods, but is still lacks inter-track information such as chords and accompaniment. Additionally, the evaluation relied heavily on subjective methods, and lacks quantitative measures of its generative ability. Cosenza et al. [4] introduced Polyphemus, a GraphVAE based model capable of generating polyphonic, multi-instrument music using a chord-level graph. The model generated music that was coherent in rhythm and pitch across multiple tracks, which was an improvement over melody-only models like PopMNet and Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 987 W.Q. Lim et al. MELONS. The model was highly reproducible, with clear evaluation of the latent space. However, Polyphemus still lacked long-term structure. Although these studies show the potential for graph-based music generation, challenges such as long-term structure and multi-instrumentality have great room for exploration. Fig. 1. Two-step hierarchical approach: Music is made up of phrases, and each phrase can be sectioned into bars. We train a model for each level of hierarchy. 3Methodology We propose GraphMuGen, a two-step approach that uses two generative models, aphrase generation model, and a song structure generation model,togenerate polyphonic music with coherent long-term structure, as illustrated in Figure 1. 3.1 Dataset and preprocessing To generate Chinese pop music, we used POP909 for training both the phrase generation and structure generation models. POP909 consists of 909 Chinese Pop songs in MIDI format, with 3 tracks per song: Melody, Bridge, and Piano accompaniment. We also use structure labels of POP909 [7], which are humanlabelled, and contains information about the phrase types (i.e. intro, melodic, non-melodic, bridge, outro), and the phrase length in number of bars. For phrase generation, we filtered the data for 4/4 time songs only. Each song is then split into phrases based on the structure labels. Phrases that are longer than 4-bars are split further to ensure that all phrases are 4-bars long (e.g. 7 bars are split into 4-bars, and (3 + 1 empty bar) tracks). Those with two concurrent empty bars are dropped. This method of pre-processing differs from Polyphemus, which extracted a moving window of N-bars. While a moving window creates a larger and more Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 988 Hierarchical Symbolic Pop Music Generation with Graph Neural Networks varied dataset, it loses the structural information within each phrase, since each example could contain bars in between phrases. Hence, we split them by phrases to ensure that the model learns structure within a phrase. Arguably, some structural information is still lost when we split 8-bar phrases into 2⇥4-bar phrases. For structure generation, the phrase labels were processed to extract the encoded phrase type and phrase lengths. Songs with more than 12 phrases were truncated (i.e. only the first 12 are included). For both models, we used a train-val-test split of 70-20-10. 3.2 Graph Representation Fig. 2. Graph representation of phrase structure of a single bar (left) and song structure (right) in POP909. Bar Level At the bar level, we adopt Polyphemus’ method of graph representation (Figure 2). Each node in the graph represents a chord at a single time step in each track. A chord is defined as a group of notes played concurrently in a single track. Connecting the nodes are three types of edges, representing different relationships between the nodes: (i) Next Edges connect consecutive nodes across different tracks, (ii) Track edges connect consecutive nodes within the same track, and (iii) Onset edges connect nodes with the same onset time, across all tracks. Each node is represented by an encoding of its Content, which contains information about the pitch and duration of each pitch. Phrase Level For song structure generation, the graph is at phrase-level. Each node represents a 4-bar phrase (Figure 2). There are two types of graph edges: (i) Next edges, which connect consecutive phrases, and (ii) Melody/Non-Melody edges, which connect consecutive melody/non-melody phrases. Similar to the Bar-level nodes, each Phrase-level node is represented by an encoding of its Content, which contains information about the phrase length (i.e. number of bars), and phrase type (e.g.. intro/outro, bridge, etc.). 3.3 Model Architecture Both the phrase generation and song structure generation models use a similar architecture inspired by the Polyphemus model, as shown in Figure 3. Each model is a VAE which contains two key components: a structure encoder/decoder, and acontentencoder/decoder. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 989 W.Q. Lim et al. Fig. 3. Model VAE Architecture. Both models follow a similar architecture. Polyphemus Architecture The content encoder uses a Graph Convolutional Network (GCN), and is responsible for encoding information about pitch and duration, while taking into account the structure of the bars. It works by first encoding the pitch and duration of notes into note embeddings ( N ), and then encoding N into a chord representation. The chord representation is then passed through the GCN with the bar-level graph representation. The GCN outputs an embedding for each chord, which is compressed into a bar-level content embeddings. They are then concatenated and further compressed to obtain a content embedding (Zc). The structure encoder is a Convolutional Neural Network (CNN) with two convolutional layers that takes in a structure tensor representing node activations across time for each track (i.e. whether a node is present or not at each time step). The CNN outputs bar-level structure embeddings, which are concatenated and then compressed to obtain a structure embedding ( Zs ). Zc and Zs are concatenated and merged to obtain a graph latent code Zg .Parameters µ and  are obtained by passing Zg through a linear layer. A latent vector Z is then sampled from N(µ,). Both decoders largely mirror their respective encoders, using specular layers to decode Z into content and structure tensors. The structure decoder decodes Z to output a structure tensor S indicating node activations. The content decoder decodes Z while taking into account S ,andoutputsacontenttensoroflogitsfor the pitches and duration of each node. Modifications from Polyphemus For the phrase generation model, we modified Polyphemus’ CNN [4] to work for three tracks instead of four. The drumrelated layers in the content encoder that accounted for percussion-pitched notes were removed since POP909 does not have a percussion track. For the song structure generation model, the CNN was further modified to work for two tracks. The number of layers in all models remained the same. 3.4 Training Similar to Polyphemus, we used a  -VAE [12] loss function, where the Lagrangian multiplier (  )isappendedtotheKL-terminthelossfunctiontoadjustthe Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 990 Hierarchical Symbolic Pop Music Generation with Graph Neural Networks trade-offbetween reconstruction accuracy and latent bottleneck capacity. We trained the phrase generation model with a batch size of 32.  started at 0 for 5,000 steps of weight updates, then increased by 0.001 every 2,000 steps. The structure generation model had a batch size of 8.  started at 0 for 2,000 steps, then increased by 0.001 every 800 steps. Both models follow the hyper-parameter settings in Polyphemus: 8 layers in the encoder and decoder GCN, with a latent dimension of 512. Adam optimizer was used with an initial learning rate of 1e-4, which was decayed exponentially after 8000 gradient updates with a decay factor of 15e-6. Both models were trained for 150 epochs. 4Evaluation We evaluated our phrase generation model and song structure generation model separately. By sampling from the latent space to generate new phrases and song structures, we compare the distribution with that of the ground truths of human-composed music1. 4.1 Phrase Generation Model We randomly generated 100 latent codes, and used the model decoder to generate 4-bar phrases from the latent codes. We measured the quality of the generated phrases by evaluating their (i) similarity to the training distribution, and (ii) ability to reproduce structural nuances. Phrase-level distribution: The following are metrics for each phrase: 1. Empty Bar Rate ( PEB ): Defined as the proportion of empty bars to the total number of bars, calculated as PEB = Nempty bars Ntotal bars , where Nempty bars is the number of empty bars and Ntotal bars is the total number of bars in the piece. 2. Used Pitch Class ( NUPC ): The number of different pitch classes used per bar, where each count reflects the unique pitch classes in a given bar. We compare the metrics for 100 generated phrases (GraphMuGen) and the pre-processed phrases (POP909) in Table 1. We observe that our model generates fewer pitch classes and more empty bars compared to POP909 phrases. For a benchmark, we randomly generated 100 tracks using Polyphemus (Polyph-LMD4), and compared its metrics against LMD-4 (Table 2). We observe that the generated tracks for Drums and Bass have empty bars and pitch classes similar to LMD-4. However, for the Strings and Guitar tracks, the PEB is significantly higher in the generated tracks as compared to the original data. This is similar to what we observed for GraphMuGen. Furthermore, the NUPC for Strings is also higher in LMD-4 than from the generated tracks. This shows that the model has varying ability to generate pitch classes and activations for different types of tracks. The model performs well for tracks with more consistent rhythm like Drums and Bass, but less so for Strings, Guitar, and Piano parts. 1Visualization of metrics and audio samples: https://graphmugen.notion.site/ Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 991 W.Q. Lim et al. Table 1. Distribution metrics for POP909 (Ground Truth) and GraphMuGen output. M: Melody; B: Bridge; P: Piano Empty Bar (EB) Used Pitch Class (UPC) MB P MB P POP909 1.01 4.69 0.39 3.33 2.62 4.68 GraphMuGen 9.85 21.8 1.25 2.03 1.39 3.33 Table 2. Metrics for Polyphemus. D: Drums; B: Bass; S: Strings; G: Guitar Empty Bar (EB) Used Pitch Class (UPC) DBSGBS G LMD (4-bar) 0.79 0.64 1.1 1.5 2.27 2.91 1.91 Polyph-LMD4 0.25 0.5 6.3 5.7 2.14 1.24 1.70 Phrase-level music theory attributes:Toaddressthechallengeshighlighted by Dai [6], we adopted the metrics in their analysis to evaluate the model’s ability to reproduce phrase-level harmonic trends. We define them as follows: 1. Chord Frequency Probability (PCF): Defined for each chord ias PCF,i =Nchord,i Nchords, bar ,(1) where Nchord,i is the count of the i -th chord and Nchords, bar is the total number of chords per bar. 2. Melody Pitch Class Frequency Probability ( PPCF ): Calculated for each pitch class jas PPCF,j =Npitch class,j Npitches, bar ,(2) where Npitch class,j is the count of the j -th pitch class and Npitches, bar is the total number of pitches per bar. 3. Melody Pitch Entropy ( HMPE ): The Shannon entropy for pitches per bar is computed as HMPE = K X k=1 ppitch,k log2(ppitch,k),(3) where ppitch,k is the probability of the k -th pitch, and K is the total number of unique pitches. The Chord and Pitch classes are based on the inferred Key of the phrase, using the music21 library [5]. In Table 3, we observe that the chord frequency Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 992 Hierarchical Symbolic Pop Music Generation with Graph Neural Networks probabilities for GraphMuGen remains mostly similar across all 4 bars. Chords V and I had significantly higher probabilities than other chords, with Chord V having the highest probability in Bar 1, but levelled out with Chord I in bar 4. Comparatively, in POP909, Chord I is is most the frequent chord throughout the phrase. While there are some differences from the chord progressions of POP909, the generated phrases showed similar chord distributions, where chords I and V were most commonly used, and chords VII and IV were more sparsely used. Table 3. Chord and melody frequency probabilities (%) by phrase position on POP909 and 100 GraphMuGen generated phrases Bar POP909 (Chord) GraphMuGen (Chord) POP909 (Melody) GraphMuGen (Melody) IIIIIIIVVVIVIIIIIIIIIVVVIVII1234567123456 7 13213119249 22313178334 216162131818711202071816 8 23111147211222915148323 115171931819719231061716 8 32812159181622614177286 217171831819712201541822 9 427141311211322810189269 117191841719721151951816 7 Comparing melody pitch classes within a phrase, the generated phrases had more variance in pitch class probabilities (Table 3), while POP909 phrases showed consistent pitch class probabilities. The generated phrases still captured some of the distributions, where pitch classes 4 and 7 were rarely used. Despite the varying pitch class distribution in our generated phrases, the entropy of melody pitches was lower than that of POP909 . This might be due to our model generating more empty bars than seen in POP909, resulting in sparser pitches. Despite this, the generated phrases captured the structural trend in melody pitch entropies, where the start of phrases have a lower entropy than the rest of the phrases. 4.2 Song Structure Generation Model Our model is able to reconstruct the type, length and Melody/Non-Melody activations, with 88.7%. 83.4%, and 91.2% accuracies respectively on the Test set. To evaluate its ability to generate new song structures, we randomly generated 100 latent codes, and used the model decoder to generate song structures from them. We measured the quality of the generated song structures by comparing its similarity to the training distribution. We adopted the metrics from Dai et al. [7] in their analysis of structure in POP909. We define them as follows: 1. Number of Phrases per Song (Nphrases): Nphrases =total phrases in song (4) 2. Number of Unique Phrases per Song (Uphrases): Uphrases =unique phrases in song (5) Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 993