Generación de componentes para la creación y evaluación de poemas musicales: módulo de componentes para la generación musical

Generación de componentes para la creación y evaluación de poemas musicales: Módulo de componentes para la generación musical Trabajo de Fin de Grado Curso 2019–2020 Autor Guillermo Villeta Torres Director Gonzalo Méndez Pozo Pablo Gervás Grado en Ingeniería Informática Facultad de Informática Universidad Complutense de Madrid Generación de componentes para la creación y evaluación de poemas musicales: Módulo de componentes para la generación musical Trabajo de Fin de Grado en Ingeniería Informática Departamento de Ingeniería del Software e Inteligencia Artificial Autor Guillermo Villeta Torres Director Gonzalo Méndez Pozo Pablo Gervás Convocatoria: Febrero2020 Calificación: Nota Grado en Ingeniería Informática Facultad de Informática Universidad Complutense de Madrid 23 de enero de 2020 Autorización de difusión El abajo firmante, matriculado en el Grado en Ingeniería en Informática de la Facultad de Informática, autoriza a la Universidad Complutense de Madrid (UCM) a difundir y utilizar con fines académicos, no comerciales y mencionando expresamente a su autor el presente Trabajo Fin de Grado: “Generación de componentes para la creación y evaluación de poemas musicales: Módulo de componentes para la generación musical”, realizado durante el curso académico 2019 - 2020 bajo la dirección de Gonzalo Mendez Pozo y Pablo Gervás en el Departamento de Departamento de Ingeniería del Software e Inteligencia Artificial, y a la Biblioteca de la UCM a depositarlo en el Archivo Institucional E-Prints Complutense con el objeto de incrementar la difusión, uso e impacto del trabajo en Internet y garantizar su preservación y acceso a largo plazo. Guillermo Villeta Torres 23 de enero de 2020 v Dedicatoria A Mor, por estar en las buenas y en las malas. En Madrid, en Barcelona o en el fin del Mundo. vii Agradecimientos Gracias a mi familia, por el apoyo y la comprensión recibidos durante los últimos años durante mi tiempo en la Universidad. Gracias a los directores de este proyecto, Gonzalo y Pablo, por su apoyo incondicional y la ayuda prestada, sus consejos y sus guías durante todo este tiempo. Gracias a Marisa y Carlos, compañeros de aventura desde el primer hasta el último día, sin vosotros no hubiera sido lo mismo. Gracias por hacerme disfrutar del viaje y poder compartirlo con vosotros. Hace casi cinco años entré en la Facultad de Informática sin saber que en ella descubiría mi pasión, gracias a todas aquellas personas que se han cruzado en mi camino durante todo el tiempo que he tenido la suerte de estar en la Facultad de Informática. Dont cry because it is over, smile because it happened Dr Seus ix 3.1.6. MIDO........................... 25 3.1.7. LilyPond ......................... 25 3.1.8. Musescore......................... 25 3.2. Hierarchical Data Format: HDF5 . . . . . . . . . . . . . . . . 25 4. Development approach towards music generation 27 4.1. Music Theory Concepts . . . . . . . . . . . . . . . . . . . . . 28 4.2. Dataset .............................. 29 4.2.1. Dataset type: the instrument . . . . . . . . . . . . . . 29 4.2.2. Dataset type: notes, chords and songs . . . . . . . . . 29 4.2.3. Dataset type: Genre . . . . . . . . . . . . . . . . . . . 30 4.2.4. Datasetsize........................ 30 4.3. Implemented models . . . . . . . . . . . . . . . . . . . . . . . 31 4.3.1. Markov Models . . . . . . . . . . . . . . . . . . . . . . 31 4.3.2. Neural Networks . . . . . . . . . . . . . . . . . . . . . 40 4.3.3. Architecture of the solution . . . . . . . . . . . . . . . 41 4.3.4. Neural Networks . . . . . . . . . . . . . . . . . . . . . 42 4.3.5. A theoretical approach to Sentiment Analysis . . . . . 48 4.3.6. Learned lessons and result comparison . . . . . . . . . 50 4.4. Results and conclusions of the developing phase . . . . . . . . 51 5. Conclusiones y Trabajo Futuro 55 5.1. Composición para improvisación teatral . . . . . . . . . . . . 55 5.2. Terminación de obras clásicas . . . . . . . . . . . . . . . . . . 56 5.3. Composición para películas y videos . . . . . . . . . . . . . . 57 5.4. Integración con asistentes de voz . . . . . . . . . . . . . . . . 57 5.5. Aplicaciones en el ámbito educativo . . . . . . . . . . . . . . . 58 5. Conclusions and Future Work 59 5.1. Composition for theatre improvisation . . . . . . . . . . . . . 60 5.2. Classical pieces completion . . . . . . . . . . . . . . . . . . . . 60 5.3. Composición para películas y videos . . . . . . . . . . . . . . 61 5.4. Voice assistant integration . . . . . . . . . . . . . . . . . . . . 61 5.5. Applications in the educational environment . . . . . . . . . . 61 5.6. Annex A: Output Sample . . . . . . . . . . . . . . . . . . . . 63 Bibliography 67 List of figures 2.1. Neural Nework types depending on the structure . . . . . . . 14 2.2. Activation function types . . . . . . . . . . . . . . . . . . . . 16 2.3. Typical structure of a Feed Forward Neural Network . . . . . 17 2.4. Typical structure of a Multilayer Perceptron . . . . . . . . . . 18 2.5. LSTM typical structure . . . . . . . . . . . . . . . . . . . . . 18 4.1. Correlation between the chromatic scale and the piano keys . 30 4.2. Markov Matrix Diagram . . . . . . . . . . . . . . . . . . . . . 32 4.3. Markov Matrix Output . . . . . . . . . . . . . . . . . . . . . . 34 4.4. Notes staff notation to midi frecuency correlation . . . . . . . 35 4.5. Markov Matrix merging . . . . . . . . . . . . . . . . . . . . . 35 4.6. Full Diagram Flow . . . . . . . . . . . . . . . . . . . . . . . . 36 4.7. Execution of the piece with Chopin Opera 18 . . . . . . . . . 38 4.8. Execution of merged pieces with Chopin Opera 18 and Opera 7 38 4.9. Diagram of an unrolled neural network . . . . . . . . . . . . . 40 4.10. Dropout layers and overfitting . . . . . . . . . . . . . . . . . . 45 4.11. Hyperbolic tangent graph . . . . . . . . . . . . . . . . . . . . 45 xix List of tables 3.1. Midi correlation . . . . . . . . . . . . . . . . . . . . . . . . . . 24 xxi Chapter 1 Introduction “Music begins where the possibilities of language end” — Sibelius 1.1. Background This TFG is embedded on a larger project designed to generate music to accompany, also generated, poems. A study of their matching is also performed. The TFG that cover the modules related to Lyrics generation and music-lyrics matching were presented last September by Maria Luisa Quiroga and Carlos Martín Testillano in a TFG titled «Components generation for the creation and evaluation of musical poems» That TFG had as aim to compose and evaluate the matching capabilities of both music and lyrics. That TFG took, as part of the input to compose those models, the output of the present TFG, that is the composed music. 1.2. Objective My aim in this project is to study the capacity of computers to compose music using different techniques belonging to the AI field. These techniques were chosen due to their versatility, the ductility of the models and the tools. Moreover, I consider AI to be an interesting research field due to the exponential growth and the trending emerging technologies. The scope of the project, therefore, is to design and implement different models, using different techniques, able to generate music and the later 1 2Chapter 1. Introduction evaluation of the result of each one of the models. Thus, the project will focus on the generation and evaluation of music and the comparative study of the possibilities offered by the field of music generation through Artificial Intelligence methods. In the same way, the development of this project will allow me to deepen the knowledge of the structures and models used in AI and at the same time, to combine it with music. Furthermore, it is intended that this project can be extended and reused by possible next TFGs or serve as an inspiration to continue finding utility to the generative models beyond the mere computational potential of these models. 1.2.1. Motivation Music is one of the unique ways we humans communicate and transmit ideas and feelings. On the other hand, the computational capacity of today’s computers and Artificial Intelligence models allow us to create systems capable of imitating human skills, both in terms of the results obtained and the learning process. At the same time, studying the internal processes of these systems and the variants that lead these systems to generate better results. In order to consider what is good or bad music, there are varied and diverse criteria within the music theory researchers themselves. That is why during the development of this project, when evaluating the pieces resulting from the composition, we will not base ourselves on objective, parameterized studies and characteristics that try to approach such comparative studies. Additionally, and through the empirical method of generating music with different techniques, we will try to parameterize certain aspects involved in the generation of music in order to compose better pieces as well as to improve the learning capacity of the models used. Since music serves, among other things, to transmit feelings and each piece of music carries with it a sentimental degree, a study will be carried out which aims to analyse the association and relationship between feelings and the pieces of music. For this study, the resources used by the musical composers to transmit feelings (fugues, mishaps, dialogues,...) will be taken into account and the factors intervening in this aspect will be developed. In order to recreate these compositional techniques, the differential factors 1.2. Objective 3 of each of the techniques and when they are present in the musical pieces will be studied. The aim of this study is to see in which situations these techniques are useful when expressing feelings. Through this study, the aim is to carry out the parameterization of the techniques in order to recreate them later. To do so, the variables that make possible the musical composition associated with a certain feeling will be parameterized, making possible the enrichment of the musical pieces generated. The final objective is to try to get the Artificial Intelligence systems used to compose music, not to approach human compositions, but to be able to provide the composed pieces with feeling and meaning beyond the correct succession of notes and silences. This factor, together with the possibilities offered by the models used, will be the differential factor in making this project an approach to music generation through the unification and use of different compositional techniques belonging to the field of music theory and Artificial Intelligence. 10 Chapter 2. State of the Art the 7094 to sing were John Kelly and Carrol Lock, but the song was already composed, the computer only was able to play it (Ellis, 2006). Therefore was not a proper composition, even though results were impressing and planted a seed among computer researchers. The task was repeated later, when computer HAL 9000 who also sang this same song. This event was recreated later in 2001 in the film «A Space Odyssey»(Boylan, 1985). If we dig into the history of proper computer-based music composition, we have to go back to 1951 when the computer scientist Alan Turing, known for decrypting the Nazi code during WWII, created a computer capable of composing little musical pieces. Unfortunately, the computer was destroyed but Jack Copeland and Jason Long (Copeland and Long, 2017) , two researchers from the University of Canterbury in New Zealand back in 2016 were capable of reproducing the computer and some of the pieces Alan Turing composed with that computer. Computer-based music composition is a wide and studied field, and we can distinguish among the researchers and currents according to the method used and the approach taken in order to create a computer-generated musical piece. In the forthcoming sections I will expose the main currents and techniques used by researchers who dig into this field. 2.2. Composition from previous learning Most researchers start by studying the way humans compose music and they try to figure out what can we learn from those techniques and processes and how to transmit that knowledge to a computer, as well as factors that can be put into consideration when trying to make a computer come up with musical pieces. Composing music is nothing else than to arrange a number of limited notes (tones, semitones and silences) one after another. A note is formed by a pitch (tune) and a duration. But there are several factors that arise when deciding what note to put next. Humans compose music, influenced by their previously learned or heard music. Therefore, the first thought would be to feed the computer with music pieces and find a way to analyse them in order to find common patterns or structure in order to learn from those factors, and then, try to add them into the computer composed pieces. This approach is taken by Carr and 2.3. Most used techniques and methods 11 Zukowski (2018), in whose research papers come up with the explanation of the bits and pieces that are taken into consideration in order to build a model that works based on this methodology. Another important question that can be found in many researchers is the one related to taking into consideration the control over the process when generating music. Controlling the output involves that the user is able to change certain aspects of the configuration of the model itself, or certain aspects directly related to the generation process. If the output can be controlled or driven while music is composed (generation time), the process is called «conditional» (Carr and Zukowski, 2018), otherwise is called «unconditional». Therefore the first type of composition will allow the user to decide or reconfigure certain aspects so that the output adapts to those changes in order for the it to be more pleasant or fit the established requirements. 2.3. Most used techniques and methods Simplifying music composition, we can define it as the process through which notes (formed by pitch and duration) are put one after the other to form a full musical composition. Of course, there are different variations of the same group of notes, but the way music notes are arranged has a probabilistic variable attached. Of course, there are several techniques being used and points of view from they address this challenge, but they all share this important fact we now want to recall. A project that shows this fact very well is the one created in PyPoser (William (2018)), which is a project where music is composed merely based on previously set probabilities. Although Oliveira’s work was mainly focussed on Poetry Generation with PoeTryMe, or Tra-la-lyrics, both related to computer generation of lyrics, it serves to show there are many possibilities and application of this technique. The Poetry Generation methods used by Oliveira can illustrate very well the challenges faced by Hugo and the kind of problems I could be facing during the development of my project as well as the possible ways to face and resolve those problems and challenges. Due to the fact that the field that covers music generation is widely open and diverse we will structure the relevant references to work developed by other investigators in a way that served as inspiration to our work. 12 Chapter 2. State of the Art Music Composition sets a very big challenge for investigators who wants to develop their abilities as computer and Artificial Intelligence researchers as in order to generate music it is necessary to have a previous knowledge of the field. When digging into other investigators work, I selected those who were mainly focused on music generation such as Fraçois Pachett (Aucouturier and Pachet, 2003). His work on music generation and research related to music composition and genre classification of music drove us to conclude that one possibility was to use Artificial Intelligence to develop our work. We started studding several techniques applied to music generation such as Markov Chains and Long-Short Term Memory Network (from now on LSTM), that we later complemented with sentiment analysis research. Pachet is has developed several components for classification and composition of music in different genres and musical currents. On his work called "Taxonomy of musical Genres" he investigates about the different facts that makes a piece of music belong to a certain Music Genre. To classify music pieces and fit them into a determined genre he takes into account some facts as Geographical inclusion, Agregation, Repetition, historical period, etc. We focused on extracting from his work the musical aspect on music generation that made him generate music from an specific genre or classify music into an specific genre using the TFG developed by Caparrini López, Antonio and Perez Molina, Laura that investigated "Genre classification of electronic music"(Antonio Caparrini López, 2017). The work developed measures the accuracy of music genre classification techniques. This work served us to see the important facts concerning genre classification and therefore those ones to take into account in order to classify music pieces and follow a given pattern. This work served to connect with the sentiment analysis area of our work. Not necessarily focussed on music composition but in creation of RNN models, it must be recalled the work of Andej Karpathy Karpathy (2015), whose explanation of the internal functioning of Neural Networks was found to be very useful to understand in depth the structure of a Neural Net. During all the research phase some important aspects to take into account on music generation could be brought up: 2.3. Most used techniques and methods 13 2.3.1. Frequently Used Techniques Hereafter we will expose the main techniques used to create computer models capable of generating music compositions. It is worth recalling that most of these models were born in the mathematical field, and they turned out to be useful in order to create computer-based generated music. 2.3.1.1. Markov Models This technique is used as the starting point by most researchers that take into account the statistical fact of music composition. Markov Models are models first introduced by Andrey Andreyevich Markov in 1906 (Basharin et al., 2004). Andreyevich was a Russian mathematician that conceived for the first time this kind of models that, later would give birth to the nowadays known as Markov Chains. A Markov model is a an automate system arranged in form of state chains. The change among states is only determined by the current state, not taking into account previous states (Kouemou and Dymarski, 2011). This states are autonomous from each other and the relations among them is stochastic, so the current state is the only factor that is taken into account to find out what the next factor would be. Putting it into music terms, the probability model studies the chance of a note to be followed by any other given note. This way wee create a branch map where we set correlations among the different notes conceived in a composition. This way we can start creating a brand new composition starting with the probabilities for a note to be followed by any other given note. The most common way to put this relation down to paper is by creating a relation table where probabilities among the notes are pictured. The model, starting from the knowledge adquired from the training song, will create a brand new song taking into account the probability table created by the song. The main advantage of this model is that, musicality factors are nearly always ensured, as long as the piece used for training the model has in it these particular characteristics. One example of a musician that used Markov Models is Ianis Xenakis, a musician that in 1958 used a Markov Model to create a piece of music called 14 Chapter 2. State of the Art Analogue (Xenakis, 1958). 2.3.1.2. Neural Networks A Neural Network is a structure formed by neurons. A neuron is a independent structure that performs an operation on a given input to deliver an output. The Neural Network is formed by one or more neurons arranged in layers. As with Neurons itself, Neural Networks can have one or more layers. Depending on the way layers and Neurons are arranged, there are several types of neural networks. The Network can receive an input formed by one or many arrays as an input and can deliver one or many arrays as result. As shown in the next figure, extracted from (Karpathy, 2015), depending on the relation among the number of outputs and inputs, we can distinguish among one-to-one, one-to-many, many-to-one or many to many where the first argument indicates the number of inputs and the second argument, the number of outputs. Figure 2.1: Neural Nework types depending on the structure Among the different layers, the output is forwarded, therefore the output of the previous layer serves as input of the coming layer. At the same time, the output of the later layers can also become part of the input of the previous layers. This property is called «backpropagation». Which turns the Neural Network into a Recurrent Neural Network. As well as the output of the later layers, the final output of the network can also serve as the input of some of the previous layers or it can be incorporated into the dataset, turning it into the input of the main Neural Network. This backpropagation capability of the Neural Network, makes a huge impact on the output of the Neural Network, and it is the main difference 2.3. Most used techniques and methods 15 among Network and the previously discussed models. This backpropagation property, makes it possible for the Neural Network to learn from previous compossed music and it enriches the output making it more human alike. This capacity to remember previous factors, can allow the Network to compose music with a given structure (which other models did not allow) and it allows for the resulting composition to be more human-alike. One of the main advantages of this model, as well as the continuous learning factor, it the fact that it also takes into account the evolution of the pieces, therefore, if well configured and properly fed, the Neural Network will allow us to come up with a composition that preserves the structure of a usual song (intro, verse, chorus, verse, chorus, and so on). The main advantage this kind of models create is the powerful potential of this kind of models when it comes to analyzing music compositions and other kind of structures. Neural Network models present a big advantage when generating songs are the starting model for music composition, come to embrace and resolve a main issue that appears when using other kind of techniques: the memory factor. To understand the memory factor we can first start by recalling the structure and coherence along the play of any song. This makes the song to be rithm consistent, but feeling variable. To make things simple, we can understand this as follows: "the songs sounds good, but it sound the same all along". The way Neural networks tries to resolve this problem is by taking into account, not only the relation among the different notes, or the probability of a note to be followed by any other, but also the main structure of the song and the different probabilities depending on the part of the song we are creating and learning from. The result of this technique, and what we find to be the biggest advantage, is to create richest musical pieces and more human-alike compositions. Karpathy wrote a paper back in 2015 titled "The Unreasonable effect of Recurrent Neural Networks" (Karpathy, 2015) on which he relates the functional model of a Neural Network and their possibilities in computational creativity and data handling and generation. He generates Shakespeare text and monologues from training a Recurrent Neural Network. 16 Chapter 2. State of the Art The way a Recurrent Neural Network works is similar to the Neural Networks found on human brains: the network must receive a set of input data that allows the model to «learn» from it. Iterating over the dataset (a number of specified epochs or iterations), the network is able to determine the weights. The weight is related to the memory of the neural network, as it determines the quantity and/or the quality of the connections among the cells. The Propagation function is the resulting potential gotten from the interaction of the current neuron with their neighbours. And finally, the activation functions are the ones that determine the state of liveliness of the neuron. The liveliness of the neuron is the crucial factor that determines if it takes part on the functioning on the network. The activation function is based on the propagation function and the previous to the interaction. In the following figure2we can see the most common types of activation functions: Figure 2.2: Activation function types Finally, when all previous calculations have been performed, the exit function delivers the result to the neighbouring neurons. If the neuron is inactive, which means that the activation function has not been triggered, the exit function will not deliver any results, turning the neuron into an isolated cell. 2https://towardsdatascience.com/complete-guide-of-activation-functions34076e95d044 2.3. Most used techniques and methods 17 2.3.1.3. Main Neural Network Types As said before in the theoretical part, a Neural Network is formed by neurons arranged in layers. The number of neurons and their attributes can vary depending on the layer and their role in the model. Keras allows us to create a model, stating as parameter of the layers their attributes and the type of NN. The attributes we can specify can be: dropout layers, number of neurons, type of NN, activation function,... Several types of neural networks can be defined in order to create our model. It is worth recalling that there are nearly infinite types of Neural Networks. This is due to the diversity and variety of possibilities on the design of the network and their behaviour. Hereafter we specify some of the models that we took took into consideration for the creation of our network and the reasons why we decided to use or discard them in each case: Feedforward This is the simplest model of the neural network possibilities. Data travelling through the neural network, only goes in one direction onwards to the output nodes from the input nodes. Its simplicity relies on the position of the neurons and design of the network. It is a first approach to NN The following figure extracted from (O’Shea and Nash, 2015) shows the typical structure of a Feed Forward Neural Net Figure 2.3: Typical structure of a Feed Forward Neural Network Multilayer Perceptron 18 Chapter 2. State of the Art The following figure3shows the structure of a Multilayer Perceptron Neural Network: Figure 2.4: Typical structure of a Multilayer Perceptron The Multilayer Perceptron (Rosenblatt, 1961) is a neural network that, given a number of inputs, they deliver a series of discrete outputs. This network is normally used, and gives a good result, when the aim is to classify a series of inputs. The main characteristic of this network is that the number of outputs, as the classes to classify, can not be the same as the number of inputs. Recurrent Neural Networks - Long-Short Term Memory Networks The main characteristic of this network is that it exploits the fact of memory-lasting training that the neural networks are expected to deliver. The following figure4shows the usual structure of a LSTM Network: Figure 2.5: LSTM typical structure This is done by registering, and re-feeding the input of the network and its neurons with the results of previous trainings. This model also offers 3http://www.jpathinformatics.org/ 4https://colah.github.io/posts/2015-08-Understanding-LSTMs/ 2.3. Most used techniques and methods 19 the posibility of back-propagation. Backpropagation is a characteristic that relies on the fact that the output of some of the layers of the network, can be turned into the input of some of the other layers. This is a common feature in Neuran network theory. This technique gives as a result a better understanding of the structure of the song used for training the network. The result is similar to the one we would get if we chop the song into its parts, and use those several parts as the input of the network. The result of doing this enriches the music with taking into consideration the parts a music composition is divided on, and respects the changes in the feeling and sensations transmitted by music when it is played. A full theory why this types of networks are a good options when it comes to compose music can be found in the article by François Pachet titled "Sampling variation of sequences for structured music generation" (Pachet et al., 2017). A more general approach to this fact can be found in (Karpathy, 2015) This network, due to the capabilities that it offers, and the nature of its structure, is considered the best possible solution for this project’s aim and will become one of the main axes and chosen techniques of the development of the music generation module. 2.3.1.4. Genetic Algorithms A genetic algorithm, is a computer algorithm that deploys the factor of DNA mixture among two parents. On the DNA structure of a child, we can find common structures found on the DNAs coming from the parents that generated the child. Brought to music composition, this techniques mixes factors studied in two musical pieces and performs changes into those factors and the combine them to create a brand new musical piece (Horner and Goldberg, 1991). The most common variations of this factors could be: Permutation Swapping position and mixing values among the factors to create new relations Crossovers Chapter 4 Development approach towards music generation The following chapter will put down in paper the development of the created models for music generation. Therefore, it contains the techniques used, the final model created, and the tests performed with the output generated will be explained hereafter. In order to clarify information of the development phase, I will split this chapter into three main parts: generation, sentiment analysis and evaluation of the resulting pieces according to diferent criteria. Each part will contain the key points of its phase and they will be presented as the natural steps to follow to compose music using the developed techniques. Although it is split in three apparently independent parts, the development phase was an evolutionary and continuous improvement path and during which it was necessary to go back to previous steps, phases and techniques in order to re-design or re-evaluate previous decisions. Although it was thought to be an autonomous system and concieved as a whole, the parts that formed de development will be taken as independent parts, and treated as independent components according to the techniques used in each one of them. 27 28 Chapter 4. Development approach towards music generation 4.1. Music Theory Concepts In order to start developing the model, it is worth starting by pointing out some basic music concepts: Music Notation: the music notation is the system used to visualize music sounds. Music notation is formed by staff, notes, silences and variations. Staff: the staff is the set of horizontal lines that the notes will be placed on. Music note: a music note is the symbol denoting a musical sound. Pitch: is the music frequency of the note. Duration: determines the length of time for which a note is played. Music chord: a music chord is a set of notes played in the same time beat. Beat: is the basic unit of time used in music. Tempo: is what sets the speed of the beat in the composition. Scale: is a group of notes arranged in ascending or descending order. The most common scales are the major and minor scales. This scales will be used to develop the theoretical approach to the sentiment analysis section. Chromatic scale: a chromatic scale is the range of notes by which the music piece will be formed. Ascending/descending order: the order specifies the succession of notes according to the relation of their pitches. If the pitches go from low to high, the scale is said to be ascendant, otherwise, it is descendant. 4.2. Dataset 29 4.2. Dataset In order to train our model it was necessary to define a dataset. A dataset is a set of inputs chosen to train our model. During the developing of the project, several datasets have been used. The first approach was taken using the Magenta MAESTRO1dataset, a dataset composed by piano virtuoso composition. Moreover, for the testing of the Markov Models, the dataset was enriched with certain classical compositions obtained from Classical Archives2. Additionally, due to the musical training already acquired before enrolling this project, I could record myself using an electronic keyboard and throughout the use of Audacity3, a simple music manipulation and recording software. Using this simple software, I was be able to record myself playing certain pieces on the piano and training the model using those songs. The main reason why I chose to record myself playing those pieces, and the comparison between the expected and obtained result using them will be explained in the following sections. 4.2.1. Dataset type: the instrument During the development of the project we will focus on generating music composed and played for piano. Piano is a widely known instrument, with a chromatic scale big enough to embrace music pieces from different genres, and scales, and therefore we will not limit the resulting pieces to a fixed chromatic scale. The range of notes stablished in the piano can be seen in the following figure4: 4.2.2. Dataset type: notes, chords and songs As mentioned above, the dataset can be formed by the three main components that music is formed with: notes, chords and songs. Due to the fact that the developed Markov Models, as it will be explained 1https://magenta.tensorflow.org/datasets/maestro 2https://www.classicalarchives.com/midi.html 3https://www.audacityteam.org/ 4http://bigit.karikaturize.com/piano-music-notes-chart/piano-key-chart-beginnershere-are-a-few-easy-exercises-to.html 30 Chapter 4. Development approach towards music generation Figure 4.1: Correlation between the chromatic scale and the piano keys later, is focused on the probabilistic approach of preceding notes, it is necessary to have a full range of notes preceding one-another in order to study the probability of a note to be the succeeding one of a given note. Moreover, if we look at it from the research approach, as said previously, the aim of this project is not to focus only in music generation, we would also like to generate music based on the learning of already-existing compositions. Therefore, it is necessary to feed the models with the pieces it us supposed to learn from. 4.2.3. Dataset type: Genre During the development of the work classical music was used to train the models. The reason for it is the variety of notes found in classical music compared to other genres, and, therefore the amplitude on the learning process it impulses. The chromatic variety of the classical genre allows a deeper study of the inputted songs. 4.2.4. Dataset size The size of the chosen dataset is determined by the number of pieces that forms it, those pieces will serve as the input of the system. It is necessary to take into account the size of the dataset. As, it determines in a big way the behaviour of the model. A dataset too big will make the model to come up with pieces incoherent and formed with a huge amount of information. 4.3. Implemented models 31 The reason why a big dataset, in our case, deploys an unfavourable output is because the model tries to incorporate into the resulting pieces, all the learning acquired during the training phase. On the other hand, a small dataset, as it was experimented, makes the model come up with a result too similar to the input pieces or note. These aspects will be explained in the following sections. 4.3. Implemented models Hereafter, the models created to generate the music pieces, their structure and architecture will be explained. In this module two main techniques will be used: Markov Chains and Neural Networks. 4.3.1. Markov Models The Markov Model created for this project is nothing else than a matrix where each component (formed by «x», and «y» variables) is composed by the probability that states the chance for a note to precede another note. If we imagine an usual matrix, the rows and columns that will conform the matrix will be the notes themselves (formed by «pitch» and «duration»), and therefore will be the axis of the resulting matrix. The body, on the other hand, will be the probability of a note to be followed by any other note. In this way, if we read the matrix shown in figure (X), we can say that the note «XX» is followed by the note «YY» with a probability of «ZZ», on the other hand, note «XX» is followed by note «YY» with a higher probability. The matrix will be constructed from the dataset given to the model (in this case, a song), studying the number of times a given note of the piece is preceded by another specific note, and therefore, its probability. Once the matrix is constructed by different probabilities of a note to be followed by another note, we will start by randomly picking an element of the matrix. Once an element has been chosen, we will maximize the probabilities creating a tree of one branch, on which, we position one element after the other. The result will be a list of elements by their coordenates (x,y) where «x» is the preceeding note and «y» the successor. The list therefore, if we maintain only the «y» of all elements, and the «x» value of the first element, will be a list of notes that will be the sequence of notes that will form the 32 Chapter 4. Development approach towards music generation Figure 4.2: Markov Matrix Diagram final music piece. In figure 4.2, a high level representation of this matrix has been designed. The use of Markov Chains is all about predicting the next data on a sequence given some probabilities and maximizing them. The use of Markov Models for music composition, is based on the piano roll mechanism of implementing music, as it condenses the notes of the musical composition chromatic scale into a matrix taking into account the appearance, and «activating» the cell of the notes the song contains. 4.3.1.1. Architecture of the solution In order to develop all the above mentioned model, the task will start by dividing the implementation in three submodules: Generator Module 4.3. Implemented models 33 Converter Module Markov Module This structure of the solution allows to see the different steps taken to generate a full musical composition using the Markov Models approach. The internal functioning of each component is explained hereafter: 4.3.1.2. Generator Module This class will be used as the main driver of the composer method and will be the main controller of the model. For the implementation of this module, the Mido library has been used for the handling and manipulation of the midi events. On this class we will create the Markov Chain from the specified song or melody using the Markov Chain Module, and once created, we will generate the new melody that will form the output of the module. The Markov Matrix will be created using the Markov Chain class that will be explained later. And it will be saved in a structure of a dictionary that will store the different probabilities of the matrices. Once the Matrix is created, then it is necessary to create a list of notes, extracted from the resulting matrix, that will be converted into midi events so they can form a brand new song. For that purpose, we will use the get next function that will allow us to get, according to the current note, the one that must be the successor of the actual note according to the probability established by the matrix. The duration of the pieces is determined by the setting of a variable that enunciates the number of note the resulting pieces will have. 4.3.1.3. Converter Module This module will allow us to convert the midi file into a Markov Chain by extracting the data that the midi contains and creates the list of events that will be treated as the Markov Chain module to create the Matrix. In order to create de nodes, it goes through the midi file and extracts the duration, pitch and velocity of the notes. From that information, the class will create the sequence of the notes that will form the song in midi format. 34 Chapter 4. Development approach towards music generation The velocity argument of the note is how it known the strength with which the note is played on the composition. This argument will be a part of the sentiment analysis section in the further research section. 4.3.1.4. Markov Module This last module is the one in charge of creating the Markov Chain from the information parsed by the Converter module and manage this information correctly to create the new generated melody. This class will create the matrix of notes, formed by duration and pitch of each note that appears on the piece. The form that this matrix will take it is shown in the following figure. Figure 4.3: Markov Matrix Output As we can see in the figure, the columns and rows of the matrix are formed by two arguments: the first argument refers to the midi frequency correlated to the note and its duration. In the following figure we can see a representation: Additionally to the specified function of the module stated above, some utilities functions have been used: Print Matrix: 4.3. Implemented models 35 Figure 4.4: Notes staff notation to midi frecuency correlation This function allows the user to see the matrix built from the input that has been generated. It prints the matrix in the format specified above.It takes as a self-argument a chain of notes extracted from the midi file. Due to the fact that a big amount of notes may appear in a song, when printing the matrix only a small portion of those notes will be printed, showing the user the most significant part of the matrix, that is, the notes with the highest probability of appearance. Combine two matrices: merge This utility allows joining two matrices of two given music pieces. This function allows the model to take two arguments instead of one. The functioning of this version of the model allows to extract the matrices related to each of the arguments (midi files) and combine them to create a new matrix from which to generate the output of the model, as shown in the following figure. Figure 4.5: Markov Matrix merging The combination of both matrices is done by summing the values of the same components of the matrix, that is, in our model, if two notes 42 Chapter 4. Development approach towards music generation and the outputs, a MIDI file and a music sheet, will be passed to the evaluator, who, after performing the corresponding calculations and methods, will give the components feedback in the following form: Changes on a certain note or group of notes: the evaluator can identify which notes do note match with the given music and the Lyrics generated and tells the composer which notes to be changed. The composer has a module on modifications with utilities functions in order to perform available modifications. Make music sadder or happier according to the sentiment analysis performed. In this case, the music composer can make use of the Markov Chain modules to generate the resulting Markov Matrix of the composed piece. Then, it will calculate de Markov Matrix of a sadder or happier song, depending on the requirements and will merge those two matrices. The happier or sadder song will be picked from the corresponding dataset. After the new matrix has been generated, a new song will be created and the result, as the "parets" are sad, it will be sad. During the development phase will see if, this process has to be performed several times in order to appreciate the change in the music composition. In case several runs are needed, we will straight away apply as many as needed for it to take effect. 4.3.4. Neural Networks To keep exploring the possibilities of Computer Generated Music the next approach taken was to create a simple model based on general Neural Networks using Keras. The model will allow,to test the different Neural Netwoks models (such as LSTM), as well as setting the parameters of the networks (activation functions) and defining the created model. The use of RNN will allow two main things: Creation of music by the iterations over a Neural Network: 4.3. Implemented models 43 As we talk about a RNN, the network is fed with recurrent data. That means that the data that enters into the net, is combined with the data generated by previous steps in the training phase. As we create the model using LSTM, the resulting output will take into account all input regardless of how back in time they are. Get rid of the vanished gradient problem and therefore create richer compositions and better training. The Vanished Gradient Problem states that, when training a Neural Network, due to the nature of the training, the network will try to cover all possible outputs, and take into consideration all variants along the song and could cause the vanishing of part of the information along the way. That fact drives us to conclude that the resulting tests performed with RNN no LSTM, did not provide the pieces with a defined structure, variants or modulation, and as we will see, create pieces filled with eighth notes. Due to the nature of the LSTM network, the resulting song should have a defined structure and not be monotonic or seem too random. 4.3.4.1. Network type The approach taken towards Music Generation using Neural Netowrks, focused in the developing of a Recurrent Neural Networ, and the use of a Long-Short Term Memory Network (LSTM). The model was created using using Python Anaconda Spider as the main tool to test and implement the model. 4.3.4.2. Music parsing We started by creating the formatting input of the LSTM network as a sequence of events using Music21. Given that MIDI files is coded this way, the resulting parser is a simple implementation that saves the midi events into an array. Due to the fact that RNN behave better with numerical values, rather than music notation values such as ABC notation, it was necessary to create 44 Chapter 4. Development approach towards music generation a parsing function to create transpose the different inputs into a numeric value. Therefore, the resulting input of the network will not be a midi file, but a set of numerical consecutive values. 4.3.4.3. Input trimming In order to decide the amount of information given to the network as the input parameter, as well as the information obtained from it, it is important to «trim» the data into smaller pieces. 4.3.4.4. Data normalization It is not strictly necessary to normalize the data, but it has been shown that the AI algorithms perform better with normalize data when dealing with big datasets and eases the back propagation training process of the networks (Sola and Sevilla, 1997). For that purpose, we started by parsing the midi file from the input and turning it into a series or sequences of event formatted in order to become the input for the Neural Network. The format chosen is to define a correlation list of lists of notes an integer values. 4.3.4.5. Model Creation The next step is to create the model itself. The model will be formed by a Neural Network formed by several training layers. We will also include some regularization layers with Dropout Srivastava (2014). Dropout is a technique for regularization of neural networks that allows the dropout of random neurons along the network to avoid over fitting. See figure 4.11. In the model definition, we will also include the activation function chosen. During the development of the project, several activation functions have been tested. The results obtained form the tests will be explained later on in the chapter devoted to analyse the obtained results. The activation functions (see figure 2.2) tested are: Sigmoid 4.3. Implemented models 45 Figure 4.10: Dropout layers and overfitting The sigmoid activation function is a function whose values range between 0 to 1 that has the shape of an S Hyperbolic Tangent It is defined between 1 and -1 Figure 4.11: Hyperbolic tangent graph Rectified Linear Unit (RELU) This activation function comes to resolve a problem attached to the use of the Sigmoid and Tangent functions called the Gradient Problem. 46 Chapter 4. Development approach towards music generation Apart from fixing the gradient problem, it also provides more sensitivity to the output of the nodes and avoids saturation of the nodes. The use of this function also provides the possibility to decrease the number of dropout layers in the neural network. The function works in a similar way to the max function in common mathematics. It return the maximum of two given values, but it has the characteristic that half of the input for the RELU function will deliver a lower boundary value previously set (normally zero), and at some point in the development of the function, will start performing like a linear function. This fact, makes the activation function to filter the output and discard or delete the union of two given nodes. We can imagine, the activation function works like a switch, deactivating or activating the connections of the nodes depending on the inputted value. Hereafter, we will show some advantages we could see using this function as the activation function, some of them recalled in (Glorot et al., 2011): Computational Simplicity As it can be coded as a max function. Representational Sparsity This function is capable of returning a zero value, unlike the others that will always return an approximate to zero value Linear behaviour As it behaves very similarly to the linear function, but the fact of setting the lower boundary, allows the function to be more sensitive and returning a more filtered output. Used in Deep Neural Network Due to the Rectified Linear activation, and the back propagation of the recurrent networks, it allows fo the networks where it is used to be trained faster and with better results in computational terms, and as it was shown during the testing phase, better musical results. Moreover, Keras allowes us to change some characteristics of the model implementing different optimizers as well as different activation functions. 4.3. Implemented models 47 The optimizer is a parameter for the Keras Neural Network model created that has the aim of optimizing the performance of the Network and checking the effects of activation function modification in the resulting music pieces. There are several Optimizer we have tested: Adelta (Zeiler, 2012) Adam (Kingma and Ba, 2014) Adamx (Kingma and Ba, 2014) Nadam (Dozat, 2016) Although they all have been tested, the one that delivered the best result was the RMSprop, as said in (Bengio, 2015). The above mentioned parameters were set and adjusted during the testing and developing phase and the information about them has been collected from the Keras Documentation5and from the cited sources. 4.3.4.6. Network Training In order to train the model, the previously prepared data will serve as input. To train the model, it is necessary to specify the number of times the model will go through the training data set, that is, the epochs or iterations, as well as the number of items that will form the training dataset. After the training of the network has completed, the resulting weights are saved into a «.hdf5» file. 4.3.4.7. Music Generation Finally, in order to generate the resulting piece, it is necessary to combine the information collected throughout the training, that is, the weights, and input data. It is worth recalling that the input data, as well as the training data, have to go throughout the same parsing. This implies that we have to convert the input data into the sequences of numerical values, as the training data in order to be input to the model as it has been previously explained. 5https://keras.io/ 48 Chapter 4. Development approach towards music generation To generate the array of notes, it is necessary to specify the number of items, that is, notes, and this allow us to define the length of the resulting piece. The output of the model, therefore, will be an array of notes. 4.3.4.8. Output handling: reverse parsing In order to convert the array of notes and chords into a music piece, it is necessary to decode it, that is, perform the opposite process performed in the Data Preparation Section. The list of notes, then, will be converted into a Music21 Object and, with that item, we can save the corresponding Midi file, finally obtaining the result of the model. The resulting midi file will can be subjected to the same process as the output from the Markov model in order to convert it into a Music Sheet using Lilypond. 4.3.5. A theoretical approach to Sentiment Analysis This part of the project states a theoretical approach to Sentiment Analysis, proven and tested with the models specified above. In order to create music, we must not forget about the sentimental part. The Sentiment Analysis is the science or discipline that studies the facts and ingredients that are necessary to create moving music pieces or compositions. Driven by the possibilities of musical creation that the chosen methods provided us, it was considered the best option to create a personal and own strategy for music composition to create sentiment analysed music. Given the fact that this project is thought to be included in the project presented by Marisa y Carlos, the strategy must be as simple as possible in order for the evaluator to give feedback about the "score" of music that will be analysed. Given the fact that the factors or variables that the evaluator will take into account are directly related to style and music theoretical approach inherited from pieces that were the originators pieces, that is, the output will inherit some factors from their "parents" that the evaluator will take into account. Bearing that in mind, we will have to take them into account in order to 4.3. Implemented models 49 regenerate music, several factors such as those ones are metronome (tempo), key, mode, etc. Therefore the main idea is to create a personalized dataset that contains classified pieces of music and according to the feedback given by the evaluator, the re-generation or modification of music, will take into account two main facts: the original dataset from which the song has been generated and the creation of a new dataset to regenerate the whole song or part of it to comply with the given feedback. The resulting piece, therefore, could be generated by a Wagner Dataset, but if the Evaluator module created by Carlos, tells us to soften a the musical composition, we may introduce into the dataset some Chopin sonatas as they are characterized for being in a slow tempo and lower pitch of the chromatic scale, compared to the Wagner composition. 4.3.5.1. Tempo & Scale To figure out how this datasets can be created we had to stablish some initial constraints or assumptions and if the feedback of the song spins around the sadness or happiness of it. We take into account this two factors: tempo and scale. Tempo of the song: We will assume, as stated in (Wassermann et al., 2003), that most of the slow music compositions created in classical music, tend to be quiet and calm or sad. Therefore, the songs created or composed from pieces with this ingredient, are more likely to be sad or calm. Scale of the song If the scale of the song is composed by major chords or notes that are normally high, we assume the song is a happy one and therefore the resulting pieces generated from this piece are more likely to be happier. 4.3.5.2. Velocity As told before, velocity is an argument in a midi event that is used to determine the strength with which the note is played by the instrument. As we can all recognize, aggressive and happy songs, are related more to notes being played with a high intensity, which in music would correspond 50 Chapter 4. Development approach towards music generation to forte or mezzoforte modulations. The main aim of this fact is that the pieces try to transmit a sense of joy and pleasure to the listener. On the other hand, sad or melancholic songs are normally related to a softer play, a more delicate and probably quiet sound. Bearing this in mind, therefore, by the analysis of the music pieces, we can recall that the quiet songs will be more likely related to sad songs while "harder" songs will, somehow be related to happy songs. The sentiment analysis module will take all this into account and will drive the instructions from which the music will be changed and modified according to the feedback given by the evaluator. Even though the sentiment analysis module is based on a simple principle, as shown in the testing phase, returns good results and the outputs can be considered sentiment sensible. Of course, it has some limitations, due to the size of the data set and the nature of the methodology, but as the approach returns good results, it can be considered a success. 4.3.6. Learned lessons and result comparison Hereafter I will point out the most significant keypoints concerning outputs and models creation: Importance of Parameters In both models the values set to the parameters that control de model, it turned out to be highly important, and conditions the results obtained throughout the use of the model. MIDI manipulation Regarding MIDI and audio manipulation has been surprising the amount of possibilities to handle and manipulate those files To handle music objects, several libraries and sources were found and many of them, tested. We focused on two main modules, mido and music21 as MIDI handlers, but other, such as pretty-midi were tested. Approach of Piano Rolls During the first few steps on the creation of the Neural Network model, the possibility of using piano rolls as the input was conceived. A piano Roll was a structure used a the begining of music history as a roll 4.4. Results and conclusions of the developing phase 51 perforated paper which controls the movement of the keys in a pianola or similar instrument. This approach was attractive, because, as a computer structure, the piano roll is nothing more than a matrix or a dictionary. As the approach seemed attractive we decided to start exploring this path and we starting creating the system to handle piano rolls and convert them into handleable music objects. Notation On the notation field, we also tried different approaches, such as ABC notation. The use of ABC notation to fill the models were discarded due to the previously explained fact concerning the Network Capabilities and its decrease when using no numerical values. ABC notation was attractive at the beginning due to the simplicity and the familiarity of the notation. The notation allowed simple CSV modifications to be performed as well as many other operations. As can be seen, many paths leaded to empty roads during the developing of the Music Composers, but from all paths we learned something new, and we do not discard to use them later on in the future work. All approaches taken were valid, but we only kept the ones we thought, had possibilities to serve our aim. 4.4. Results and conclusions of the developing phase During the testing phase of the developed models, some problems were faced and some conclusions were deducted from the results. We can recall the next conclusions depending on the model being tested: Neural Networks Composition Time: Given the complexity of the process in computational terms, the Neural Network returned good results when trained for a long time. Even though the training time highly improved the result, not much difference was achieved when the time overpassed six or seven hours. 58 Chapter 5. Conclusiones y Trabajo Futuro 5.5. Aplicaciones en el ámbito educativo Tanto este proyecto, como las técnicas utilizadas y los resultados obtenidos resultan relevantes para los entornos educativos de cara a enseñar música a estudiantes de diferentes edades. Dado que la generación de música mediante los métodos utilizados explotan los factores creativos vinculados a la música, al mismo tiempo que incentivan la creatividad y la curiosidad por los entornos de desarrollo de software y tecnologías emergentes. Como se puede observar, mediante las líneas definidas como trabajo futuro, el espectro de aplicación de este sistema compositivo, así como las técnicas utilizadas, dejan abierto un amplio abanico de posibilidades para la creación de nuevos sistemas y aplicaciones en diferentes ámbitos, contribuyendo así a la divulgación de las técnicas generadoras y teoría y generación musical. Chapter 5 Conclusions and Future Work The work developed during the months of investigation and implementation of the models allowed me to evidence the great possibilities that the music generation field has to offer, as well as digging into the developed AI-related techniques. The used techniques and the different implementations developed and tested, which, as already said, have attached their own already-specified advantages and disadvantages, allowed me to evidence the possible applications that those techniques offer, bearing in mind their nature. The research phase of the project, although sometimes difficult due to the early stages on which the techniques are, and the amount of information related, allowed me to see the growing process and continuous improvements these techniques are subjected to. These facts, as well as the continuous learning and improvement process have been greatly satisfying. The music composition generated throughout the execution of the models, even though can not be considered to be perfect in musical terms, can be considered as a tangible proof of the possibilities that the Markov Models and Neural Networks have to offer. In this final chapter, I will expose the main ideas and uses these techniques can be subjected to, beyond the mere music generation process. 59 60 Chapter 5. Conclusions and Future Work 5.1. Composition for theatre improvisation One of the use and improvements that I would like to see in the following years to come, is the use of these techniques when creating the soundtrack of a conversation had by two people. In the last few years, there has been an increasing tendency of the production of theatrical productions based on the improvisation of two or more people. In those production, the use of music is subjected to the flow of the conversation, and by nature unpredictable and free. The model must be able to "listen" the show and improvise, along the actors, possible music accompaniments that fits the subject brought to action. This composition must be able to change along the way throughout the developing of the play, and adjust music parameters such as dynamics, to fit the play. 5.2. Classical pieces completion Another possibility that arises from the use of these models, is the completion of classical music pieces that, due to different factors, never got to be finished. This implies that, the model, starting from the studying of classical compositions, must be able to, given a part of an uncompleted song, complete the given piece, following the tendencies and characteristics of the original composer. This technique would be aplicable on pieces such as the Mozart Requiem, in the Aria for choir «Lacrymosa», which, he was not be able to finish. Additionally, this technique could also be applicable to the Schubert’s Eighth Symphony, popularly known as the «Unfinished» or the Bruckner’s «Nine Symphony», which, as «Lacrymosa» were not finished by the original composer. Big tech and technological companies have be focused on the developing and arising tendency of these techniques. This is the case of Huawei (Huawei, 2019), which tried to finish up the «Unfinished Schubert’s Symphony». Even though the results have not been the best ones expected (Mantilla, 2019). , the fact that companies such as Huawei interest themselves on the developing of these techniques, certainly shows that they are facing the right way and it is worthy to keep researching. 5.3. Composición para películas y videos 61 5.3. Composición para películas y videos As it has been explained in the section devoted to the «State of the Art», the german composer Hans Zimmer, decided to invest and get on board on a project focused on music generation from computer generated techniques throughout the VJAM platform. This techniques, as Hans Zimmer, could be focused on the creation for music for films, videos and cartoons. In this same research line, I would like to recall the studied brought up in Muphic (Johan Bertrand, 2012), a composition system based on static images from which a music is composed that matches the images. 5.4. Voice assistant integration During the development of the project there has been an unsuccessful approach to the viability to include the models into a skill created for voice assistants such as Google Home and Alexa. This skill would be able to compose music using these assistants. The increase in the use of these assistants have grown exponentially in the last few years, and they turned out to be an interesting field for the development of skills to enrich those systems. These skill would be capable, for instance, of dynamically composing music pieces such as lullabies or ambient music to encourage concentration. This creations, would, at the same time, exploit the "Mozart Effect"(Jenkins, 2001), which states that the continuous hearing of musical pieces composed by Mozart, given their nature and dynamic composition, causes improvements in the long terms for people’s development capacities. From the reiterated studied of Mozart pieces, the skill will be able to generate music based on trendings used by Mozart in his compositions and, therefore, impulse the, so called, "Mozart Effect" in the listeners. 5.5. Applications in the educational environment This project, as well as the used techniques to develop it, and the obtained results, turned out to be relevant for educational environments in order to teach music to students of all ages. 62 Chapter 5. Conclusions and Future Work Given the fact that music generation throughout used methods exploit and try to imitate the creativity factor inherent in human beings, at the same time, they expect to impulse the use of emerging technologies and software development activities. As it has been proven, throughout the different activity lines defined for future word, the spectrum conceived for the application of music generation systems, as well as the techniques used to develop them, leave open a huge range of possibilities for the creation of of new systems and applications in different fields, including popularization of generation and composition musical theory and techniques. Annex A 5.6. Annex A: Output Sample The following music sheet is a sample of the outputs generated by the system 63                  2           3               4          6      7          8           9          10             11           12         13   64 Chapter 5. Conclusions and Future Work 2          15  Music engraving by LilyPond 2.18.2—www.lilypond.org 5.6. Annex A: Output Sample 65 Bibliography If I have seen further than others, it is by standing upon the shoulders of giants. Isaac Newton Antonio Caparrini López, L. P. M. Genre classification of electronic music. 2017. Aucouturier, J.-J. and Pachet, F. Representing musical genre: A state of the art. Journal of New Music Research, Vol. 32(1), 83–93, 2003. Basharin, G. P.,Langville, A. N. and Naumov, V. A. The life and work of aa markov. Linear algebra and its applications, Vol. 386, 3–26, 2004. Bengio, Y. Rmsprop and equilibrated adaptive learning rates for nonconvex optimization. corr abs/1502.04390, 2015. Boylan, J. H. Hal in" 2001: A space odyssey": The lover sings his song. Journal of Popular Culture, Vol. 18(4), 53, 1985. Braga, M. The verbasizer was david bowie’s 1995 lyric-writing mac app. 2016. Bryner, B. The piano roll: a valuable recording medium of the twentieth century. PhD thesis, Department of Music, University of Utah, 2002. Carr, C. and Zukowski, Z. Generating albums with samplernn to imitate metal, rock, and punk bands. In Proceedings of the 6th International Workshop on Musical Metacreation (MUME 2018). 2018. 67