COMACROB - Instant Synthesis Through Live Coding and AI

Figure 1: A LiveLily session. COMACROB - Instant Synthesis Through Live Coding and AI Alexandros Drymonitis Department of Design and Media Cyprus University of Technology Limassol [email protected] Abstract COMACROB is the acronym of the postdoc research titled Composing for Acoustic Robots - Instant Synthesis for Computer-Controlled Acoustic Instruments Through Live Coding and AI. The COMACROB performance aims to demonstrate the advancements on instant composition in this field that have been realized through this postdoc research. The research was split in two parts, where the first part resulted in a series of short instant compositions for a Yamaha Disklavier robotic piano, and the second part resulted in a single instant composition for a MIDIcontrolled church organ. The COMACROB performance mixes these two parts, by utilizing the resulting technology that was developed during this research and combining it with a human performer on a standard piano and electronics. 1 Composing for acoustic robots Composing For Acoustic Robots - Instant Synthesis for Computer-Controlled Acoustic Instruments Through Live Coding and AI is the title of the postdoc research of Alexandros Drymonitis, funded by the Cyprus University of Technology. This research commenced in July 2024 and ended on May 31st 2025. It was inspired by research by Drymonitis (2023a) where AI was used to create single bars of music and Bach’s chorals were used as a proof-of-concept. The software used is the LiveLily system for live scoring and live sequencing through live coding by Drymonitis (2023b). A LiveLily session is shown in Figure 1. The COMACROB research resulted in two concerts. The first was a series of short instant compositions for a Yamaha Disklavier robotic piano, where eight pianists performed on it one at a time, alongside the actual robotic piano which was controlled by a computer. Details on this part of the Proceedings of the 6th Conference on AI Music Creativity (AIMC 2025), Brussels, Belgium, September 10th-12th research can be found in Drymonitis and Koutsomichalis (2025). The second concert was an instant composition for a MIDI-controlled church organ that was played both by a computer and a human performer. The aim of this research was to investigate how AI can enhance live scoring through live coding, but also how can the sight-reading capacity of a performer be augmented, by accompanying them with a computer-controlled acoustic instrument. The AI was used as a way to create music content fast, so that a steady pace could be retained throughout the performance. This was an attempt to address the issue of slow evolution in live coding performances, as pointed out by Magnusson (2011) where he states that "there is little fun in watching a stressed programmer designing algorithms for minutes before a simple sine oscillator is applied in the playback of a silly melody." By splitting this research into two parts, I was able to test the technology used in the first part in a concert setting and determine whether it performed well enough to address the questions of my research. It turned out that the AI technology needed refinement, and in the second concert, the AI changed from code derived from the Classical Piano Composer 1 to the Notochord model by Shepardson et al. (2022). The Notochord model provided a responsive tool that could provide fresh music content fast enough to keep the desired steady pace. Being an already trained model, using it liberated me from the cumbersome tasks of assembling a training dataset and training the AI model. Another aspect of my research was the ethical use of AI from a data transparency and a creative point of view. In the first concert I collaborated with four composers who provided original music to be used as a training dataset. Their names were announced both at the beginning of the concert and in Drymonitis and Koutsomichalis (2025). The Notochord model though is trained on the Lakh MIDI dataset 2 which consists of 176,581 unique MIDI files. Referring to all composers of this dataset is not possible, but a link to credit for these pieces is available 3 . Despite using a trained AI model, I used original compositions by Thanos Polymeneas Liontiris and myself to prompt it during the performance, where again the names of the composers were mentioned at the beginning of the concert. This way of using the Notochord model is called in-context learning, where the model’s reactions get closer to its prompts as the performance evolves. In addition to using original compositions, my participation as a composer in this second concert aimed at highlighting the importance of the creative role of AI users. In contrast to many online generative AI services, where users prompt AI models with a natural language to create music content, but have little to no control in the actual creative process, I wanted to stress the importance of having an active role as a creator when using AI in music. 2 The AIMC concert The COMACROB concert presented at the AIMC is a combination of the two concerts of my postdoc research, where the instrument of the first concert will be used with the AI technology of the second concert. In this performance though, the piano will not be a robotic one, but a classical one, played by a human, with live electronics taking the role of the computer-controlled part. To maintain the approach to an ethical use of AI in music, all the prompts that will be used during the performance will be part of a composition of mine for piano and electronics. Alongside the use of AI in live coding and live scoring, this performance aims to promote the use of live coding combined with acoustic instruments, as this combination has not been thoroughly explored. Mclean (2015) has explored such combinations, but live scoring is not present in his research. The COMACROB concert introduces live scoring to this combination. 3 Methodology and methods The methodology of my research is split into micro-methodology and macro-methodology. This approach was planned from the start, as this postdoc research was designed to realize two different compositions in two concerts. At the beginning of this research, the first micro-methodoloy and the macro-methodology were laid out, while the second micro-methodology was laid out after the first 1https://github.com/Skuldur/Classical-Piano-Composer/ 2https://colinraffel.com/projects/lmd/ 3https://colinraffel.com/projects/lmd/copyrights.txt 2 Stage 1: Collection of sheet music in MusicXML format Stage 2: Creation of training datasets for AI models Stage 3: Training of AI models and testing of their output Is output satisfactory? Stage 4: Rehearsals with human performers and computer-controlled instruments Are compositions satisfactory? No Yes YesNo Presentation in concert Rework on training dataset Test different querying method Figure 2: First iteration micro-methodology flowchart. iteration of this research had been realized. Figure 2 illustrates the micro-methodology of the first iteration, Figure 3 illustrates the micro-methodology of the second iteration, while Figure 4 illustrates the macro-methodology. The main difference between the two micro-methodologies lies in the assembling of a training dataset for the AI and the actual training process, where in the second iteration no such process took place. Instead of collecting pieces and assembling a training dataset, the first stage of the second micro-methodology consisted of the collection of fewer pieces that were used only for querying. The first micro-methodology could possibly roll back to two stages, depending on the performance of the AI model, while the second micro-methodology could initiate a loop within a single stage, since no assembling of a dataset or training took place. The last stage of the two micro-methodologies is the same. In this stage, rehearsals with the computercontrolled instrument and performers playing on it were realized. Even though the AI output had already been tested, this stage helped determine whether the AI-generated music content was within the playing capacity of the performer, in a sight-reading setting. Once the AI-output was tuned to fit the sight-reading capabilities of the performers, I moved on to present this project in a concert. Acknowledgments and Disclosure of Funding The research this performance is based on has been funded by the POSTDOCTORAL program of the Cyprus University of Technology. References Drymonitis, A. (2023a). Creating music scores live through interacting with a character-level text generator. 3 Stage 1: Collection of sheet music in MusicXML format Stage 2: Creation of Python script to query AI model based on sheet music Stage 3: Translation of AI output to the [ANONYMOUS] language Is output satisfactory? Stage 4: Rehearsals with human performers and computer-controlled instruments Are compositions satisfactory? No Yes YesNo Presentation in concert Test different querying method Figure 3: Second iteration micro-methodology flowchart. Drymonitis, A. (2023b). Livelily: An expressive live sequencing and live scoring system through live coding with the lilypond language. In Ortiz, M. and Marquez-Borbon, A., editors, Proceedings of the International Conference on New Interfaces for Musical Expression, pages 256–261, Mexico City, Mexico. Drymonitis, A. and Koutsomichalis, M. (2025). Composing for acoustic robots - instant synthesis for computer-controlled acoustic instruments through live coding and ai: Research in progress. Chroma: Journal of the Australasian Computer Music Association, 40(1). Magnusson, T. (2011). The IXI Lang: A Supercollider Parasite for Live Coding. In Proceedings of the International Computer Music Conference, ICMC, pages 503–506, Huddersfield, UK. Mclean, A. (2015). Reflections on Live Coding Collaboration. In Proceedings of 3rd conference on Computation, Communication, Aesthetics and X (xCoAx), pages 213–220, Glasgow, Scotland. Shepardson, V., Armitage, J., and Magnusson, T. (2022). Notochord: a flexible probabilistic model for embodied midi performance. In Proceedings of the 3rd Conference on AI Music Creativity. AIMC. 4 Stage 1: Evaluation of technology and aesthetics of first iteration Stage 2: Evaluation of technology and aesthetics of second iteration Is technology efficient? Test different technology Are aesthetics satisfactory? No Yes Use the same technology Research conclusion Figure 4: Macro-methodology flowchart. 5