Live Coder in the Loop: Performing with an Autonomous Agent

Live Coder in the Loop: Performing with an Autonomous Agent Elizabeth Wilson (performing/creating as "digital selves") Creative Computing Institute University of the Arts London London, UK Abstract Live Coder in the Loop: Performing with an Autonomous Agent in TidalCycles is a proposed performance wherein collaborative musical interaction is explored with an agent system in the live coding language TidalCycles. The performance will demonstrate a hybrid approach where the human can accept and reject suggestions from an autonomous agent, and the live coder has explicit control over the suggestions from the agent. The agent can generate patterns that are modifiable based on changing affective states to model human emotion. Specifically, this performance interrogates the practical and aesthetic dimensions of co-creating with a code-generating AI agent, embodying the paper’s call for systems that prioritise reflection, shared agency, and liveness. 1 Introduction This proposed performance presents Tidal-MerzA—a novel system designed for collaborative performances between a human live coder and a machine agent in the context of live coding, specifically focusing on the generation of musical patterns. By integrating affective modelling with computational generation, this system leverages reinforcement learning techniques to dynamically adapt music composition parameters within the TidalCycles framework, ensuring both affective qualities to the patterns and syntactical correctness. Live coding is a term used to refer to performers creating art by writing computer code, usually in front of an audience (Collins et al., 2003). In live coding, computer language is the primary medium for notation and describing the rules with which to synthesise artworks, in this case we consider the case where the output is musical pattern. The practice of live coding places a strong focus on the elements of liveness, embracing error, the use of random processes and clear mappings between syntax and output. The TidalCycles live coding language is used to create autonomous patterns by Tidal-MerzA. TidalCycles is an expressive language, known for its flexibility and versatility in creating complex structural ideas, through its functional programming style and "mini-notation" syntax (McLean and Wiggins, 2010). In Tidal-MerzA, these "mini-notation" strings—symbolic groupings to denote wider functions in TidalCycles are a crucial aspect of representing and generating new patterns . For a previous iteration of this performance, performed at St Mary’s Church, Brighton: https: //www.youtube.com/live/d0RMUqcbhmQ?si=n8IaWeZ8QcKpOFbm&t=8036 An audio composition extract from the performance can be found here https://on.soundcloud. com/Qav93RYJoUYJg7s57. Proceedings of the 6th Conference on AI Music Creativity (AIMC 2025), Brussels, Belgium, September 10th-12th Figure 1: Live performance with the Tidal-MerzA autonomous agent system described. Image credit: Jonathan Reus 2 Methods Taken in Developing The Work The development of Tidal-MerzA involved a multidisciplinary approach, blending computational creativity, affective computing, and live coding practices. The system was built upon the TidalCycles platform, a domain-specific language for live coding of music, which provided a flexible and expressive environment for musical pattern generation. The machine agent component was trained using reinforcement learning, with the goal of optimising musical parameters based on affective goals (such as valence and arousal levels). To achieve this, a reward function was defined that evaluates generated musical patterns according to affective estimations derived from pre-trained emotion recognition models, while simultaneously maintaining musicality and syntax validity. The training process included iterative interactions with human coders in simulated live sessions, where feedback was collected both explicitly (via annotations) and implicitly (through musical choices and timing). For this hybrid system, two agents are consecutively built to attempt to capture all the dimensions outlined in the affective model-ALCAA. An overview of how these are implemented are outlined in figure 2. First, reinforcement learning (RL) techniques are used to dynamically adapt the parameters generated by the affective model within the flexible framework of TidalCycles. This framework enables the model to harness TidalCycles’ extensive library of functions and patterns to generate music compositions that not only encapsulate desired emotional attributes but also adhere to the syntactical correctness of TidalCycles code. Secondly, specific mini-notation strings are produced that harness TidalCycles internal parsing of short-hand events. In MerzA, the RL agent’s actions correspond to the selection of musical elements within the TidalCycles framework. These actions are guided by a reward mechanism that evaluates the alignment between the generated music’s affective attributes and the target affective states defined by the ALCAA model. Through trial and error, the agent refines its decision-making strategies, gradually learning which musical elements and TidalCycles functions to employ in order to evoke specific emotional responses. RL is particularly well-suited for this task because it allows the agent to 2 learn and adapt in a dynamic environment and optimise its actions. In particular, modelling equations for different musical structural parameters were defined, namely: rhythmic structure, sound level/perceptual loudness, and tempo, modality, pitch register and pitch contour. Through the creation of two agents, these parameters are incorporated, preserving the original equations in this new mode of generation. In this proposed performance, the live coder will be engaging in a constructive dialogue with the Tidal-MerzA system, shaping musical structure in real time alongside the machine agent. The live coders role will involve curating and responding to the agent’s outputs, steering the direction of the composition through code, and influencing the affective trajectory of the performance. Rather than functioning as a fixed controller, the live coder will be operating in a co-creative capacity—adapting my coding strategies in response to the machine’s decisions and emotional cues. Figure 2: Live performance with the Tidal-MerzA autonomous agent system described 3 Technical Requirements 3.1 Technical Requirements from the Venue 3.1.1 Sound an Technical 1. PA System: • 2 or more coaxial speakers, with frequency range minimum 5017000Hz • 1 or more subwoofers • Capable of stereo sound reproduction 3.1.2 Lighting and Visuals 1. Projector 2. Projection Screen 3.1.3 Cables 1. HDMI cable (>10m, or alternatively this must be long enough to connect hdmi output from the artist’s laptop to the projector) 2. 2x XLR cables to connect from PA to Artist’s Mixer or Interface 3 3.2 Artist Will bring 1. Laptop 2. Audio interface (> 4 input, >4 output) 3. 8 Channel Digital Audio Mixer 4. Any additional hardware synthesisers to be used (1x Korg Volca FM, 1x Moog Werkstatt) 5. USB-c to HDMI and USB adapter 6. Additional 3.5mm to phono cable 7. Additional Jack cables 3.3 Additional Requests • Timing should be allocated for load-in/soundcheck • Organisers should aim to ensure diversity in performers. Acknowledgments and Disclosure of Funding This work was supported by EPSRC and AHRC under the EP/L01632X/1 (Centre for Doctoral Training in Media and Arts Technology) grant a References Collins, N., McLean, A., Rohrhuber, J., and Ward, A. (2003). Live coding in laptop performance. Organised sound, 8(3):321–330. McLean, A. and Wiggins, G. (2010). Tidal–pattern language for the live coding of music. In Proceedings of the 7th sound and music computing conference, pages 331–334. 4