DawBi: A WebSocket-Based Plugin for Semantic Dialogue Between DAW and KOBI AI

DawBi: A WebSocket-Based Plugin for Semantic Dialogue Between DAW and KOBI AI Alberto Maria Gatti Conservatorio Santa Cecilia Via dei Greci 34, Roma alber[email protected] Lorenzo Ballerini Conservatorio Santa Cecilia Via dei Greci 34, Roma [email protected]o Massimiliano Cerioni Conservatorio A. Casella L’Aquila massimiliano.cer[email protected] Stefano Giacomelli Conservatorio A. Casella L’Aquila stefano.giacomelli1992@- gmail.com Andrea Guidi Rome Fine Arts Academy Rome [email protected] ABSTRACT Lately, the rise of AI generative systems has significantly influenced academic discourse on assisted composition, reshaping research agendas and scholarly practices. While generative tools can streamline exploratory workflows, they also automate key real-time choices, spectral shaping, rhythmic articulation, and gesture timing, thereby confining the composer’s reflective agency to the post-hoc evaluation of material that has already been generated. In response to this issue, we present DawBi, a prototypical Max for Live plugin that opens a WebSocket-based, bidirectional dialogue between a composer’s Digital Audio Workstation and °’°KOBI, a web-based knowledge ecosystem that enhances creativity through semantic analysis and reflective feedback. Rather than generating music, the framework runs a real-time analytic loop: DawBi streams audio descriptors from the DAW to °’°KOBI, which hosts an annotated corpus of compositional works; °’°KOBI matches the incoming data to this corpus and returns the semantic tags of the closest musical pieces as a natural-language reply. The immediate link between evolving material and critically informed semantic descriptors prompts the composer to question, refine, and reposition the work in progress, sustaining reflective agency. This continuous and asynchronous interaction between DawBi and °’°KOBI promotes a vision of assisted composition not as automatic substitution, but as reflective practice. Here, the system is not designed to produce music, but rather expands the critical, perceptual, and epistemic affordances of the compositional process, opening up new forms of co-creation at the intersection of art, code, and listening. Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Attribution: owner/author(s). Web Audio Conference WAC-2025, November 19–21, 2025, Paris, France. © 2025 Copyright held by the owner/author(s). 1. INTRODUCTION Generative artificial intelligence (AI) is transforming how digital music is conceived and produced. Models like AudioCraft, MusicLM, AudioLM, Music2Latent [23], AudioGPT, MelGan, and Custom Conditioned Audio Diffusion (PyTorch) [15] enable sound creation from text or audio prompts, expanding the composer’s palette. Yet, their focus on automation often distances users from the formal and reflective processes central to artistic practice. Multimodal models such as SALMONN [26] perform well in classification, captioning, and audio-based information retrieval, aligning audio and text through attention mechanisms. However, these operations remain bounded to specific tasks and rarely support situated interpretation or artistic dialogue. To address this, we present DawBi, a Max for Live1plugin designed to foster critical reflection within music production workflows. Rather than producing music, DawBi facilitates real-time communication between the Digital Audio Workstation (Ableton Live2) and °’°KOBI, a knowledgebased environment for semantic interaction and reflective engagement [10, 12]. During a composition session, the composer can send textual prompts and operating commands to KOBI, alongside real-time audio descriptors. Information flows bidirectionally via WebSocket [29], enabling asynchronous feedback without interrupting the creative flow. KOBI’s responses (e.g., “the increase in spectral density evokes tension”) are presented within the DAW, fostering compositional insight and prompting revisions or new directions. KOBI integrates musicological, historical, and theoretical sources, including multimedia materials from archives such as the Research Catalogue3. Its conversational model merges perceptual descriptors, segment analysis, and conceptual associations, creating a cognitive space for shared exploration. Here, AI-assisted composition is reframed as a process of critical interaction with distributed knowledge. DawBi and KOBI form a perceptual-conceptual interface that supports 1https://www.ableton.com/en/live/max-for-live/ 2https://www.ableton.com 3https://www.researchcatalogue.net reflection, comparison, and divergence. Instead of providing answers, the system elicits questions: What does this fragment resemble? What gesture does it suggest? Where might it lead? These emerge from the interaction itself—between human intention and machine perception. The remainder of this paper is structured as follows: Section 2 reviews related work and theoretical foundations. Section 3 presents the DawBi plugin and its functionality. Section 4 details the communication architecture. Section 5 discusses implications and open challenges. Finally, Section 6 outlines future development directions. 2. BACKGROUND 2.1 Related work The convergence of AI, music analysis, and digital composition tools has generated diverse approaches to creative assistance. Systems now span generative models, analytical frameworks, and interactive feedback tools. One foundational project is Magenta, developed by Google on TensorFlow. It supports generation and analysis of MIDI sequences via models like Performance RNN and MusicVAE, which handle melodic structure, rhythmic density, and intervallic content [1]. Twilio’s tutorials extend this approach by demonstrating how to train models on specific datasets [2]. Other platforms enhance structural analysis and visualization. MidiTok Visualizer provides a web-based interface for tokenized MIDI sequences, aiding structural understanding in ML contexts [30]. Midi Miner applies tonal tension metrics (e.g., Spiral Array) to classify functional roles in MIDI tracks (melody, bass, harmony) [13]. Text-based scripting and algorithmic composition environments also contribute to the field. Opusmodus, a Lisp-based system, recently introduced Opusmodus GPT, which supports OMN code generation through natural language prompts—a method comparable to KOBI’s conversational model [18]. More recently, feedback-oriented systems like AI TrackMate have emerged, offering LLM-based evaluation of audio and MIDI content. These tools provide performance and structural suggestions (e.g., “add dynamics”), functioning as intelligent production assistants [3]. Though valuable, these systems tend to focus on generation, analysis, or task-specific evaluation. Unlike conventional systems, DawBi and KOBI foster a reflective engagement with sound, creating a space for dialogue within the compositional process. 2.2 Design Philosophy and Theoretical Grounding °’°KOBI (an acronym inspired by the Greek term koin`os b´ıos, “common life”) is a digital, web-based ecosystem designed to support creativity in artistic research [11, 19] and musical composition by fostering semantic, perceptual, and situated forms of interaction with knowledge and sound. Developed within the framework of the European HORIZON EU4ART-differences project, it is among the first of its kind to promote the recognition of artistic research as an autonomous mode of knowledge production (see Figure 1). Its theoretical basis lies in a conception of creativity as both a subjective attribute and a collective and contextual epistemic practice. Building on this foundation, the system is structured around three key directions: the valorization of collective intelligence, the use of AI as a relational and transformative tool, and the design of hybrid experiential environments capable of hosting plural forms of knowledge, selected and curated by the project’s contributors to ensure cultural relevance and critical consistency. Figure 1: Kobi Scheme 2.3 System Architecture and Functionality The °’°KOBI infrastructure relies on three main servers: a knowledge server hosting a structured knowledge base organized as a semantic graph4; a language server responsible for textual processing and encoding through open-source LLMs; and a multimedia server dedicated to the management of audiovisual content. Interaction between these modules is coordinated by a Communication Manager, which interfaces with an AI Gateway for the deployment of models such as GPT-4. The system draws from high-quality sources such as the Research Catalogue [20], an international platform for the documentation and publication of artistic research [4], transforming these materials into a multidimensional semantic database. Collected elements such as texts, images, audio, concepts and keywords, are linked through criteria of semantic and contextual affinity, generating a cognitive landscape or space of imagination that can be explored via web interfaces or augmented reality environments. 2.4 Compositional Analysis and Transdisciplinary Implications In the compositional domain, °’°KOBI adopts a two-stage methodology: segmentation and qualification. Works are divided into temporal units through audio slicing, markers, or pre-existing analyses. Each segment is then described through three informational channels: Each segment is then described through three informational channels (see Figure 1): •Musicological publications, processed using LLMs to extract recurring descriptive categories and encode them as semantic tags. •Audio analysis, based on perceptual descriptors (e.g., envelope, timbral instability, spectral density), computed in real time using tools such as FluCoMa and IrcamDescriptor. •Theoretical dictionaries, derived from models such as 4see more about Neo4j at https://github.com/neo4j-labs/ neosemantics Delalande [9], Smalley [22], and Chion [7], useful for interpreting the gestural and semantic qualities of sound. This information converges into an integrated semantic network, supported by a vector archive managed with ChromaDB5, which enables the projection of semantic embeddings and the discovery of latent relationships, perceptual analogies, and deviations. This approach aims to build a learning and creative environment grounded in a principle of co-evolution between human and system, fostering an experience of radically situated aesthetic inquiry [25]. °’°KOBI thus represents a transdisciplinary cognitive laboratory where AI, aesthetic theory, and immersive technologies intertwine to generate new modes of accessing, organizing, and transforming artistic knowledge. In a context where creativity is increasingly central to educational and epistemological debates, it opens pathways for radically new research practices—where technology does not replace human thought, but expands its expressive, associative, and transformative potential. 3. WHAT IS DAWBI? DawBi is a Max for Live plugin designed to integrate realtime semantic interaction within a digital audio workstation. It enables a continuous exchange between the composer’s creative environment, currently Ableton Live, and °’°KOBI. The system supports a reflective workflow by sending audio descriptors, text annotations, and operational suggestions to °’°KOBI and receiving the processed responses in the DAW. This exchange is implemented via a Node.js module in Max for Live, using JSON messages over a persistent WebSocket connection. User Query Audio Descriptors Audio Recording DAWBI HTTP WebSocket KOBI KOBI Answer Stored AudioFile Dataset entry to ChromaDB WebSocket Figure 2: DawBi Scheme 5https://trychroma.com/ 3.1 System Architecture and Data Transmission As shown in Figure 2, the system is designed to support continuous, responsive and non-intrusive interaction, thanks to a regular data transmission structured in JSON packets, which ensures smooth communication. This balance keeps an information channel open, feeding the backend in real time while remaining receptive to significant responses from the AI. KOBI’s reactions, in the form of semantic suggestions, linguistic transformations or divergent interpretations, are fed back into the DAW as creative stimuli, feeding a two-way process in which analysis and composition influence each other. This establishes a real dialogue, in which the system acts as an active interlocutor, capable of proposing interpretative trajectories based on what it hears. At the end of the session, the audio portion listened to and recorded by DawBi is sent to the KOBI server. The file is archived and remains accessible to the user for further listening and analysis. With explicit consent, it may also contribute to °’°KOBI’s audio knowledge base 3.2 UX and Mediated Creative Interaction Once inserted into an audio track, whether it’s a single channel, a return track or the master output, the plugin is immediately ready to receive the audio stream and interact with the user. As shown in Figure 3, The DawBi interface has been designed to maintain a simple, intuitive structure that is consistent with the workflow of composition. The plugin offers a limited but targeted set of controls, designed to ensure immediacy of use and transparency during listening, annotation and dialogue with the °’°KOBI system. The user can manage the input signal volume via a dedicated control and observe the evolution of the audio descriptors in real time through an integrated graphical display (created with Max’s jsui object), which dynamically represents the extracted descriptors. A section allows the user to activate or deactivate the listening mode (Listen), during which the system records the descriptors and constructs the semantic context. A visual indicator (status, green/red) shows the connection status of the node.script module, while a Reconnect button allows communication to be re-established in case of disconnection. The right side of the plugin is divided into two text sections: in the upper part, the user can enter their own prompt for °’°KOBI, which is sent automatically at the end of the listening phase or by clicking on the Send Prompt button; in the lower part, the dialogue generated by °’°KOBI is displayed, structured as an open and contextual response to the session just concluded. This dialogue is not a simple static output, but a reflective moment in which AI returns observations, stimuli or connections relevant to the sound material received. In addition, a button labelled Kobi Browser allows the °’°KOBI platform to be opened in the browser directly from the DAW. Within the web interface, the user can enter text prompts, listen to the audio sent via a dedicated player and view the development of the dialogue in a larger space, maintaining continuity between the local environment and the online infrastructure. Figure 3: DawBi plug-in inside the DAW during a composition workflow. 3.3 Perceptual Segmentation and Audio Descriptors as Tools for Representation and Inquiry The adoption of a perceptual segmentation and analysis module within DawBi addresses the need to mediate between the continuity of the sonic flow and its formal articulation into analyzable and reflective units. Audio segmentation, a long-standing concern in Music Information Retrieval (MIR) technologies [17], has been approached through techniques based on novelty detection, hierarchical models, or similarityand repetition-based approaches. More recently, solutions have emerged that employ multitask architectures with semantic attention, as well as graph-based methods for detecting structural discontinuities. DawBi integrates these developments through the use of FluCoMa6and IrcamDescriptors7to detect perceptual discontinuities and compute significant musical descriptors [6]. The audio descriptors computed in time and frequency domain such as Spectral Centroid, Spectral Spread, Spectral Rolloff, Loudness, Total Energy, Zero Crossing Rate, Chroma, MFCC, and pitch detection [14] are not simply acoustic measurements, but perceptual traces: clues that reflect qualities intuitively grasped by the listener, such as tension, instability, density, or articulation [21, 27] . This analytical layer constitutes a meaningful alphabet that enables reflection on gesture, temporal articulation, and timbral or dynamic profile. Cyclically sent to KOBI, the extracted information allows for a semantic and situated interpretation of the sonic material, revealing relationships between segments, suggesting continuities, evoking contrasts, or arranging sonic elements in an open and interrogative form. As Xenakis emphasizes, the formalization and segmentation of musical material are fundamental cognitive operations for activating a conscious compositional thought, capable of reflecting upon its own structure and transforma6https://www.flucoma.org 7https://forum.ircam.fr/projects/detail/max-sound-box/ tion [31]. In this spirit, DawBi aims to support the emergence of questions through the user’s interaction with the analysis: What does this fragment resemble? What kind of musical gesture does it represent? How might it evolve? Embedded within the compositional workflow, the system functions as a perceptual and conceptual interface. It invites the composer to pause and observe the material from new perspectives, to compare fragments with each other or with harmonic constellations and aesthetic categories. It is a tool for assisted musical writing not in the sense of automation, but of activation — a form of writing that opens itself to doubt, analogy, and the generative potential of disorientation. 4. COMMUNICATION ARCHITECTURE: WEBSOCKET STRUCTURE AND FUNCTIONALITY In DawBi, communication between the operating environment and the semantic backend is guaranteed by a persistent connection based on WebSocket. This protocol was chosen for its ability to maintain an open bidirectional channel, reducing the overhead associated with creating multiple connections and improving system responsiveness compared to request-response solutions (such as HTTP). During the session, DawBi sends JSON packets containing audio descriptor frames at regular two-second intervals. The format was selected for its compatibility with heterogeneous environments, its light transmission and its ability to represent hierarchical and modular data structures. In addition to descriptive frames, the plugin can send control commands (e.g. start, stop, clear) and text prompts, generated directly by the user or the interface. The regular transmission allows a constant flow of data to be maintained, which is useful both for progressive analysis of the audio content and for real-time processing by the backend. At the same time, it avoids the risk of connection congestion by distributing the load evenly. Furthermore, the alternative of transmitting the entire dataset at the end of the session would have concentrated the computational load, compromising the scalability of the system and its real-time vocation. A central aspect of the architecture is the distinction between the semantic channel and the multimedia channel. The former, based on WebSocket, is active throughout the session and manages continuous communication between the plugin and the backend. The latter, activated only at the end of the session, uses an HTTP POST request to send the complete audio file to the server. The audio file is stored and associated with the semantic metadata generated during the session, making it available for further analysis, playback or comparison within the KOBI ecosystem. Furthermore, in addition to sending data, the WebSocket channel also allows for the asynchronous reception of responses from AI: suggestions, transformations or semantic stimuli that re-emerge in the DAW in the form of interpretative and narrative materials available to the composer. 4.1 Architectural Choices and Protocol Comparison The adoption of WebSocket is an architectural decision directly related to the communicative model of DawBi. Available alternatives such as HTTP or WebRTC, present structural limitations with respect to the operational and semantic scope of the system. In the case of HTTP, each transmission requires the opening and closing of an independent connection, following a synchronous and stateless paradigm. In interactive sonic environments, this increases computational load and prevents contextual continuity, hindering smooth and coherent communication. By contrast, WebSocket allows for a persistent connection through which asynchronous real-time messages can be sent and received, maintaining the state of the dialogue and collaboration [5]. WebRTC is a powerful solution for peer-to-peer audio/video streaming [24], at the same time it introduces infrastructure complexity such as codec management, signalling, and NAT traversal, that can be challenging in distributed musical applications. In our project, the transmission of descriptors and prompts does not require media synchrony, but rather sustained semantic coherence over time. The relevance of WebSockets in web-based musical contexts is highlighted by frameworks such as WAAX, which emphasise their applicability to participatory and distributed interaction models [8]. The Plink project 8demonstrates a similar use of WebSocket for enabling collaborative music-making directly in the browser, allowing multiple users to interact in real time through a shared audiovisual environment. The Soundworks framework also relies on WebSocket as its infrastructure for distributed musical applications, where multiple clients synchronize with a central controller. This model demonstrates how WebSocket is suited not only for binary connections, but also for dynamic mesh topologies [16]. In terms of upcoming evolutions, the emerging WebTransport protocol9over HTTP/3 and QUIC opens promising scenarios: independent multiple streams, burst-friendly communication, and latency optimization on modern networks. 8https://dinahmoelabs.com/plink/ 9https://www.w3.org/TR/webtransport/ In a system like DawBi, oriented toward modularity and scalability, this technology may represent a natural transition. 4.2 Semantic and reflective communication In the model proposed by DawBi, communication between machine and composer is configured as a structured and intentional interaction. Each message, whether descriptor, prompt or response, becomes part of a temporal grammar that structures the interaction. WebSocket therefore, in addition to transporting data, becomes a semantic layer; the architecture seeks responsiveness as a means of fuelling a dialectical relationship in which artificial intelligence acts as a critical interlocutor, capable of listening, analysing, commenting and suggesting, with which the composer engages in dialogue. This approach transforms the data flow into a shared listening field, where communication is an integral part of creative practice. In systems such as the one developed by Vinay and Boulanger [28], WebSocket is used to enable collaborative performance through shared orchestral environments and the real-time exchange of scores and control data. These approaches emphasise synchronous interaction and distributed authorship, providing fertile ground for participatory composition and live coding. DawBi, on the other hand, employs the same communication protocol within a different temporality, focusing on the interpretative dimension of musical dialogue, where each message opens a space for critical distance, semantic interpretation, and conceptual reformulation within the compositional process. The choice of WebSocket, in addition to its technical efficiency, is rooted in its dialogic structure: persistent connection, asynchronous messages, full-duplex. It is this structure that allows the machine to be an agent participating in the construction of meaning. WebSocket thus becomes an operational punctuation mark, a dialogic rhythm that supports human-machine co-composition, acting both as an effective bridge between plugins and backends and as a true semantic layer capable of supporting a new mode of listening and composition. The interaction between composer and system becomes bidirectional, dialogic, based on a continuous network of segments, analysis, stimuli and feedback. Communication technology takes on the role of an epistemic component of the creative process: it allows the gesture to be perceived, analysed and reformulated in real time, and the machine to act as a critical and reflective interlocutor. 4.3 DAW integration: DawBi workflow The decision to implement DawBi as a plugin in Max for Live responds to the need to place the system within a production environment that is widely adopted in contemporary electronic and electroacoustic composition. Max for Live acts as a bridge between Ableton Live and Max MSP 10, allowing the linear workflow of a DAW to be integrated with the modular, data-flow-oriented architecture typical of Max. This makes it particularly suitable for educational and academic contexts, where Max for Live is often used as a platform for prototyping and experimentation. A key element of DawBi’s structure is the use of the node.script object, which allows Node.js code to be ex10https://cycling74.com ecuted directly within Max. This enables smooth management of network communication via WebSocket and HTTP, as well as the manipulation of complex data structures such as JSON-formatted dictionaries, while maintaining consistency and lightness in the information flow. Access to Node.js ecosystem also opens up the possibility of natively integrating asynchronous logic, semantic parsing, and interfaces with external services. Communication between Max and the Node.js module is handled via the native APIs of node.script, specifically through the use of the max.addHandler function, which allows Max commands to be associated with Node.js functions. In our case, a Max message called sendDescriptor periodically triggers the reading of the JSON dictionary, containing the audio descriptors calculated in real time. The content is serialised in JSON format and sent via WebSocket to the °’°KOBI backend. This communication scheme, based on explicit commands and shared data structures, allows full control over the information flow and ensures the modularity of the system. Each packet contains a consistent set of data: timestamps, numerical descriptors, text prompts, and session metadata, allowing the remote system to accurately reconstruct the state of the current compositional flow. In this sense, Max for Live is an ideal environment for rapid prototyping of complex instruments, allowing distributed architectures and real-time interaction models to be tested without leaving the DAW operating context. At the same time, the separation between compositional-analytical logic (Max) and communication logic (Node.js) provides a solid foundation for future evolution towards cross-platform plugins, potentially developed as standalone applications, Web Audio components or VST modules, while its reliance on web-native protocols such as WebSocket places DawBi within the broader Web Audio research agenda by enabling distributed, browser-accessible interaction models for composition. 5. REFLECTIONS ON THE DAWBI-KOBI SYSTEM After addressing the state-of-the-art models in Sections 1 and 2 where we discussed systems such as Magenta, MusicVAE [1], and Opusmodus [18], focused on symbolic or rule-based generativity, as well as tools like Hookpad and AI TrackMate, which operate on MIDI representations, we proposed a shift in perspective by introducing a different paradigm grounded in semantic feedback and perceptual analysis. Rather than presenting KOBI as an isolated contribution, we propose a reflection on the broader implications of the integrated DawBi-KOBI system for compositional practice. By combining a perceptual analysis engine (DawBi) with a textual feedback interface (KOBI), the system enables new forms of human–machine interaction based on semantic feedback. KOBI acts as a dialogic agent, supporting the aesthetic reasoning and decision-making of the composer. Unlike systems that manipulate symbolic musical material, KOBI interprets descriptors to generate feedback organized around aesthetic categories (e.g., tension, transparency, closure, motor activity, symbolic resonance). This textual interface, designed in synergy with DawBi, functions as an ’internal critic’ or ’cognitive mirror’, capable of offering multilevel readings of the musical material. It does not prescribe solutions, but encourages exploration by highlighting tensions, transitions, or metaphorical potentials within the music. Through this approach, the system opens a space for co-creative interaction that enhances, rather than replaces, the agency of the composer. The implications of this model challenge the dominant discourse of AI in music as primarily generative and autonomous, instead proposing a vision of AI as a reflective, interpretative partner. However, several open challenges remain. Among them are the need to contextualize feedback more precisely within different compositional idioms, the potential integration of the system into real-time creative workflows, and the development of methodologies to evaluate its impact on creative processes across a range of musical genres and practices. 5.1 Limitations and Outlook Some features of the DawBi–KOBI system are still experimental, particularly in the area of contextual memory management: the system does not yet maintain a persistent trace of previous sessions, which can make feedback less coherent in extended or complex compositional scenarios. Additionally, the current implementation relies on a unidirectional communication flow: although the architecture supports bidirectional exchange, interactions are always initiated by the user. This aspect remains underdeveloped and opens the door to future enhancements in which the system might achieve a degree of proactivity, for instance, initiating exchanges based on the evolving history of data and recognizing recurring patterns in the composer’s process. Another structural limitation concerns the knowledge base: while rich and multidimensional, its content inevitably reflects specific aesthetic paradigms. This may influence how effectively the system resonates with the composer’s musical practice. For instance, a knowledge base primarily built around atonal or spectral repertoire might better support non-tonal compositions, while potentially misaligning with tonal or rhythm-driven approaches. Future developments could therefore include adaptive or user-tailored corpora, increasing the system’s inclusivity across diverse musical idioms. Furthermore, the interface does not allow direct manipulation of sound or notational material: suggestions received from KOBI remain at a conceptual or textual level. This is not a limitation, but a deliberate design choice. The goal is not to apply automatic transformations, but to activate a critical and situated reflection. The composer retains full agency in interpreting and translating the system’s feedback into operational gestures within their own environment. From an architectural point of view, WebSocket has proven well suited to the asynchronous and symbolic nature of the project. However, more dynamic or multi-client scenarios may benefit from protocols such as WebRTC, which support real-time multimedia streaming, node synchronisation and distributed interaction. This evolution would enable not only semantic but also audiovisual dialogue, expanding the system’s applicability to live or installation contexts. A further direction involves the development of a multiplatform VST/AU plugin version of DawBi. This transition poses several challenges, from managing external dependencies (e.g. Node.js) to ensuring the consistency of descriptor analysis in native environments, where behavior or quality may differ from the current Max for Live implementation. The overarching aim is to retain modularity and lightweight design while broadening accessibility and integration into professional workflows. Altogether, these critical aspects delineate future pathways for system refinement, aiming at greater narrative coherence, increased interactional fluidity, architectural scalability and adaptability to heterogeneous musical contexts. 6. CONCLUSIONS AND FUTURE DEVELOPMENTS The DawBi–KOBI system is an experimental infrastructure for assisted musical composition, aimed at critical stimulation and situated reflection. The entire project stems from the desire to maintain operational continuity between the creative environment and the analytical sphere, giving the artist a space for semantic comparison that does not interrupt the compositional gesture but accompanies, destabilizes, and relaunches it. DawBi acts as a local interface between the composer and the system, segmenting the sound material and calculating perceptual descriptors in near-real time, then sending them regularly to KOBI via the WebSocket protocol. In turn, KOBI organizes and interprets these data from a musicological and theoretical perspective, drawing on a structured knowledge base (Chroma DB, dictionaries, semantic embeddings) and returning comments, evocations, and suggestions aimed at re-signifying the material produced. Although many existing systems offer powerful tools, they often follow production-oriented and self-generating paradigms, where the human subject is relegated to the role of a mere prompter and interaction is reduced to a superficial input-output process. In contrast, DawBi and KOBI offer an alternative device, not centered on automatic content generation but rather on the critical stimulation of the compositional process. The system activates a semantic and situated relationship with the sound material, linking perceptual listening with theoretical sources, musicological models and archival knowledge. The continuous transmission of segments, descriptors and prompts, conveyed via WebSocket, does not interrupt the artist’s working flow but accompanies it with an interpretative counterpoint, capable of suggesting deviations, comparisons, analogies and rethinking. In this sense, DawBi acts as a cognitive partner: a node within a dialogue between subject and environment, where computational data and artistic intuitions coexist and feed off each other. The compositional gesture is here critically examined in light of stimuli that do not close off meaning, but rather multiply it and relaunch it in a continuous cycle of questions. The architecture is modular and distributed, inspired by emerging practices in the field of decentralized intelligent music system design, but distinct in its focus on the reflexive temporality of the artistic gesture. While frameworks like Soundworks prioritize real-time coordination in networked performances, DawBi focuses on asynchronous, semantically rich interaction that unfolds over the long arc of compositional development. The combined use of WebSocket for the regular exchange of structured data and HTTP for postsession transmission of the audio file allows for a clear separation between moments of flow and moments of archive, opening up prospects for expandability and infrastructural sustainability. In these constantly evolving scenarios, the focus is on creating spaces for investigation—contexts in which technology acts as a stimulus for reflection and divergence. DawBi and KOBI operate as critical partners, capable of accompanying the artist in the construction of a more conscious, dynamic, and situated sonic thinking, in which technology does not replace subjectivity, but questions, enhances, and challenges it. Among the directions currently under consideration, several developments are being explored to expand the system’s flexibility and scope. These include: designing a multiplatform plugin version for integration into a wider range of DAWs; developing visual-temporal interfaces capable of representing compositional processes over time; investigating new network protocols better suited to asynchronous and multi-stream communication; and experimenting with remote modes of artistic interaction, in which the instrument becomes a shared medium for reflecting on individual work through distributed dialogue. In the future, closer integration between the semantic layer and the musical structure could also be explored, for example, through suggested mappings or visualization tools that facilitate the autonomous application of suggestions, without ever replacing the user’s creative agency. 7. REFERENCES [1] S. Agnew. Generating music with python and neural networks using magenta. https://www.twilio.com/blog/ generate-music-python-neural-networks-magenta, 2018. Accessed: 2025-06-23. [2] S. Agnew. Training a neural network on midi data with magenta and python. https://www.twilio.com/ blog/train-neural-network-midi-magenta-python, 2019. Accessed: 2025-06-23. [3] AI TrackMate. Ai-assisted feedback in music production. https://aitrackmate.com, 2024. Accessed: 2025-06-23. [4] H. Borgdorff. The Conflict of the Faculties: Perspectives on Artistic Research and Academia. Leiden University Press, Leiden, 2012. [5] M. Buffa, J. Lebrun, J. Kleimola, O. Larkin, and S. Letz. Towards an open web audio plugin standard. In Proceedings of The Web Conference (WWW), Lyon, France, April 2018. [6] D. Cast´an, A. Ortega, A. Miguel, and E. Lleida. Audio segmentation-by-classification approach based on factor analysis in broadcast news domain. EURASIP Journal on Audio, Speech, and Music Processing, (Article 34), 2014. [7] M. Chion. Guide to Sound Objects: Pierre Schaeffer and Musical Research. INA/BUC, Paris, 1994. English translation by John Dack and Christine North. [8] H. Choi and J. Berger. Waax: Web audio api extension. In Proceedings of the International Conference on New Interfaces for Musical Expression, pages 499–502, Daejeon, Republic of Korea, May 2013. Graduate School of Culture Technology, KAIST. [9] F. Delalande. La musique est un jeu d’enfant. In J.-J. Nattiez, editor, Les univers de la musique, pages 65–93. Actes Sud, Arles, 1993. [10] A. Giretti, D. Durmus, M. Lemma, M. Vaccarini, M. Zambelli, A. Guidi, and F. Meana. Knowledge Engagement in Art and Design Education: About the Role of AI in Creativity Education, pages 3–24. August 2024. [11] A. Giretti, D. Durmus, M. Vaccarini, M. Zambelli, A. Guidi, and F. Ripa di Meana. Integrating large language models in art and design education. In Proceedings of the International Conference on Cognition and Exploratory Learning in the Digital Age 2023, Madeira Island, Portugal, October 2023. [12] A. Giretti, M. Lemma, M. Zambelli, F. R. di Meana, and A. Guidi. Exploiting collective intelligence. In Proceedings of the 2023 GARR Conference - Saperi Interconnessi, Florence, Italy, June 2023. [13] T. Guo, D. Herremans, and T. Magnusson. Midi miner: Tonal tension and track classification in symbolic music. https://arxiv.org/abs/1912.13187, 2019. arXiv preprint arXiv:1912.13187. [14] A. Lerch. An Introduction to Audio Content Analysis: Applications in Signal Processing and Music Informatics. John Wiley & Sons, 2 edition, 2022. [15] Y. Ma, Y. Chen, Z. Wang, and Y.-H. Yang. Foundation models for music: A survey. arXiv preprint arXiv:2408.12345, 2024. [16] B. Matuszewski. Soundworks: A framework for networked music systems on the web – state of affairs and new developments. In Proceedings of the Web Audio Conference (WAC), Trondheim, Norway, December 2019. [17] O. Nieto, G. J. Mysore, C. Wang, J. B. L. Smith, J. Schl¨ uter, T. Grill, and B. McFee. Audio-based music structure analysis: Current trends, open challenges, and applications. Transactions of ISMIR, 3(1):246–263, 2020. [18] Opusmodus Team. Introducing opusmodus gpt: Ai-enhanced composition environment. https://opusmodus.com/forums/, 2024. Accessed: 2025-06-23. [19] F. Ripa di Meana, A. Giretti, A. Guidi, M. Vaccarini, and M. Zambelli. kobi: A knowledge ecosystem for research and education. Cineca, 4:69–83, February 2024. [20] C. Schipper. The research catalogue. IASJ Journal of Applied Jazz Research, 1(1), 2022. Article 9. [21] D. Schwarz and N. Schnell. A modular sound descriptor analysis framework for relaxed-real-time applications. In Proceedings of the International Computer Music Conference (ICMC), New York, NY, USA, 2010. ICMC. [22] D. Smalley. Spectromorphology: Explaining sound-shapes. Organised Sound, 2(2):107–126, 1997. [23] Sony CSL Paris. music2latent: Tools for exploring latent representations of symbolic music. https://github.com/SonyCSLParis/music2latent, 2023. Accessed: 2025-06-24. [24] S. Stickland, N. Scott, and R. Athauda. Design of a real-time multiparty daw collaboration application using web MIDI and WebRTC apis. In Proceedings of the Web Audio Conference (WAC), Trondheim, Norway, December 2019. [25] L. A. Suchman. Human-Machine Reconfigurations: Plans and Situated Actions. MIT Press, Cambridge, MA, 2nd edition, 2007. [26] C. Tang, W. Yu, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. Ma, and C. Zhang. Salmonn: Towards generic hearing abilities for large language models. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. [27] P. A. Tremblay, O. Green, J. Bradbury, and T. Moore. Teaching critical machine listening and machine learning with flucoma. In Proceedings of the International Computer Music Conference (ICMC) 2022: Standing Wave, Limerick, Ireland, July 2022. Oral presentation; Centre for Research in New Music, University of Huddersfield. [28] A. Vinay and R. Boulanger. Building web-based interactive systems with csound pnacl and websockets. In 2015 International Conference on Semantic Computing (ICSC), pages 258–261. IEEE, 2015. [29] WHATWG Working Group. WebSocket Standard — Living Specification. https://websockets.spec.whatwg.org/, 2024. Accessed: 2025-06-14. [30] M. Wiszenko, A. Papadopoulos, and G. Recio. Miditok visualizer: Tool for visual analysis of tokenized symbolic music data. https://arxiv.org/abs/2404.12345, 2024. arXiv preprint arXiv:2404.12345. [31] I. Xenakis. Formalized Music: Thought and Mathematics in Composition. Pendragon Press, Hillsdale, NY, revised edition, 1992.