Grammatical Synthesis of Sonic Forms: A Symbolic-Acoustic Framework for Creative Recomposition in Riot Spa Areal Nit Asian Tutu

Grammatical Synthesis of Sonic Forms: A Symbolic-Acoustic Framework for Creative Recomposition in Riot Spa Areal Nit Asian Tutu Ivan Eiji Simurra∗ Instituto de Artes-IA/Núcleo Interdisciplinar de Comunicação Sonora-NICS Universidade Estadual de Campinas - UNICAMP Cidade Universitária “Zeferino Vaz”, Barão Geraldo, Campinas, São Paulo, Brasil [email protected] Abstract This paper presents a symbolic-acoustic system for musical recomposition that bridges audio descriptor analysis, machine listening, and symbolic AI through generative grammars. The system integrates spectral-temporal clustering and formal grammar induction techniques: N-Grams, Sequitur, and L-Systems, to construct rule-based sonic recomposition from real-world audio. Unlike corpus-trained neural models, the system emphasizes local savvy, interpretability, and structural pellucidity, enabling compositional control at both the micro and macro levels. The resulting compositions, created through probabilistic rules and dynamic transformations, demonstrate a hybrid approach to musical creativity: one that is algorithmic yet perceptually grounded, symbolic yet acoustic. The work contributes to contemporary discussions on AI music creativity by offering a flexible, explainable, and artistically guided compositional method based on sound analysis, symbolic logic, and generative synthesis. From this project, we designed the 2025 electro-acoustic music composition called Riot Spa Areal Nit Asian Tutu which is the first creative music experiment from this system. 1 Introduction: Toward a Hybrid Intelligence in Composition Recent advances in artificial intelligence have transformed how composers engage with materials, structure, and sound. In particular, the intersection between symbolic modeling and perceptually grounded audio analysis has become fertile ground for creative exploration. While many contemporary AI music systems rely heavily on large symbolic datasets and neural architectures Yang et al. (2017); Huang et al. (2018), this work proposes an alternative, localized hybrid approach, where symbolic grammars emerge from machine listening applied to real audio samples. The system described in this paper, developed in Python and applied in a generative composition presented at AIMC 2025 , enables sonic recomposition through a three-stage process: (1) audio descriptor analysis and clustering, (2) symbolic sequence modeling through grammatical formalisms, and (3) rule-based sample manipulation and synthesis. Each step is guided by the composer’s control and algorithmic reasoning, ensuring that the system is not a black box, but rather a thin partner in creative decision-making. The title of the composition, Riot Spa Areal Nit Asian Tutu, is derived from a recursive linguistic permutation process, reflecting the compositional strategies employed in the system itself. It evokes a playful, generative logic of reordering and emergence, much like the symbolic grammars and algorithmic transformations used in the piece’s creation. This name functions as both a poetic fragment and a conceptual mirror of the underlying framework. The available audio file from the music composition can be downloaded at https://shorturl.at/rGfaZ. ∗ Use footnote for providing further information about author (webpage, alternative address)—not for acknowledging funding agencies. Proceedings of the 6th Conference on AI Music Creativity (AIMC 2025), Brussels, Belgium, September 10th-12th 2 Methods: From Sound to Symbol and Back Again 2.1 Audio Descriptor Analysis and Clustering The process begins with the extraction of spectral and temporal descriptors using the librosa Python library. Audio features list includes spectral centroid, bandwidth, roll-off, flatness, RMS, zerocrossing rate, MFCCs, and chroma vectors. These descriptors form a multidimensional feature space, capturing essential qualities of the input sounds—such as brightness, density, harmonicity, and dynamic envelope. Clustering is applied to this space using unsupervised learning techniques such as KMeans, Gaussian Mixture Models (GMM), and Agglomerative Clustering. As discussed by Herremans et al. Herremans et al. (2017), clustering offers a robust strategy for partitioning perceptually similar sonic segments, enabling subsequent symbolic labeling. Each resulting cluster is assigned a symbol (e.g., ‘A’, ‘B’, ‘C’), effectively translating perceptual features into discrete compositional units. This move resonates with prior musicological and computational paradigms. Roads Roads (2001), for example, emphasizes the importance of spectral morphology in microsound composition, where sound objects are categorized and recombined based on features rather than notes. Similarly, Wishart Wishart (1994) explores the notion of ‘acousmatic grammar’ to organize sonic materials by auditory attributes, a concept formalized here through clustering. 2.2 Grammar Induction: Sequitur, N-Grams, and L-Systems Once symbolic sequences are obtained, the system applies multiple formal grammar induction techniques to model patterns and structures. N-Gram analysis Shannon (1948) identifies frequent symbolic sequences and transition probabilities, forming the statistical basis for pattern recognition and stylistic continuity. Sequitur, as introduced by Cleary and Witten Cleary and Witten (1984), detects recursive structures and hierarchical compressions. It converts symbol sequences into grammars that reveal both repetition and structural depth. This aligns with Cope’s Cope (1996) early symbolic AI work in Experiments in Musical Intelligence, where stylistic grammar was induced from symbolic corpora. Lindenmayer Systems, a.k.a., L-Systems, originally formulated for modeling biological growth Lindenmayer (1968), are adapted here for sonic growth and transformation. By applying parallel rule substitutions, L-Systems expand short symbolic ‘axioms’ into evolving musical sequences, yielding emergent complexity and formal development. This multi-grammar architecture bridges symbolic AI and algorithmic composition, situating the system within the tradition of Nierhaus Nierhaus (2009) and Keller and Capasso Keller and Capasso (2006), while extending these ideas toward real-time audio manipulation. 2.3 Symbol-to-Sound Recomposition and Sonic Rendering Each symbol in the grammar corresponds to an audio sample, retrieved either from specific local sound database or the freesound.org online repository. For rendering, the system applies a range of transformations and effects: • Pitch shifting and time-stretching via librosa effects; • Segment slicing with random overlapping; • Probabilistic reversals, dynamic gain variation, and spatial stereo panning; • Convolutional reverb using scipy.signal to simulate acoustic environments. This synthesis method parallels Roads’ microsound principles Roads (2001), where audio is constructed from granular and segmental units. It also echoes Di Scipio’s work on sound-based composition systems that use perceptual parameters as control interfaces Di Scipio (2000). The final output is a collage of transformed samples, ordered by grammar and shaped by the interplay between analysis and rule-driven synthesis. The following Figure 1 depicts the first page from the application graphical user interaction. 2 Figure 1: First page window from the GUI interface. 3 Artistic Motivation and Contribution This project was born from an interest in making musical structure emerge from sound itself. Rather than composing with abstract pitch-duration values, the system listens to sound, categorizes it, and generates its own grammar. The composer becomes an ‘architect’ of settings, navigating between algorithmic deputation and artistic intention. This approach contrasts with deep generative models, which often obscure inner workings and rely on massive data. By contrast, the system here is plain, small-data-oriented, and interpretable. It propels the user into the process—whether artist, researcher, or listener—to co-compose with algorithmic logic. In doing so, it contributes to AI Music Creativity by proposing: • A descriptor-to-symbol pipeline that links perceptual qualities to compositional structures; • A flexible grammar engine that is modular and musically expressive; • A rendering process that prioritizes timbral richness and transformation over symbolic fidelity. Conclusion This companion work speaks directly to the AIMC 2025 theme of creativity with and through AI. It proposes a middle path between machine learning and symbolic AI, where computational creativity is structured, explainable, and musically grounded. In doing so, we designed the electro-acoustic music called Riot Spa Areal Nit Asian Tutu which is the first creative music experiment from this system. Rather than treating AI as a generator of surprises, this project treats it as a partner in structuring material, drawing from ideas of emergent form, recursive rules, and interactive listening. It embodies AI as musical intelligence—not only producing material, but understanding, shaping, and transforming it. Acknowledgments and Disclosure of Funding Use unnumbered first level headings for the acknowledgments. All acknowledgments go at the end of the paper before the list of references. 3 Do not include acknowledgments in the anonymized submission, only in the final paper. You can use the ack environment provided in the style file to automatically hide this section in the anonymized submission. References Cleary, J. G. and Witten, I. H. (1984). Data compression using adaptive coding and partial string matching. IEEE Transactions on Communications, 32(4):396–402. Cope, D. (1996). Experiments in Musical Intelligence. A-R Editions. Di Scipio, A. (2000). Sound is the interface: From interactive to ecosystemic signal processing. Organised Sound, 5(3):203–221. Herremans, D., Chuan, C.-H., and Chew, E. (2017). A functional taxonomy of music generation systems. ACM Computing Surveys, 50(5):1–30. Huang, C.-Z. A., Vaswani, A., Uszkoreit, J., et al. (2018). Music transformer: Generating music with long-term structure. ICLR. Keller, D. and Capasso, A. (2006). New concepts and techniques in algorithmic composition. Leonardo Music Journal, 16:61–65. Lindenmayer, A. (1968). Mathematical models for cellular interaction in development. Journal of Theoretical Biology, 18(3):280–315. Nierhaus, G. (2009). Algorithmic Composition. Springer. Roads, C. (2001). Microsound. MIT Press. Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27(3):379–423. Wishart, T. (1994). Audible Design. Orpheus the Pantomime Ltd. Yang, L.-C., Chou, S.-Y., and Yang, Y.-H. (2017). Midinet: A convolutional generative adversarial network for symbolic-domain music generation. ISMIR. 4