We're in this together: Expanding Interactive Sonic Spaces

This work is licensed under a Creative Commons Attribution 4.0 International License. NIME ’25, June 24–27, 2025, Canberra, Australia © 2025 Copyright held by the owner/author(s). We’re in this together: Expanding Interactive Sonic Spaces DAVOR VINCZE, Hong Kong Baptist University [email protected] MAURICE OESER, SWR Experimentalstudio Freiburg mauric[email protected] Program Notes We’re in this together is a solo electronic performance exploring novel paradigms of audience interaction through smartphones, expanding on work developed previously for the opera Freedom Collective [1] and further refined during the Exo Signals performance at Ubimus conference in Macau, where it was adapted to OTTOsonics [2] system. The performance will be a ten minute immersive sound experience, combining real-time audience input via a smartphone web app (see Figure 1) with generative and augmented sonic textures. Figure. 1. Screen displaying 3 different modes of interaction with the Web-app The web app allows audience members to directly engage with the sonic environment through their smartphones, creating spatialized, layered, and unpredictable soundscapes, enhancing the collective experience while challenging NIME '25 June 24-27, 2025, Canberra, Australia Vincze and Oeser Figure. 2. Audience using the Web-app the boundaries between performer, audience, and space. This iteration introduces new interactive features, such as audience-driven modulation and recording short audio snips, focusing on simplicity, inclusivity, and minimal latency to ensure a seamless, intuitive interface (see Figure 2). Project Description We’re in this together builds on the technical and conceptual frameworks of the Freedom Collective web-app but transposes its application from a theatre setting into performance for solo electronics. Hence, the videos provided serve as documentation of previous performances exploring similar concepts; they offer reviewers insight into the musical aesthetics and the functionality of the web-app technology but do not constitute representations of the specific project proposed here. The interaction design leverages the variability of smartphone audio latency and quality to generate distributed sound fields that resonate in real-time with the performance's central sonic themes (e.g. granulation, density, layering and spatial sound distribution). Ideally, the performance space should be set up with an immersive ambisonic sound system for supporting low-frequency content and reinforcing the performer’s electronic sounds. Audience smartphones act as distributed sound sources, enabling a shared participatory instrument. This distributed design also mitigates the need for specialized hardware, making the experience accessible across diverse demographics and venues. In addition to the sonic engagement, the interface includes simple visual prompts on participants' screens, aligning with the performance’s dramaturgy. These features encourage active yet unobtrusive engagement, ensuring that the audience's role complements, rather than distracts from, the sonic focus. Given that reviewers expressed two key potential issues – instability of WIFI in the performance venue and potential audiences with no smartphones – we will have a local router as an alternative ready, as well as a few spare smartphones for the audiences that might need that. That said, from our previous experience in a theatre with much larger audiences, we do not expect major issues with the WIFI. Furthermore, prior attempts during Freedom Collective showed that 95% of all audiences had and knew how to use their smartphone, and those that did not engage simply were not interested in engaging. We’re in this together: Expanding Interactive Sonic Spaces NIME '25 June 24-27, 2025, Canberra, Australia Performance notes Figure 3. Proposition of a hypothetical performance setup Requirements: • Stable WiFi network capable of supporting multiple simultaneous connections (we will provide our own router and create a local network as a backup) • Speaker set-up for electronic sound reinforcement (we will adapt to the possibilities of the organizer; it can ba as simple as stereo, quadro, 2nd or 3rd order ambisonics) • Mixing board (e.g. Yamaha 01V96i – optional) • Audio interface (has to correspond to the number of outputs in the given space) • Corresponding cables • Performer setup (that we provide): o Laptop with Max/MSP patch for electronics with admin’s interface for the web-app integrated o Webpage hosting o MIDI controllers o Printed papers with QR codes to be displayed in the venue. NIME '25 June 24-27, 2025, Canberra, Australia Vincze and Oeser Setup Time: • Approximately 1 hour for equipment installation and network testing • 1 hour for rehearsals to test audience interaction and calibrate sound levels The server facilitates bi-directional communication between the admin and audience devices, ensuring seamless synchronization across all connected smartphones. The system uses standard HTML/CSS/JavaScript for maximum compatibility with a wide range of devices. Audio playback is handled using the Howler.js library. Accessibility: The performance is designed to be inclusive, with straightforward onboarding for audience members via QR code scanning and responsive smartphone-friendly interaction. Media Link(s) https://youtu.be/U4TCjb_mZmw Acknowledgments The initial development of the web-app was made possible by support from SWR Experimentalstudio in Freiburg & NOperas! grant, while the subsequent research was supported by the Hong Kong Baptist University. The interactive framework draws on contributions from Freedom Collective & Exo Signals. Ethical Statement This performance complies fully with the NIME Principles & Code of Practice on Ethical Research. Audience participation is entirely voluntary and initiated via QR code access to a web-based interface. No personal data is collected, stored, or transmitted before, during, or after the event. The system does not log user identities, device information, or interaction patterns. Audio snippets recorded during the performance are explicitly triggered by participants, used exclusively for realtime processing, and permanently deleted immediately after the show. No data persists beyond the event. Accessibility and inclusion were integral to the interaction and technical design, ensuring ease of participation without requiring prior technical expertise. Ethical Standards This submission fully complies with the NIME Principles & Code of Practice on Ethical Research. Audience members must provide explicit consent to use the application upon scanning the QR code. No data is recorded or tracked before or after the performance. During the show, the only data collected consists of one-second audio snippets, which are voluntarily recorded by participants by pressing a record button for each individual sample. These samples are solely used for real-time generation of multi-layered audio during the performance and are permanently deleted at the conclusion of the event. Consequently, no personally identifiable data is collected, and accessibility considerations have been carefully integrated into both the interaction design and the technical infrastructure. References [1] Davor Vincze and Maurice Oeser, Freedom Collective: Interactive Mobile Web-App For Immersive Audience Participation. Proceedings of the International Symposium on Uniquitous Music, 2024. [2] Tomás, Enrique, Florian Goeschke, and Martin Kaltenbrunner. "Exploring Design Patterns for We’re in this together: Expanding Interactive Sonic Spaces NIME '25 June 24-27, 2025, Canberra, Australia Spatial Instruments: User-Driven Strategies, Spatialized Synthesis and Loudspeaker Topologies." Proceedings of the International Conference on New Interfaces for Musical Expression. 2024.