Listening is Better Understood with: Minds, Milieux, Machines & Musics

Listening is Better Understood with: Minds, Milieux, Machines & Musics Etienne Thoret1[0000→0002→8214→6278] and Vincent Lostanlen2[0000→0003→0580→1651] 1Aix-Marseille University, CNRS, INT, UMR7289, F-13005 Marseille, France, https://etiennethoret.free.fr/ 2Nantes Université, École Centrale Nantes, CNRS, LS2N, UMR 6004, F-44000 Nantes, France, https://audio.ls2n.fr Abstract. This paper proposes a conceptual framework for understanding listening as a four-way interaction in-and-between minds, milieux, machines, and musics. Building on cognitive, ecological, computational, and cultural approaches to sound, we explore how each of these elements contributes uniquely to auditory experience and interpretation. Minds bring perceptual, cognitive, and emotional grounding; milieux provide ecological and sociocultural contexts; machines o!er tools for sensing, modeling, and synthesis in sonic environments; and musics embody structured forms of human expression and shared aesthetic meanings. Through pairwise and three-way analyses, we identify the pitfalls of excluding any one component—leading to disembodiment, ine"ciency, or cultural disconnection. We advocate for a transdisciplinary synthesis that situates listening within networks of perception, technology, and environment. Our goal is to recenter listening as a reflexive, situated, and multiscalar practice across disciplines. Keywords: Sound ecology - Soundscapes - Auditory perception - Musics -Ecoacoustics-Machinelistening 1Introduction Pauline Oliveros’s Sonic Meditations (1971) are a series of artistic performances which are “based on the structure of human attention”, with textual instructions such as “walk [at night] so silently that the bottom of your feet become ears” [ 22 ]. In a 1999 essay, Oliveros pointed out that Sonic Meditations “gave [her] work a whole new direction”, letting her “understand just how important listening is to creative music making” [ 23 ,p.32].Yet,forOliveros,listeningskillsgowell beyond the realm of musical creativity as classically understood. They are not only “the key to performance” [ 23 ,p.37]and“thefoundationofmusicianship”, but “the basis of all culture” (emphasis is ours). Shortly put: “how we listen creates our life” [ 23 , p. 33]. Thus Oliveros goes on to enumerate the implications of her practice onto various occupations: “if you are an artist, listening leads you to your material and to shape the material. If you are a scientist, listening leads you to theory and experiment” [23, p. 51]. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 480 2E.ThoretandV.Lostanlen We are two early-career scientists who recognize the value of listening in our research, both theoretical and experimental. At the same time, we routinely experience the visuocentrism of academia: “hearing”, Oliveros writes, “has not seemed as important to scientists and technologists as seeing” [ 23 ,p.40].Scientific inquiry hinges on written and graphical representations (more so than oral and auditory representations) [ 14 ] which have established specific norms of objectivity as part of what Lorraine Daston and Peter Galison have called “epistemologies of the eye” (more so than of the ear) [ 7 ]. In this paper, we advocate for a re-centering of listening as a mode of inquiry in its own right—one that bridges perception, environment, technology, and culture. 2 Elements 2.1 Minds The treatise of Hermann von Helmholtz, On the Sensations of Tone (1863), is a landmark in the history of psychoacoustics, understood as the science of auditory perception [ 15 ]. Helmholtz established a distinction between physiology versus psychology; “the material ear of the body” versus “the mental ear of the imagination” [ 32 ]. Such dualism has the clear merit of rejecting the “mathematism” of Enlightenment scholars (Rameau, Euler), who largely disregarded the role of the listener in the perception of musical harmony [4]. Since the Sensations of Tone,ourunderstandingof“thematerialearof the body” has greatly progressed, thanks to ever-more-precise measurement protocols and advanced numerical modeling techniques. The concept of critical band, for example, was introduced by Feldtkeller and Zwicker’s The Ear as a Communication Receiver (1956), a treatise whose cybernetic inspiration is clearly asserted in the title. Compared to the time of Helmholtz, psychoacoustics research in the midXXth century emphasized the importance of threshold e!ects and nonlinearities in the physiological processing of complex tones. Yet, at its core, it remained aligned with the postulate of Helmholtz; namely, that “the material ear always does exactly the same thing, what the mathematician does with Fourier’s theorem: [...] it analyzes [complex] waveforms as a sum of simple waves”. Under such postulate, the ear is idealized as a kind of open-loop and bottom-up system, leaving little room for top-down feedback and contextual factors. Crucially, the belief in this idealization has impoverished the design of auditory stimuli in psychoacoustics research: as of today, most classical experiments rely on simple combinations of electromagnetic sources, such as pure tones and narrowband noise bursts [ 37 ]. Looking back, the cybernetic approach to psychoacoustics has accomplished a lot for the first half of Helmholtz’s program, dedicated to “the material ear of the body”; but relatively little for the second half, about “the mental ear of the imagination”. Listening through minds refers to how soundscapes are experienced, interpreted, and acted upon. A longstanding line of research in auditory perception has Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 481 Listening is Better Understood with: Minds, Milieux, Machines & Musics 3 investigated the recognition of sound sources as an ecological process, grounded in invariant properties of acoustic events and the a!ordances they imply. This approach, initiated by Gibson [ 11 ]andexpandeduponbyGaver[ 9 , 10 ], emphasizes the direct perception of events rather than abstract acoustical features. Subsequent works has explored the perceptual cues and multimodal interactions that support source identification in complex environments [20,2,18]. Bridging ecological and reductionist paradigms, Bregman’s theory of Auditory Scene Analysis [ 5 ]introducedtheideathattheauditorysystemorganizessound into perceptual streams based on heuristics such as continuity, similarity, and temporal proximity. While grounded in the tradition of psychoacoustics, Bregman’s framework preserves a commitment to the ecological complexity of auditory environments, making it foundational to contemporary soundscape studies. These investigations have led to a twofold modeling tradition. On the one hand, high-level ecological models aim to identify invariants that are stable across exemplars and contexts—i.e., the "what" of the auditory scene. On the other, more recent low-level models have drawn on auditory texture analysis (e.g., McDermott and colleagues [ 21 ]) to describe statistical properties of auditory environments without necessarily attributing them to discrete sources. Complementing these behavioral and psychophysical approaches, cognitive neuroscience has provided deeper insight into the neural substrates of auditory recognition. Studies by Formisano [ 29 ], Giordano [ 13 , 12 ], Lemaitre & Heller [ 18 ], and Robert [ 27 ], among others, have investigated how the brain encodes sourcerelated information, mediates auditory attention, and integrates multisensory cues in realistic environments. Together, these perspectives converge on a central challenge: understanding how minds make sense of complex, naturalistic sonic environments under ecological constraints. 2.2 Machines In a recent article, James Parker and Sean Dockray have surveyed the sales pitch of some tech companies which develop machine listening products [ 24 ]: “cover all possible sounds”, “give all machines a sense of hearing”, “intelligent sound recognition everywhere”, “curing our machine deafness”, to name a few. They conclude that “the field paradoxically imagines itself in a state of perpetual immaturity, even as it proliferates, and its influence accrues”. Here, we concur with the authors: it is politically dangerous to entertain the claim that “the entire world of sound [is] is computationally knowable”. Rather, it is prudent to retain the word of caution from composer Robert Rowe, written in 1992: To say that building a listening capability into a computer program can enhance its musicality is not to imply that there is a generally agreed theory of what music listening is. Many competing versions of the cognitive process of listening exist, within and among fields specifically concerned with the question, such as music theory and music cognition. Building a machine listener demands making choices about the matter and means dealing with the consequences of those choices in live performance. [28] Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 482 4E.ThoretandV.Lostanlen Said otherwise, the potential diversity of machine listening systems is irreducible and value-laden from the earliest stage of engineering and through the whole life cycle of production, maintenance, and dismantling. In recent decades, the proliferation of machine learning has transformed the landscape of machine listening. Rather than designing auditory features by hand, researchers now often rely on data-driven architectures—ranging from convolutional and recurrent neural networks to attention-based models—to learn representations directly from audio. These models underpin current advances in automatic speech recognition (ASR), environmental sound classification, and musical information retrieval (MIR), but also raise epistemological questions: what kinds of auditory invariants are discovered through training, and how do they relate to human perception? Bridging these traditions, machine listening emerges not only as a technical challenge but also as a theoretical endeavor. Contributions such as scattering transforms and time-frequency scattering networks ([ 1 ]) illustrate how mathematical priors can formalize perceptually relevant operations, combining the rigor of signal processing with the adaptability of learning-based approaches. Similarly, di!erentiable auditory models, such as cochlear models trained via backpropagation, suggest new possibilities for integrating domain knowledge into end-to-end systems. Beyond algorithmic architectures, the machinic dimension of listening also includes the physical and infrastructural apparatuses involved in sound capture and processing. Microphone arrays, embedded sensors, mobile phones, and cloud platforms all mediate the act of listening, shaping what is heard and how it is interpreted. Thus, machines are not only listeners—they are also producers of listening conditions, encoding within their design the a!ordances, biases, and ontologies of their makers. 2.3 Milieux Listening through milieux shifts the focus from internal mechanisms or technological systems to the environments in which sounds unfold, propagate, and are socially and biologically situated. The French term milieu captures a broader ecology than its English cognates “environment” or “context”. In sonic terms, this includes the physical properties of places, the biological organisms that inhabit them, and the cultural practices through which sounds are produced and understood. Historically, the notion of sonic milieu finds one of its earliest articulations in the work of Pierre Schae!er, whose Traité des objets musicaux framed listening as a phenomenological engagement with sound objects abstracted from their sources. Yet Schae!er’s focus on reduced listening contrasts sharply with the ecological turn in later acoustic thought. In parallel, an acoustic namesake, Murray Schafer’s soundscape studies reintroduced environment as an active player in listening, highlighting the socio-cultural and ecological dimensions of sonic experience—urban noise, natural ambiances, and the evolving acoustic signature of human life. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 483 Listening is Better Understood with: Minds, Milieux, Machines & Musics 5 This ecological reorientation has since deepened into the interdisciplinary field of ecoacoustics, where sound is used as a proxy to monitor biodiversity and environmental change. Drawing from forerunners such as Bernie Krause, Bryan Pijanowski, Stuart Gage, Jérôme Sueur and Almo Farina, ecoacoustics positions the soundscape as a dynamic index of ecological health [ 25 , 33 ]. Methodologically, it blends field recording, acoustic indices, and long-term monitoring to study how species, habitats, and ecosystems interact through sound. More recent work has refined these tools to disentangle biophony (biological sound), geophony (nonbiological natural sound), and anthropophony (human-made sound), enabling nuanced assessments of ecological disruption and resilience. At the same time, the sonic milieu is not purely natural—it is also infrastructural, political, and historical. Urban environments, transportation systems, and industrial activities all imprint their own acoustic signatures onto everyday life. Studies in sound studies and anthropology have interrogated how sonic territories are organized, contested, and policed—raising questions of acoustic justice and the right to silence. These perspectives remind us that milieux are not neutral containers for sound but are themselves shaped by systems of power, access, and design. In sum, the concept of milieux invites us to listen across scales: from the microecology of a forest floor to the macrostructures of urban planning, from the behavior of a single species to the sonic imprint of human presence. It situates listening within the flux of environmental relations, foregrounding the intertwined dynamics of sound, space, and life. 2.4 Musics In his 2020 book From Music to Sound,MakisSolomosdevotesachaptertothe formation of the concept of listening in XXth century Europe and North America [ 31 ]. The practical use of this concept by di!erent composers, past and present, o!ers a fertile ground for scientific inquiry. Among them, the most recognizable figure is perhaps John Cage, whose silent piece 4’33" radically subverts the classical dualism between the production and the reception of artworks. In this sense, some of John Cage’s pieces resemble Marcel Duchamp’s readymades : in both cases, the viewer or listener is invited to recenter their aesthetic experience onto bare sensations, while purposefully disregarding external determinations such as provenance or function. According to Solomos, enjoying the music of Cage requires to reform one’s prior conceptions of what makes a sound musical, and to learn a di!erent way of listening. The same ambition is present in Pierre Schae!er’s 1955 Treatise on Musical Objects,hailedasafoundingtextformusique concrète.Underthenameof acousmatic art, Schae!er proposes to foster the practice of listening to sounds via tape recorders, without visual or textual reference. This practice takes inspiration from the phenomenological !¨΅῭% (épochè), a concept by Edmund Husserl which consists in mentally putting the natural world “between parentheses”, so to speak, thus suspending any belief or judgment about spatiotemporal existence. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 484 6E.ThoretandV.Lostanlen Another form of critical listening practice which is based on musical composition is found in the computer-based synthesis of auditory illusions. The fractal sounds of Roger Shepard, Jean-Claude Risset, and Diana Deutsch o!er an awe-inspiring demonstration of the distinction between instantaneous frequency and pitch. On a related note, it is particularly significant that Risset was capable of expressing certain uncanny and dream-like sensations—e.g., an infinite fall in Computer Suite for Little Boy (1967)—by employing sounds which also served as auditory stimuli for psychoacoustic studies. It suggests that listeners may recognize an aesthetic value in auditory illusions; and more generally, that computer science research and computer music research can inform each other. 3 Pairwise interactions 3.1 Minds and Machines The dialogue between minds and machines has given rise to the interdisciplinary field of machine listening, where computational models are designed to emulate or extend human auditory capacities. Drawing inspiration from auditory neuroscience, signal processing frameworks have increasingly incorporated biologically motivated architectures—such as spectro-temporal filtering, auditory filterbanks, or modulation-based representations—to process sound in ways that approximate perceptual relevance. While some of these developments (e.g., scattering networks or cochlear models) have emerged independently of direct neurophysiological inspiration, they often converge with findings from auditory cortex studies, including work by Shamma and others. Contributions in this direction, such as those by Lostanlen [ 1 ], demonstrate how signal processing representations can benefit from embedding auditory priors rooted in perceptual and neural plausibility. Conversely, machines have also become instrumental in advancing our understanding of the auditory mind. Computational models are increasingly used not just as tools for classification or feature extraction, but as testable instantiations of cognitive and neural hypotheses. For instance, McDermott and collaborators have employed generative models to explore how listeners internalize auditory statistics [ 21 ], while Giordano and colleague [ 12 ]hasinvestigatedhowmachinederived features relate to perceptual and neural data in naturalistic listening tasks. These reciprocal influences—designing machines that hear like humans, and using machines to reveal how humans hear—underscore the growing synergies between computational and cognitive approaches to sound. 3.2 Minds and Milieux The relationship between minds and milieux occupies a central place in contemporary cognitive science, particularly in phenomenological, ecological and enactive paradigms that reject Cartesian separation between agent and environment. This emerging intersection finds resonance in both auditory cognition, philosophy and social epistemology. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 485 Listening is Better Understood with: Minds, Milieux, Machines & Musics 7 In auditory perception, the concept of an acoustic niche provides a powerful framework to understand how auditory systems—human and non-human—have evolved in relation to their milieux. Models of auditory processing, such as time-frequency decomposition, source segregation, or spectro-temporal receptive fields, can serve not only to interpret sounds but to characterize the milieu itself—revealing how it a!ords certain percepts while occluding others. Thus, auditory minds are not merely passive detectors of sound, but co-constructors of the perceived environment. Conversely, studying milieux, particularly ecological and social soundscapes, o!ers novel hypotheses about the organization and plasticity of auditory minds. For example, shifts in urban sonic environments, mediated by technological infrastructures and social norms, can exert cognitive and a!ective pressures that influence attentional dynamics, memory encoding, or even long-term neural adaptation. Research in psychoacoustics and neuroecology suggests that minds adapt not only to physical acoustics but also to cultural sonic codes. To make these reciprocal influences intelligible, one must attend to contextual layers often overlooked in reductionist models: the social, historical, and political conditions under which listening occurs. Here, the contribution of social sciences becomes indispensable. Concepts such as listening positionality, sonic citizenship, or sound justice challenge the neutrality of hearing and draw attention to who listens, to what, and under what power structures. Thus, the minds–milieux axis is not simply a bidirectional exchange of information, but a deeply entangled process whereby cognitive architectures and environmental dynamics co-evolve. Understanding one requires interrogating the other—not just through scientific modeling, but through transdisciplinary dialogue that includes sound studies, anthropology, and political ecology. 3.3 Minds and Musics The intersection of minds and musics o!ers a privileged perspective on how the auditory system encodes, anticipates, and responds to organized sound. At the lowest level, the study of musical timbre has illuminated how spectral and temporal structures are mapped onto perceptual dimensions. Techniques such as spectro-temporal receptive fields (STRFs) [ 6 ]andjointtimeandfrequency scattering [ 1 ]haveproveninstrumentalinlinkingphysicalfeaturesofmusical instruments to their perceived identity and expressivity. Beyond signal-level analysis, cognitive neuroscience has increasingly focused on the brain’s capacity to predict and emotionally engage with music. Predictive coding frameworks suggest that musical listening involves active inference processes, in which the brain continuously generates and updates hypotheses about rhythmic, harmonic, and melodic structures. Work by Morillon and colleagues has shown how motor and auditory systems interact to support temporal prediction, highlighting the role of endogenous oscillations in aligning attention with musical events [36]. Simultaneously, a!ective neuroscience has explored how music engages emotional and reward systems in the brain. Research by Zatorre and others points Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 486 8E.ThoretandV.Lostanlen to the activation of limbic and dopaminergic pathways in response to pleasurable or emotionally moving passages, revealing the depth of music’s impact on a!ective states [ 8 ]. Together, these perspectives frame musical listening as a deeply embodied and emotionally charged act of auditory cognition—bridging sensation, anticipation, and aesthetic experience within the auditory mind. 3.4 Machines and Milieux The interface between machines and milieux is increasingly defined by the demand to monitor, analyze, and understand environmental dynamics through computational systems. In the context of acoustic ecology, machines are not only tools of measurement—they are active participants in ecological inquiry, shaping how we access, frame, and interpret the sonic fabric of a given milieu. Environmental sound monitoring has emerged as a critical method for tracking biodiversity, anthropogenic disturbance, and climate-linked phenomena across a wide range of ecosystems. This task imposes stringent technical and ecological constraints on the machines involved. Unlike traditional laboratory setups, field deployment demands portable, miniaturized, and robust systems that can operate autonomously over extended periods and under variable conditions. To address this, there has been a shift toward event-based devices and edge computing architectures that allow for real-time processing and selective data capture. These systems reduce storage and transmission requirements, enabling more ecologically sustainable sampling—especially in remote or delicate habitats. Innovations such as solar-powered acoustic recorders, embedded AI for species detection, and compressed audio representations exemplify this trend. Acentralchallengeliesinoptimizingspatiotemporalresolution:high-resolution continuous monitoring provides rich data but at the cost of power consumption, memory, and intrusiveness. Conversely, sparse or opportunistic sampling risks missing crucial events. Machines must therefore be designed not just for technical e"ciency, but for ecological fitness—their sampling strategies must resonate with the rhythms and scales of the environments they inhabit. Ultimately, the machines–milieux axis foregrounds the coevolution of technological a!ordances and ecological sensibilities. Machine listening in the wild does not simply extract data from an environment—it participates in constructing what counts as signal, event, or anomaly. The very structure of the data we collect reflects both ecological rhythms and machine logics, urging us to rethink design not as an external imposition, but as a dialogue between sensor and surrounding. 3.5 Machines and Musics The intersection of machines and musics is well illustrated by the field of Music Information Retrieval (MIR), which develops computational methods to analyze, classify, and generate musical data. Despite its technical sophistication, MIR has long faced epistemological challenges: a gap between optimizing performance metrics and achieving cognitive, perceptual, or musicological relevance. As Aucouturier & Bigand critically noted [ 3 ], MIR sometimes advances through Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 487 Listening is Better Understood with: Minds, Milieux, Machines & Musics 9 self-referential evaluation, technically rigorous yet insu"ciently grounded in the psychology and neuroscience of musical experience. Core techniques in MIR, such as spectral feature extraction, beat tracking, and deep learning architectures, excel at modeling formal structures but often neglect the embodied, emotional, and cultural dimensions of music. This disconnect is compounded by biases in training data, the abstraction of signal from context, and the implicit assumption that musical meaning can be fully inferred from audio alone. Yet machines also enable new possibilities for musical creativity and understanding. From interactive composition systems and real-time performance augmentation to large-scale analysis of musical corpora, machine listening can expand both the practice and study of music, especially when grounded in perceptual and cultural contexts. Recent approaches that integrate timbre modeling, cross-modal interaction, or neurophysiological data o!er promising steps toward such integration. Thus, the machines–musics axis demands critical reflection, not only on algorithmic performance, but on how technologies engage with the expressive and a!ective richness of music as a human experience. 3.6 Milieux and Musics A growing number of music cognition researchers is recognizing the need to account for the ecological milieu in comparative studies of human listening worldwide. Doing so is not to produce a causal necessary link from selective pressures to evolutionary advantages, as a certain (mostly North American) tradition of “adaptationist fundamentalism” in evolutionary psychology would claim. In a way that is both more nuanced and more scientifically rigorous, Gary Tomlinson has proposed to retrace the emergence of musical ability—or “musicking”, after another book by Christopher Small [ 30 ]—through anthropogenesis. “Musicking”, Tomlinson writes, “is implicated deeply in thinking-at-a-distance”, thus “push[ing] toward hierarchic levels beyond sensual stimuli”. Hence the relation between humans and their milieu elicits and maintains anumberofoperationalsequences(aconceptfrompaleoanthropologistAndré Leroi-Gourhan [ 19 ]), such as those involved in the craft of stone tools. Not only are selective pressures in a pas de deux with niche construction, but the biocultural coevolution of organisms is subject to epicycles, which are relatively autonomous compared to the inorganic constraints of the milieu. According to Tomlinson, insofar as music is a formalized cultural system, it “stands outside co-evolutionary feedback” and “feeds forward into the broader cycle”. The implications of this thesis for the semiosis of listening are the subject of Tomlinson’s 2023 book: The Machines of Evolution and the Scope of Meaning [34]. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 488