Sensing the body through sound

1 Sensing the body through sound Ana Tajadura-Jiménez1,2, Merle T. Fairhurst3,4 & Ophelia Deroy3,4,5 1 DEI Interactive Systems Group, Department of Computer Science, Universidad Carlos III de Madrid 2 UCL Interaction Centre (UCLIC), University College London 3 Munich Centre for Neuroscience, Ludwig Maximilian University 4 Faculty of Philosophy, Ludwig Maximilian University 5 Centre for the Study of the Senses, School of Advanced Study, University of London Abstract Music makes us dance and move, but can sounds do more for our body? We may easily think that hearing is the least relevant modality for our sense of bodily self, compared to for instance touch, vision and interoception. Yet audition provides rich information about what is happening inside and crucially outside of our bodies: we hear ourselves breathing, or our joints crack; we hear our hands clapping against each other or stroking a piece of velvet; we hear the sounds of our footsteps mixing with those of others as we go down the stairs. Rarely is there an action or event that we are involved in which is silent, and yet audition remains relatively ignored as a contributor to our sense of self. This chapter aims to correct this oversight, by highlighting the surprising but also special contributions that audition brings to our sense of self. We first show how certain sounds get specifically referred to our own bodies, through other senses and our motor actions, and come to shape how we represent ourselves. Rather than cataloguing various effects, we highlight what is distinctive or superior in the auditory contributions to our body representations, compared to other senses. Conceptually, sounds are not enduring objects but are bounded in time; the sounds we produce also occur through an interaction between ourselves and another object or surface. Audition therefore tells us about our bodies as a source of events, in relation to something else, rather than informing us about our body as an independent or stable object. Informationally, audition constantly monitors a full 360-degree space around us, automatically capturing events that happen to and because of us with high temporal resolution, while flexibly combining and segregating them from the sounds produced by others or external events. Together, these various characteristics can help us to identify the various clinical and practical applications where audition shows or can show its distinctive and important contribution to our body representations. 2 1. Introduction “Let us walk in the white snow - In a soundless space; With footsteps quiet and slow, At a tranquil pace...” The opening verses of “Velvet shoes” by the American poet Elinor Wylie connects with all of us: as we walk in the snow, the soft sounds we produce tell us about the texture of the ground, the material of our soles, but also about the weight of our own bodies. If you are not convinced, think about hearing someone coming up behind you, in that snowy landscape: couldn’t you guess how fast they are approaching? Couldn’t you also hear whether it is a heavy adult or a small child? When it comes to our own actions, the information we derive from sounds can also be bound to our own bodies, through other senses or cognitive expectations. Though stepping in the snow or onto a stage may make us very aware of the sounds we make, we constantly produce sounds as we move around, scratch an itch, push a button, or walk in the street with others. The information about events that we can obtain through audition can refer to one’s inner body (e.g., breathing, eating, heartbeat sounds; Phillips et al., 1999; Albers, 2003; Tajadura-Jiménez, Väljamäe and Västfjäll, 2008) or one’s external body (e.g., my hand touching my face or my clothes; Ho et al., 2013). This information can also refer to the bodies of other agents or objects we are interacting with (e.g., my hand touching a table or dropping a ball; Tajadura-Jiménez, Väljamäe et al., 2012; Tajadura-Jiménez, Deroy, et al., 2018) or we are not interacting with (e.g., someone else walking besides us; Li, Logan and Pastore, 1991; Tonetto and Spence, 2014), and also to the geometry and size of the space in which we are (Viaud-Delmon and Warusfel, 2014). Audition however tends to be ignored when it comes to our sense of self, partly because the sounds of our bodies are often not viewed as self-specific. That is, don’t we merely use sounds to monitor our external environment and others rather than ourselves? It is true that we are very good at telling the gender, or weight of other people’s bodies from the sound of their footsteps (Li, Logan and Pastore, 1991; Pastore et al., 2008; for a review see Visell et al., 2009) and that our brains may discount or attenuate the sound of our own steps when we walk (see Martikainen, Kaneko and Hari, 2005; Mifsud and Whitford, 2017; and work on mouse’s footsteps by Schneider, Sundararajan and Mooney, 2018), but we should not be too quick to dismiss the role of hearing in the mental representations of our own body. Here we will shed light on the importance of audition to our bodily selves by discussing key concepts relating to the role of hearing in mental representations of our body. First, some sounds get specifically referred to our own bodies, through other senses and our motor actions and through various binding mechanisms (part 2). Sounds that are actively produced by our own actions or movements can be linked to our bodies via two possible (and compatible) mechanisms: one comes from the spatio-temporal co-occurrence of body-related sensory signals such as touch and proprioception, and the other relates to the top-down expectations and role of intentions. Referral here is not a binary term (i.e., a sound is either referred to the body or not). Attribution to one’s body is a matter of probability (possibly then reflected in more or less confidence, see Deroy, Spence and Noppeney, 2016). Moreover, while we focus here on the fact that sounds are spatially attributed to the body, it is also the case that how we represent our body covers multiple representations or dimensions (body schema, body image, ownership, peripersonal space). 3 Audition then offers distinctive or superior contributions to our body representations. Conceptually, through hearing, we come to represent our bodies as fundamentally in relation to something else. As in the case of touch, which puts us in contact with something else, the sounds we produce through our actions are produced by the mechanical impact and resonance between two objects, such as the sounds produced by one’s hand tapping a surface. Through hearing, we also come to represent our bodies as sources of events, rather than independent or stable objects (part 3). Informationally, audition is also special in that it constantly monitors a full 360-degree space around us, automatically capturing events that happen to and because of us with high temporal resolution, while flexibly combining and segregating them from the sounds produced by others or external events (part 4). These characteristics help identify and explain the numerous clinical and practical applications of auditory feedback for body-related issues (part 5). 2. How sounds get referred to our bodies In order to inform our body representations, sensory cues need to be referred to our own bodies. Previous research, using mostly visual, tactile and motor cues to induce body illusions (i.e., illusions that induce a feeling of ownership over an artificial body, body-part or object - Moseley, Gallace and Spence, 2012; or the experience of one's own body importantly deviating from the configuration of one’s physical body - Kilteni et al., 2015), shows that the following factors explain why certain sensory cues come to inform body representations (Tsakiris, 2010; Longo and Haggard, 2012; Kilteni et al., 2015; Kilteni and Ehrsson, 2017): (1) Temporal congruence: the cue is synchronous or closely related in time with other sensory or motor cues (e.g., Shimada, Fukuda and Hiraki, 2009). (2) Spatial congruence: the cue is co-localised or in close proximity with one’s own body (e.g., Lloyd, 2007). (3) Semantic congruence: the cue needs to resemble naturalistic bodily cues (e.g., a hand, a face, a body) in an anatomically plausible spatial configuration (Tsakiris et al., 2010; Guterstam, Petkova and Ehrsson, 2011) 1 . (4) Agency: the cue results from one’s voluntary/willed action (e.g., Tsakiris, Prabhu and Haggard, 2006). Two things must be noted here: first, for each principle, it is possible to have a graded interpretation (the more congruence there is, the more the feedback influences body representation) rather than a strict categorical interpretation (specifying for instance the exact spatio-temporal window of integration). Second, it remains unclear which of these factors are sufficient or necessary for sensory cues to be referred to our own bodies. Studies using visual feedback as a main modality have shown, for instance, that one can embody a cane that does not resemble a body part (Armel and Ramachandran, 2003; Maravita and Iriki, 2004; Martel et al., 2016), or integrate overly distant cues to generate an illusion of a long arm (Kilteni et al., 2012). But so far, temporal congruence and agency seem to be necessary - though it is still not clear whether each is in itself sufficient (Cataldo et al., submitted). 1 Note that semantic congruence is usually defined in reference to higher-level conceptual and linguistic associations, while crossmodal congruence can exist without concepts and language (for instance in infants). There is also evidence that these two forms of congruence are neurally different (Sadaghiani, Maier & Noppeney, 2009). 4 Sounds can satisfy these various conditions: sounds can be temporally and spatially congruent with some other cues; for instance, the sounds produced when we scratch our hand occur in conjunction with tactile, or eventually visual cues. Sounds can also be semantically related to our own movements, in fine grained ways: think about the exquisite richness and variability of the footsteps sounds that a ballet or flamenco dancer can produce on stage. In many cases, we also have agency in producing the sounds that occur next to, or on our bodies: we tap our fingers on the table, we rub our hands together, we go down the stairs wearing high heels. So if sounds satisfy the conditions to be body-bound, shall we not expect them to be referred to and shape our body representations? Surprisingly, evidence for this has only been provided quite recently. But by now, studies have shown that the sounds produced when an object touches one’s body (e.g., a small hammer; Senna et al., 2014), the sounds produced by one’s actions (e.g., walking; Tajadura-Jiménez, Basia, et al., 2015) or the sounds for which we have agency in their production (e.g., sounds produced when tapping our hand on a table; TajaduraJiménez, Väljamäe, et al., 2012; Tajadura-Jiménez, Tsakiris, et al., 2015) inform the representation of our body metrics, but also other aspects of our body representations (see part 4). Many of the sounds studied in the body representation literature are those naturally produced by our body interactions. Those sounds are the best candidates to be spatially and temporally congruent with what we see or feel through touch, and to be also semantically related to our own movements: footsteps in the snow sound different from footsteps on a wooden floor, and walking on a wooden floor sounds very different if we wear high heels or slippers (Tonetto and Spence, 2014; Turchet et al., 2016). From what we know in other multisensory processes, semantic congruence however is not necessary for body illusions to occur: if we stroke synchronously with our own hand a rubber hand of a different skin colour, or even an empty space, these will start ‘feeling’ like they belong to our own body, and elicit behavioural responses similar to those we show for our own hands (Holmes, Snijders and Spence, 2006; Farmer, Tajadura-Jiménez and Tsakiris, 2012; Guterstam, Gentile and Ehrsson, 2013). The same openness to semantic violations seems to characterize sounds: studies and applications have been successful in using artificial sounds to guide or correct bodily movement. Movement sonification is an approach broadly defined as the use of sound within a body– computer interface to provide information about the bodily interaction and to help refining the activity, for instance, by accompanying the exercise of raising one’s arm with a sound that changes its frequency in relation to angular movement (Ley-Flores et al., 2021). Movement sonification has been used for informing and guiding people on their body movement during sports (e.g., swimming, Cesarini et al., 2016; or rowing, Schaffert and Mattes, 2015), general physical activity (e.g., Newbold, Bianchi-Berthouze and Gold, 2017), dance (e.g., Bevilacqua, Schnell and Fdili Alaoui, 2014), as well as in the context of physical rehabilitation (e.g., chronic pain, Newbold et al., 2016; Singh et al., 2016; or stroke, Scholz et al., 2015). More recently, non-naturalistic sounds have also been used in simpler laboratory settings, with and without agency over the movement on the part of the hearer, and shown to shape body representations. For instance, pulling on one’s own finger or having one’s finger pulled while hearing a sound rising in pitch creates an illusory elongation of that finger, analogous to an “auditory Pinocchio” effect (Tajadura-Jiménez, Vakali, et al., 2017; Nava and TajaduraJiménez, 2020). Though the production of pitch sounds while pulling on one’s finger is non naturalistic, the correspondence between pitch and space is partly grounded in general 5 statistics of the environment (Parise, Knorre and Ernst, 2014), while also influenced by cultural and linguistic practice. As such correspondences are ubiquitous across the senses, beyond audition, there is no reason to think that they are limited to sounds. Nevertheless, this is partly a new insight brought by research on sounds, as what is exploited here is not the role of semantic congruence, but the role of crossmodal congruence between pitch and spatial displacement (rising pitch corresponds to a rising movement, Deroy, Fasiello and Hayward, 2016, and may also evoke the visual imagery of a rising movement, Deroy et al., 2018). Certain results bring additional insights into the complex interplay between mechanisms of multisensory integration and of movement in altering body representation. For instance, updates in body representation can occur with cues separated in time and space: when tapping our hand on a table, hearing a synchronous tapping sound that originates at double the distance at which we are actually tapping does inform the representation of one’s arm length, although tactile and auditory cues are separated in space (Tajadura-Jiménez, Väljamäe, et al., 2012). When dropping an object from our hands, the predicted and actual outcome of this action (i.e., the ball’s release and impact), which are two events causallyrelated but separated in space and time, inform one’s perceived body height (TajaduraJiménez, Deroy, et al., 2018). This last example shows that we are sensitive to the sounds occurring immediately or in close relation with other tactile cues, or a motor action, but also rely on the anticipation of the sounds which our actions will produce. Surprisingly, outcomes of our actions, independently of ownership, lead to a body representation updating: our body representation is affected by naturalistic cues not attributed to one’s body but what this body has done, in this case the cues related to the ball’s dynamics (Tajadura-Jiménez, Deroy, et al., 2018). Last but not least, sounds can expand the insights into the role of attention in the body-binding mechanisms. In the classic rubber hand illusion, focused attention on the visual stroking of the observed rubber hand is essential for the illusion to occur, which means also that the manipulation of space and semantic congruence will be in the forefront of one’s awareness (Walsh et al., 2015). All rubber-hand illusions combine temporal congruence with some attention (to the visual stroking at least), meaning that the binding may be attention-dependent and therefore not that automatic. Many of the sounds we are exposed to in our daily life do not attract our attention, or even reach our conscious perception. However, we cannot say at the moment whether attention or conscious perception are necessary for sounds to affect our body representations 2 . How much binding depends on attention (for a review, see Noppeney, 2021) and awareness (Faivre et al., 2014; Delong and Noppeney, 2021) remains a debated issue. Here, we can only cautiously suppose that the dependence or independence also extends to how sounds get bound to our bodies, and hypothesize that they can operate, though minimally, in a more automatic manner without attention and even consciousness. However, the sounds we produce through our actions can be like “touchant-touché”, or better say “écoutant-ecouté”: as we walk in the snow, the sounds we produce tell us about us and how we are acting on the world, as much as they tell us about the world and how it is acted upon. We hear how heavy we are, with our winter clothes on, but we can also hear how soft or crisp the snow is. This dual aspect of sounds, which is also found in touch, may mean that 2 Being sound an exteroceptive input, here it may be relevant to distinguish between exteroceptive versus interoceptive attention, as these two different types of attention recruit distinct neural networks (Farb, Segal and Anderson, 2013), as well as to consider the notion of exteroceptive versus interoceptive body representations (Manson et al., 2019). We thank the reviewer of this chapter for bringing this to our attention. 6 attention to one aspect -“what sounds tell about me”- is still involved and at least important when it comes to explaining auditory effects on body representation. 3. The sounding body: How sounds provide a relational and event-based body representation Despite some nuances, the mechanisms through which sounds get referred to our bodies closely resemble the mechanisms through which visual and tactile cues get referred together to our bodies. Sounds also tend to co-occur with cues through these other modalities, in particular, with touch. Some aspects of audition are unique: our auditory perception provides us with a 360 degrees spatial array so that we can potentially hear the sound of our back being scratched, though we can’t see it. It also has very fine temporal resolution, usually considered as superior to the temporal precision of vision or touch, as well as a high sensitivity for detecting structured motion (e.g., rhythm; Hermann and Ritter, 2004). Still, two further aspects of audition make it particularly noteworthy. First, sounds from different sources can combine and merge in a single auditory stream, where they remain partly distinguishable, even across multiple categories of sound (Ogg and Slevc, 2019). If we stand, it is difficult to see both our left feet and our right shoulder: we need to move our head, looking down at our feet then lifting and turning our gaze to see our right shoulder. We also may have a distinct perception of our left and right foot standing on the ground, and our back leaning on the wall. But if we walk and clap our hands, the sounds from these different events all merge into a sort of “one man band” - analogous to what happens when different musicians play together and sound both like one and many. This analogy gives us a lead into what seems then to be specific to sounds we produce not only alone, but with others. While the visual perspective on our own limbs, and the dual nature of touch means that visual and tactile information of our own bodies are very much tagged as “our own” and will be different from the sight of another body in front of us or the touch of someone else’s hand on our bodies, the sounds produced by the self and those produced by others can merge into a unified auditory stream. Think for instance about a military march, where soldiers walk in perfect synchrony, and eventually ‘sound like one’ (see part 4). We may bring “more bodies” into our own through sounds than by means of other modalities, something which may also explain the cohesive value of collective singing found across cultures (e.g., Pearce et al., 2016). These examples introduce another specific aspect of self-produced and self-referred sounds, which comes from them being events rather than objects. Touching or staring at something is also an event, with a temporal onset and offset, and a temporal course, but the issue here concerns what we hear, that is the contents of auditory perception: we see the event of a brush stroking our hand, but we also see our hand itself, as a more permanent substrate of the event. We feel the event of the brush stroking our palm, but we also feel, through touch and proprioception, that we own the whole hand and arm on which the stroking is occurring. In other words, vision and touch provide us with a representation of our bodies as objects persisting through time, as much as they provide us with a representation of the events taking place between our bodies and other objects. Much of the literature on body representations, drawn from vision, touch and even proprioception and interoception, then naturally consider that our bodies are given to us as “special objects” in the world. Phenomenologically, and 7 conceptually, our body is often described as something we own - a “unique object” (e.g., Longo et al., 2009). Though owning our bodies differs from owning a phone, or shoes, we represent all these entities as objects, that is, bundles of properties enduring through time. The colour or the visual shape is seen as a stable feature of my hand, something that does not just exist here and now. But if I tap on the table, the sounds produced by my hand only exist here and now. Philosophers like Callaghan, Nudds, Casati and others are still disputing what we exactly hear when we hear sounds (see Nudds and O’Callaghan, 2009, for overview): do we hear the event of our finger tapping on the table, or do we hear the material of the table, through the sounds thus produced? 3 Applied to the body, the view according to which we hear the source-objects of the sounds would seem to favour the external world over ourselves: it is the table that resonates and produces the sounds, more than our hands. Besides hand clapping and similar activities, many of the sounds we produce come from objects outside us. Yet we can also focus on what these sounds tell about our bodies. In this respect, the view according to which sounds are events, in this case occurring on or nearby our bodies, seems much more congenial to the effects of sounds on body representation. Through audition, we come to perceive our body, not just as an object, but as a locus or series of events linked through time: we can for instance hear the series of finger taps changing as we move our finger around, or exercise more or less force. In this sense, audition underscores the dynamic continuity, as opposed to the stability, of our body awareness. Because audition provides information about events on or near our bodies, it also highlights how our bodies are in relation to something else (the sounding surface), rather than telling us about our bodies as independent objects. 4. The sounding body: Sounds influence how we represent and use our bodies What do sounds then tell us about our bodies? Different kinds of auditory cues are used in laboratory studies (see part 2 and the comprehensive reviews by Tajadura-Jiménez, Väljamäe, et al., 2018, and Stanton and Spence, 2020), but here we focus less on the sound properties and more on the aspects of our bodily representation that are shaped and modulated by sounds, namely our body metrics, our agency, and our cohesion and relations with others. 4.1. Body metrics and materiality As passive listeners, we can determine characteristics of the bodies of others through the sounds they make. For example, simply by listening to the footsteps of an unknown walker, we can determine the weight of the source of those sounds, through the association between weight and spectral mode cues (Li, Logan and Pastore, 1991). More recently, we have shown that using the same known association and real-time feedback, varying these cues affects the perception of one’s own body weight and size and importantly also the way that one moves that body through observable changes in gait pattern: in the 3 In this video one can experience different sounds produced by touching a surface; unexpected sounds make people more aware of the material of the surface and change the touching behavior: https://www.youtube.com/watch?v=My_I9iWdPBw 8 paradigm displayed in Figure 1A people walk while listening to high/low frequency versions of their own footstep sounds. Short (~20 s) adaptation periods lead to estimates of one’s body as being thinner/lighter or wider/heavier and to changes in emotion and gait as measured by movement sensors (Tajadura-Jiménez, Basia, et al., 2015; Tajadura-Jiménez et al., 2019). Figure 1. Examples of sound-driven body illusions. Changes in limb size can also occur through spatially-altered tapping sounds: in the paradigm displayed in Figure 1B, people tap a hand on a surface while hearing sounds that originate at double the distance from the tapping location (simulated by using a loudspeaker array). Short adaptation periods (~60 s) lead to changes in perceived tactile distances on the tapping arm, feelings of arm elongation and changes in reaching behaviour (Tajadura-Jiménez, Väljamäe, et al., 2012; Tajadura-Jiménez, Tsakiris, et al., 2015; Tajadura-Jiménez et al., 2016). More surprisingly, the temporal alteration of another object’s fall can also lead to an updating of one’s body metrics: in the paradigm displayed in Figure 1C, people repeatedly drop a ball from the height of their head while the timing of the audio-tactile feedback of the ball’s impact near the feet is changed to simulate a longer ball fall (simulation involves using a piezo sensor to detect the ball, and a loudspeaker and vibration mat to provide audio-tactile feedback). Adaptation periods of 120 s lead to feeling taller and behaving as if one’s legs were longer (Tajadura-Jiménez, Deroy, et al., 2018). More recently, we took advantage of known pitch-height associations, where increasing pitches were used to increase the perceived length of one’s finger: in the paradigm displayed in Figure 1D, people pulled their finger while hearing a 2 s rising, descending or constant pitch tone (triggered when pulling is detected by a pressure sensor). Rising pitch makes people 9 perceive, via a “spatial metaphor”, their finger as longer, and overestimating their fingertip position (Tajadura-Jiménez, Vakali, et al., 2017; Nava and Tajadura-Jiménez, 2020). Here, we need to stress that, like for other modalities, it is easier to induce feelings of a “larger body” than a “smaller body”. Furthermore, increased plasticity seems to occur when it comes to implausible materiality of our own bodies: the parchment skin illusion (Jousmäki and Hari, 1998), in which the tactile sensation of relative skin roughness is altered by changing the quality of the auditory feedback produced by hand-rubbing, makes one’s hand being perceived as more moisturized, which is not so implausible. However, the marble hand illusion (Senna et al., 2014), in which hearing ‘stone-like’ sounds when seeing one’s hand being struck by a small hammer makes one’s arm being perceived as hard as marble, which is more implausible. Sounds can also be used, in combination with vision, to create an illusion of ‘ownership’ over a new limb: an illusion known as the audiovisual rubber hand illusion (Choi et al., 2016; Darnai et al., 2017; Radziun and Ehrsson, 2018). 4.2. The agentive body Sounds we produce are tightly connected to our effort and strength. Unsurprisingly, much of the information we derive from the sounds produced as we act tells us about our ‘active’ bodies: how we are acting on the external world. As sounds tell us about our actions, they can also make us change the way we act or perceive our own actions. Musical feedback, for instance, can change the way we perceive our physical exertion (Fritz et al., 2013) or the movement during physical exertion (Newbold, Bianchi-Berthouze and Gold, 2017). For example, musically resolved sonifications (i.e., ending on a perfect or harmonically stable cadence) or unresolved sonifications (i.e., ending on an imperfect or harmonically unstable cadence) accompanying squat movements can impact on the perception of the movement (Newbold, Bianchi-Berthouze and Gold, 2017). Natural surface interaction sounds, such as the sounds of our own tapping, change how we estimate our ability to tap or the applied force during hand tapping (Tajadura-Jiménez, Bianchi-Berthouze, et al., 2015). Alteration of sounds that naturally accompany one’s actions, such as the sound of one’s footsteps during walking and exertion (Figure 1A), can also lead to changes in feelings of body capabilities (e.g., feeling quicker, finding exercise easier) and to actual changes in motor actions (Tajadura-Jiménez, Basia, et al., 2015; Tajadura-Jiménez et al., 2019) 4 . Here, it is possible to consider that effort or capabilities are only indirectly perceived in the sounds - similar to our ability to interpret or infer heat from seeing a stove top that is red. Such inferences may be why we are so easily prone to misinterpret sounds and be fooled. Sounds are used for planning and control of execution for actions. This is exemplified by the study by Castiello and colleagues which showed that hearing at movement onset the sound that will be produced when grasping an object can speed-up the movement (Castiello et al., 2010). Additionally, there are studies showing that hearing actions sounds, such as the sound produced when tearing a paper into pieces, activates the same brain areas that would be recruited when preparing to perform such actions (Aglioti and Pazzaglia, 2011). Disruption of agency over the sounds produced by one’s actions, such as walking, by introducing sensory4 A demo video illustrating the feelings a person gets with such alterations of footstep sounds can be accessed here: https://www.youtube.com/watch?v=moug7lH7w04 16 Psychological medicine, 42(4), pp. 819–28. doi: 10.1017/S0033291711002091. Fairhurst, M. T., Janata, P. and Keller, P. E. (2019) ‘Distinguishing “self” from “other” in a dynamic synchronization task with an adaptive virtual partner’, bioRxiv, p. 625061. doi: 10.1101/625061. Faivre, N. et al. (2014) ‘Multisensory Integration in Complete Unawareness: Evidence From Audiovisual Congruency Priming’, Psychological Science, 25(11), pp. 2006–2016. doi: 10.1177/0956797614547916. Farb, N. A. S., Segal, Z. V and Anderson, A. K. (2013) ‘Attentional Modulation of Primary Interoceptive and Exteroceptive Cortices’, Cerebral Cortex, 23(1), pp. 114–126. doi: 10.1093/cercor/bhr385. Farmer, H., Tajadura-Jiménez, A. and Tsakiris, M. (2012) ‘Beyond the colour of my skin: How skin colour affects the sense of body-ownership’, Consciousness and Cognition, 21(3). doi: 10.1016/j.concog.2012.04.011. Fritz, T. H. et al. (2013) ‘Musical agency reduces perceived exertion during strenuous physical performance’, Proceedings of the National Academy of Sciences, 110(44), pp. 17784 LP – 17789. doi: 10.1073/pnas.1217252110. Gomez-Andrés, A. et al. (2020) ‘Enriching footsteps sounds in gait rehabilitation in chronic stroke patients: a pilot study’, Annals of the New York Academy of Sciences. John Wiley & Sons, Ltd, 1467(1), pp. 48–59. doi: 10.1111/nyas.14276. Guterstam, A., Gentile, G. and Ehrsson, H. H. (2013) ‘The Invisible Hand Illusion: Multisensory Integration Leads to the Embodiment of a Discrete Volume of Empty Space’, Journal of Cognitive Neuroscience, 25(7), pp. 1078–1099. doi: 10.1162/jocn_a_00393. Guterstam, A., Petkova, V. I. and Ehrsson, H. H. (2011) ‘The Illusion of Owning a Third Arm’, PLoS ONE, 6(2), p. e17208. doi: 10.1371/journal.pone.0017208. Hermann, T. and Ritter, H. (2004) ‘Sound and Meaning in Auditory Data Display’, Proceedings of the IEEE, 92(4), pp. 730-741, doi: 10.1109/JPROC.2004.825904. Ho, C. et al. (2013) ‘Multisensory Augmented Reality in the Context of a Retail Clothing Application’, in Bronner, K., Hirt, R., and Ringe, C. (eds) ((( ABA ))) Audio Branding Academy Yearbook 2012/2013. 1st edn. Baden-Baden: Nomos Verlagsgesellschaft mbH & Co. KG, pp. 167–175. doi: 10.5771/9783845243559-167. Holmes, N. P., Snijders, H. J. and Spence, C. (2006) ‘Reaching with alien limbs: Visual exposure to prosthetic hands in a mirror biases proprioception without accompanying illusions of ownership’, Perception & Psychophysics, 68(4), pp. 685– 701. doi: 10.3758/BF03208768. Jousmäki, V. and Hari, R. (1998) ‘Parchment-skin illusion: sound-biased touch.’, Current biology, 8(6), p. R190. doi: 10.1016/S0960-9822(98)70120-4. Kilteni, K. et al. (2012) ‘Extending Body Space in Immersive Virtual Reality: A Very Long Arm Illusion’, PLoS ONE, 7(7), p. e40867. doi: 10.1371/journal.pone.0040867. Kilteni, K. et al. (2015) ‘Over my fake body: body ownership illusions for studying the multisensory basis of own-body perception.’, Frontiers in human neuroscience, 9, p. 141. doi: 10.3389/fnhum.2015.00141. Kilteni, K. and Ehrsson, H. H. (2017) ‘Body ownership determines the attenuation of selfgenerated tactile sensations’, Proceedings of the National Academy of Sciences, 114(31), pp. 8426–8431. doi: 10.1073/pnas.1703347114. Kurihara, Y. et al. (2013) ‘Virtual Robotization of the Human Body via Data-Driven Vibrotactile Feedback’, in Proceeding of the 10th International Conference on Advances in Computer Entertainment - Volume 8253. Berlin, Heidelberg: Springer-Verlag (ACE 2013), pp. 109–122. doi: 10.1007/978-3-319-03161-3_8. Lewis, J. and McCabe, C. (2010) ‘Body Perception Disturbance (BPD) in CRPS’, Practical PAIN MANAGEMENT, PPM Communications, Inc. Lewis, J. S. et al. (2007) ‘Body perception disturbance: A contribution to pain in complex regional pain syndrome (CRPS)’, Pain, 133(1). doi: 10.1016/j.pain.2007.03.013 Ley-Flores, J. et al. (2019) ‘Altering body perception and emotion in physically inactive people through movement sonification’, in Proceedings of the 2019 International Conference on Affective Computing and Intelligent Interaction (ACII), 3rd-6th September, 2019, Cambridge, UK. 17 Ley-Flores, J. G. et al. (2021) ‘SoniBand: Understanding the Effects of Metaphorical Movement Sonifications on Body Perception and Physical Activity’, in Annual ACM Conference on Human Factors in Computing Systems, May 8-13 2021, Yokohama, Japan: ACM. Li, X., Logan, R. J. and Pastore, R. E. (1991) ‘Perception of acoustic source characteristics: Walking sounds’, The Journal of the Acoustical Society of America, 90(6), p. 3036. doi: 10.1121/1.401778. Lloyd, D. M. (2007) ‘Spatial limits on referred touch to an alien limb may reflect boundaries of visuo-tactile peripersonal space surrounding the hand’, Brain and Cognition, 64(1), pp. 104–109. doi: 10.1016/j.bandc.2006.09.013. Longo, M. R. et al. (2009) ‘Self awareness and the body image’, Acta Psychologica. Elsevier B.V., 132(2), pp. 166–172. doi: 10.1016/j.actpsy.2009.02.003. Longo, M. R. and Haggard, P. (2012) ‘What Is It Like to Have a Body?’, Current Directions in Psychological Science, 21(2), pp. 140–145. doi: 10.1177/0963721411434982. Manson, G. A. et al. (2019) ‘Auditory cues for somatosensory targets invoke visuomotor transformations: Behavioral and electrophysiological evidence’, PLoS ONE, 14(5), p. e0215518. doi:10.1371/journal.pone.0215518. Maravita, A. and Iriki, A. (2004) ‘Tools for the body (schema)’, Trends in Cognitive Sciences, 8(2), pp. 79–86. doi: https://doi.org/10.1016/j.tics.2003.12.008. Marotta, A. et al. (2018) ‘Age-related changes in the sense of body ownership: New insights from the rubber hand illusion’, PloS ONE, 13(11), pp. e0207528–e0207528. doi: 10.1371/journal.pone.0207528. Martel, M. et al. (2016) ‘Tool-use: An open window into body representation and its plasticity’, Cognitive Neuropsychology, 33, pp. 1–20. doi: 10.1080/02643294.2016.1167678. Martikainen, M. H., Kaneko, K. and Hari, R. (2005) ‘Suppressed Responses to Self-triggered Sounds in the Human Auditory Cortex’, Cerebral Cortex, 15(3), pp. 299–302. doi: 10.1093/cercor/bhh131. McCabe, C. S. et al. (2003) ‘Referred sensations in patients with complex regional pain syndrome type 1’, Rheumatology, 42(9), pp. 1067–1073. doi: 10.1093/rheumatology/keg298. Menzer, F. et al. (2010) ‘Feeling in control of your footsteps: Conscious gait monitoring and the auditory consequences of footsteps’, Cognitive Neuroscience. Routledge, 1(3), pp. 184–192. doi: 10.1080/17588921003743581. Mifsud, N. G. and Whitford, T. J. (2017) ‘Sensory attenuation of self-initiated sounds maps onto habitual associations between motor action and sound’, Neuropsychologia, 103, pp. 38–43. doi: 10.1016/j.neuropsychologia.2017.07.019. Moseley, G. L. (2005) ‘Distorted body image in complex regional pain syndrome’, Neurology, 65(5), pp. 773 LP – 773. doi: 10.1212/01.wnl.0000174515.07205.11. Moseley, G. L. et al. (2008) ‘Thinking about movement hurts: The effect of motor imagery on pain and swelling in people with chronic arm pain’, Arthritis Care & Research, 59(5), pp. 623–631. doi: 10.1002/art.23580. Moseley, G. L., Gallace, A. and Spence, C. (2012) ‘Bodily illusions in health and disease: Physiological and clinical perspectives and the concept of a cortical “body matrix”’, Neuroscience & Biobehavioral Reviews, 36(1), pp. 34–46. doi: 10.1016/j.neubiorev.2011.03.013. Moseley, L. G. (2007) ‘Using visual illusion to reduce at-level neuropathic pain in paraplegia’, Pain, 130(3), pp. 294–298. doi: 10.1016/j.pain.2007.01.007. Nava, E. et al. (2018) ‘Action Shapes the Sense of Body Ownership Across Human Development’, Frontiers in Psychology, 9, p. 2507. doi: 10.3389/fpsyg.2018.02507. Nava, E. and Tajadura-Jiménez, A. (2020) ‘Auditory-induced body distortions in children and adults’, Scientific Reports, 10(1), p. 3024. doi: 10.1038/s41598-020-59979-0. Newbold, J., Bianchi-Berthouze, N. and Gold, N. E. (2017) ‘Musical Expectancy in Squat Sonification for People Who Struggle with Physical Activity’, in Proceedings of the 23rd International Conference on Auditory Display - ICAD, pp. 65–72. Newbold, J. W. et al. (2016) ‘Musically informed sonification for chronic pain rehabilitation: Facilitating progress and avoiding over-doing’, in Conference on Human Factors in Computing Systems - Proceedings. doi: 10.1145/2858036.2858302. 18 Noppeney, U. (2021) ‘Perceptual Inference, Learning, and Attention in a Multisensory World’, Annual Review of Neuroscience, 44(1), pp. 449–473. doi: 10.1146/annurev-neuro100120-085519. Nudds, M. and O’Callaghan, C. (2009) Sounds and Perception: New Philosophical Essays. OUP Oxford. Ogg, M. and Slevc, L. R. (2019) ‘Acoustic Correlates of Auditory Object and Event Perception: Speakers, Musical Timbres, and Environmental Sounds’, Frontiers in Psychology, 10, p. 1594. doi: 10.3389/fpsyg.2019.01594. Parise, C. V, Knorre, K. and Ernst, M. O. (2014) ‘Natural auditory scene statistics shapes human spatial hearing.’, Proceedings of the National Academy of Sciences of the United States of America, 111(16), pp. 6104–8. doi: 10.1073/pnas.1322705111. Pastore, R. E. et al. (2008) ‘Auditory event perception: The source—perception loop for posture in human gait’, Perception & Psychophysics, 70(1), pp. 13–29. doi: 10.3758/PP.70.1.13. Pearce, E. et al. (2016) ‘Singing together or apart: The effect of competitive and cooperative singing on social bonding within and between sub-groups of a university Fraternity’, Psychology of Music. 44(6), pp. 1255–1273. doi: 10.1177/0305735616636208. Phillips, G. C. et al. (1999) ‘Effects of the presentation of false heart-rate feedback on the performance of two common heartbeat-detection tasks.’, Psychophysiology, 36(4), pp. 504–510. doi: 10.1017/s0048577299980071. Radziun, D. and Ehrsson, H. H. (2018) ‘Auditory cues influence the rubber-hand illusion’, Journal of Experimental Psychology: Human Perception and Performance, 44(7), pp. 1012–1021. doi: 10.1037/xhp0000508. Schaffert, N. et al. (2019) ‘A Review on the Relationship Between Sound and Movement in Sports and Rehabilitation’, Frontiers in Psychology, 10, p. 244. doi: 10.3389/fpsyg.2019.00244. Schaffert, N. and Mattes, K. (2015) ‘Interactive Sonification in Rowing: Acoustic Feedback for On-Water Training’, IEEE MultiMedia, 22(1), pp. 58–67. doi: 10.1109/MMUL.2015.9. Schneider, D. M., Sundararajan, J. and Mooney, R. (2018) ‘A cortical filter that learns to suppress the acoustic consequences of movement’, Nature, 561(7723), pp. 391– 395. doi: 10.1038/s41586-018-0520-5. Scholz, D. S. et al. (2015) ‘Moving with music for stroke rehabilitation: a sonification feasibility study’, Annals of the New York Academy of Sciences, 1337(1), pp. 69–76. doi: 10.1111/nyas.12691. Senna, I. et al. (2014) ‘The marble-hand illusion’, PLoS ONE, 9(3), pp. 1–6. doi: 10.1371/journal.pone.0091688. Shimada, S., Fukuda, K. and Hiraki, K. (2009) ‘Rubber Hand Illusion under Delayed Visual Feedback’, PLoS ONE, 4(7), p. e6185. doi: 10.1371/journal.pone.0006185. Singh, A. et al. (2016) ‘Go-with-the-Flow: Tracking, Analysis and Sonification of Movement and Breathing to Build Confidence in Activity Despite Chronic Pain’, HumanComputer Interaction, 31(3–4). doi: 10.1080/07370024.2015.1085310. Stanton, T. R. et al. (2017) ‘Feeling stiffness in the back: a protective perceptual inference in chronic back pain’, Scientific Reports, 7(1), p. 9681. doi: 10.1038/s41598-01709429-1. Stanton, T. R. and Spence, C. (2020) ‘The Influence of Auditory Cues on Bodily and Movement Perception’, Frontiers in Psychology, 10, p. 3001. doi: 10.3389/fpsyg.2019.03001. Tajadura-Jiménez, A., Väljamäe, A., et al. (2012) ‘Action sounds recalibrate perceived tactile distance’, Current Biology, 22(13), pp. R516–R517. doi: 10.1016/j.cub.2012.04.028. Tajadura-Jiménez, A., Longo, M. R., et al. (2012) ‘The person in the mirror: Using the enfacement illusion to investigate the experiential structure of self-identification’, Consciousness and Cognition, 21(4). doi: 10.1016/j.concog.2012.10.004. Tajadura-Jiménez, A., Tsakiris, M., et al. (2015) ‘Action sounds update the mental representation of arm dimension: Contributions of kinaesthesia and agency’, Frontiers in Psychology, 6, pp. 1–18. doi: 10.3389/fpsyg.2015.00689. Tajadura-Jiménez, A., Basia, M., et al. (2015) ‘As Light As Your Footsteps: Altering Walking 19 Sounds to Change Perceived Body Weight, Emotional State and Gait’, in Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. New York, NY, USA: ACM (CHI ’15), pp. 2943–2952. doi: 10.1145/2702123.2702374. Tajadura-Jiménez, A., Bianchi-Berthouze, N., et al. (2015) ‘Sonification of surface tapping changes behavior, surface perception, and emotion’, IEEE Multimedia, 22(1). doi: 10.1109/MMUL.2015.14. Tajadura-Jiménez, A. et al. (2016) ‘Action sounds modulate arm reaching movements’, Frontiers in Psychology, 7. doi: 10.3389/fpsyg.2016.01391. Tajadura-Jiménez, A., Vakali, M., et al. (2017) ‘Contingent sounds change the mental representation of one’s finger length’, Scientific Reports, 7(1). doi: 10.1038/s41598017-05870-4. Tajadura-Jiménez, A., Deroy, O., et al. (2018) ‘Audio-tactile cues from an object’s fall change estimates of one’s body height’, PLoS ONE, 13(6), p. e0199354. doi: 10.1371/journal.pone.0199354. Tajadura-Jiménez, A., Väljamäe, A., et al. (2018) ‘Principles for Designing Body-Centered Auditory Feedback’, in The Wiley Handbook of Human Computer Interaction. John Wiley & Sons, Ltd, pp. 371–403. doi: 10.1002/9781118976005.ch18. Tajadura-Jiménez, A. et al. (2019) ‘As Light as You Aspire to Be: Changing Body Perception with Sound to Support Physical Activity’, in CHI Conference on Human Factors in Computing Systems Proceedings (CHI 2019), May 4–9, 2019, Glasgow, Scotland, UK. ACM, New York, NY, USA. Tajadura-Jiménez, A., Cohen, H. and Bianchi-Berthouze, N. (2017) ‘Bodily sensory inputs and anomalous bodily experiences in complex regional pain syndrome: Evaluation of the potential effects of sound feedback’, Frontiers in Human Neuroscience, 11. doi: 10.3389/fnhum.2017.00379. Tajadura-Jiménez, A. and Tsakiris, M. (2014) ‘Balancing the “inner” and the “outer” self: Interoceptive sensitivity modulates self-other boundaries’, Journal of Experimental Psychology: General, 143(2). doi: 10.1037/a0033171. Tajadura-Jiménez, A., Väljamäe, A. and Västfjäll, D. (2008) ‘Self-representation in mediated environments: The experience of emotions modulated by auditory-vibrotactile heartbeat’, Cyberpsychology and Behavior, 11(1). doi: 10.1089/cpb.2007.0002. Takeshima, Y., 2020. Emotional information affects fission illusion induced by audio-visual interactions. Scientific Reports, 10, 998. doi: 10.1038/s41598-020-57719-y Thompson, J. K. and Stice, E. (2001) ‘Thin-Ideal Internalization: Mounting Evidence for a New Risk Factor for Body-Image Disturbance and Eating Pathology’, Current Directions in Psychological Science, 10(5), pp. 181–183. doi: 10.1111/1467-8721.00144. Tonetto, L. M. and Spence, C. (2014) ‘Modifying action sounds influences people ’ s emotional responses and bodily sensations’, i-Perception, 5(3), pp. 153–163. doi: 10.1068/i0653 Tsakiris, M. et al. (2010) ‘Hands only illusion: Multisensory integration elicits sense of ownership for body parts but not for non-corporeal objects’, Experimental Brain Research, 204(3), pp. 343–352. doi: 10.1007/s00221-009-2039-3. Tsakiris, M. (2010) ‘My body in the brain: A neurocognitive model of body-ownership’, Neuropsychologia, 48(3), pp. 703–712. doi: 10.1016/j.neuropsychologia.2009.09.034. Tsakiris, M., Prabhu, G. and Haggard, P. (2006) ‘Having a body versus moving your body: How agency structures body-ownership’, Consciousness and Cognition, 15(2), pp. 423–432. doi: https://doi.org/10.1016/j.concog.2005.09.004. Tsakiris, M., Tajadura-Jiménez, A. and Costantini, M. (2011) ‘Just a heartbeat away from one’s body:Interoceptive sensitivity predicts malleability of body-representations’, Proceedings of the Royal Society B: Biological Sciences, 278(1717). doi: 10.1098/rspb.2010.2547. Turchet, L. et al. (2016) ‘What do your footsteps sound like? An investigation on interactive footstep sounds adjustment’, Applied Acoustics, 111. doi: 10.1016/j.apacoust.2016.04.007. 20 Turton, A. et al. (2013) ‘Evaluation of a Prototype Tool for Communicating Body Perception Disturbances in Complex Regional Pain Syndrome’, Frontiers in Human Neuroscience, p. 517. doi: 10.3389/fnhum.2013.00517. Viaud-Delmon, I. and Warusfel, O. (2014) ‘From ear to body: The auditory-motor loop in spatial cognition’, Frontiers in Neuroscience, 8, pp. 1–9. doi: 10.3389/fnins.2014.00283. Visell, Y. et al. (2009) ‘Sound design and perception in walking interactions’, International Journal of Human-Computer Studies, 67(11), pp. 947–959. doi: 10.1016/j.ijhcs.2009.07.007. Vogt, K. et al. (2010) ‘PhysioSonic - Evaluated Movement Sonification As Auditory Feedback in Physiotherapy’, in Proceedings of the 6th International Conference on Auditory Display. Berlin, Heidelberg: Springer-Verlag (CMMR/ICAD’09), pp. 103–120. doi: 10.1007/978-3-642-12439-6_6. Wallis, R. I. et al. (2007) ‘Real-time sonification movement for an immersive stroke rehabilitation environment’, in Proceedings of the 13th International Conference on Auditory Display, Montréal, Canada, June 26-29, 2007. Wolpert, D. M. and Ghahramani, Z. (2000) ‘Computational principles of movement neuroscience’, Nature Neuroscience, 3(11), pp. 1212–1217. doi: 10.1038/81497.