Feeling the music: exploring emotional effects of auditory-tactile musical experiences

Feeling the music: exploring emotional effects of auditorytactile musical experiences Naama Schwartz † , Adi Snir* † and Amir Amedi* 1 The Dina Recanati School of Medicine, Reichman University, Herzliya, Israel, 2 Baruch Ivcher School of Psychology, Institute for Brain, Cognition and Technology, Reichman University, Herzliya, Israel Tactile pairing with auditory stimulation has been shown to enhance various capabilities, including the intensity of the stimulus, its location, and its comprehensibility in noise. However the effect of adding haptics on emotional state is still poorly understood, despite the key role of bodily experiences on emotional states. In the current study we aimed to investigate the impact of a multisensory audio-tactile music experience on emotional states and anxiety levels. For this purpose we developed an in-house algorithm and hardware, converting audio information to vibration perceivable through haptics, optimized for music. We compare participants’emotional experiences of music when provided with audio only versus audio-tactile feedback. We further investigate the impact of enabling participants to freely select their music of choice on the experience. Results indicate multisensory music significantly increases positive mood and decreased state anxiety when compared to the audio only condition. These findings underscore the potential of multisensory stimulation and sensoryenhanced music-touch experiences, specifically for emotional regulation. The results are further enhanced when participants are given the autonomy to choose the musical content. We discuss the importance of multisensory enhancement and embodied experiences on emotional states. We further outline the potential of multisensory experiences in producing robust representations, and discuss specific use cases for technologies that enable controlled multisensory experiences, particularly VR headsets, which increasingly incorporate multifrequency haptic feedback through their controllers and APIs. KEYWORDS multisensory (audio, tactile), music emotion, virtual reality, sensory enhancement, embodied music cognition, anxiety, music technology Introduction Imagine yourself in a live concert, expecting your favorite artist to go on stage. Suddenly, big lights turn on and you see the artist enter a magnificent stage. The visual effects together with the cool performers’outfit makes you feel even greater anticipation. Then, you hear the sound you love so much, and feel the bass vibrating under your feet. You know every note in the opening song which gave you hope so many times in your past, but the live multisensory experience of it sweeps you into a wholesome feeling of happiness. This common multisensory experience is a result of information being integrated by multiple sensory modalities in order to create a coherent and robust perception of the world. This integration is essential for adapting to environmental changes and responding to stimuli efficiently (Kristjánsson et al., 2016). Beyond improved perception, it has been shown that multisensory manipulations significantly impact cognition, emotions, and behavior in OPEN ACCESS EDITED BY Justine Saint-Aubert, Inria Rennes - Bretagne Atlantique Research Centre, France REVIEWED BY Peter Eachus, University of Salford, United Kingdom Ravali Gourishetti, Max Planck Institute for Intelligent Systems, Germany *CORRESPONDENCE Adi Snir, [email protected], [email protected] Amir Amedi, [email protected] † These authors have contributed equally to this work and share first authorship RECEIVED 12 March 2025 ACCEPTED 28 July 2025 PUBLISHED 01 October 2025 CITATION Schwartz N, Snir A and Amedi A (2025) Feeling the music: exploring emotional effects of auditory-tactile musical experiences. Front. Virtual Real. 6:1592652. doi: 10.3389/frvir.2025.1592652 COPYRIGHT © 2025 Schwartz, Snir and Amedi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. Frontiers in Virtual Reality frontiersin.org01 TYPE Original Research PUBLISHED 01 October 2025 DOI 10.3389/frvir.2025.1592652 real-world scenarios (Shams et al., 2011;Khan et al., 2022;Cornelio et al., 2021). This indicates that while sensory signals offer congruent information, they are integrated into enhanced neural and behavioral processes. Importantly, congruent sensory input also enhances emotional experiences (Garcia Lopez et al., 2022), which may serve as one reason behind concerts being a more unique and emotionally engaging experience compared to listening to music on a stereo system or headphones (Trost et al., 2024). There is plenty of evidence suggesting music listening as an effective sensory manipulation to upregulate positive emotions (e.g., happiness and relaxation) and downregulate negative emotions (Lawton, 2014;Groarke and Hogan, 2019). Music in particular, has been shown to effectively reduce stress, mitigate anxiety and decrease depression symptoms through music therapy as well as by passively listening to music (MasHerrero et al., 2023;De Witte, et al., 2020;Ferreri, et al., 2019). However, music familiarity and preferences may mediate listeners’emotional experiences (Van Den Bosch et al., 2013). Prior research indicates that familiarity with music enhances the perception of consonance, leading to higher pleasantness ratings (McLachlan et al., 2013) and activating brain regions associated with reward, while unfamiliar music elicited brain activity in regions related to recognition and novelty detection (Freitas et al., 2018). Furthermore, musical preference (selected by the participants) has been associated with an increase in dopaminergic activity in the mesolimbic system (Salimpoor et al., 2011), suggesting that familiar and preferred music would greatly impact emotional experience, and could easily be incorporated into daily living, providing an accessible and cost-effective intervention with significant therapeutic potential (Finn and Fancourt, 2018;De Witte et al., 2020;Harney, 2023). The reciprocal link between music and emotions is not surprising, as emotional experiences arise from perceiving the world through our senses (Polania et al., 2024). The Circumplex Model of emotion perspective suggests that emotional experiences can be described on two dimensions: emotional valence and emotional arousal (Russell, 1980). Valence denotes how much an emotion is positive or negative and Arousal can be defined as the degree of subjective activation, or more simply low or high emotional intensity (Pessoa and Adolphs, 2010;Nejati et al., 2021). Congruently, a large-scale study indicated that 82% of participants use music to increase valence (Lawton, 2014) but it also alters physiological experiences of arousal, including heart rate, blood pressure, and cortisol levels (De Witte et al., 2020). Indeed, neuroscientific studies indicate that sensation affects emotions very quickly, without any cognitive demand (Guo et al., 2015) even when the sensory stimuli is subliminal (Roschk and Hosseinpour, 2020). This suggests that music based sensory manipulation might be a relatively effortless way to manage emotions (Rodriguez and Kross, 2023). In recent years, technological progress of multisensory based technologies has become increasingly important for addressing sensory, cognitive and emotional processes (Amedi et al., 2024; Tuominen and Saarni, 2024). Sensory substitution devices which convey information typically delivered by one sensory modality through a different sensory modality using specific translation algorithms (Heimler and Amedi, 2020;Heimler, StriemAmit & Amedi, 2015) have been used for the enhancement of various sensory experiences (Kristjánsson et al., 2016). Virtual Reality (VR) also generates environments through creation of multisensory stimulation to further immerse users within them (Montana et al., 2020). These technologies have assisted in expanding the scope of multisensory enhancement research (Cornelio et al., 2021), showing that multisensory integration optimizes perception beyond what each sense can achieve individually, demonstrated by improved spatial perception (Snir et al., 2024), speech comprehension (Ciesla et al., 2021), attention and even memory recall using multisensory integration based stimuli (Stein et al., 2020). Moreover, it has been suggested that the existence of multisensory stimuli in VR enhances user performance, perception, experience and sense of presence (Melo et al., 2020). While most of the existing literature on multisensory enhancement has been confined to visual and auditory modalities (Cornelio et al., 2021), recent focus on audio-tactile stimulation suggested more intuitive substitution, requiring no or little training (rather than tens of hours often required by visual-auditory devices for instance; Snir et al., 2024), possibly because these modalities share common mechanisms (Russo, 2020;Senkowski and Engel, 2024). Since sound is a multi-modal phenomenon that can be experienced both sonically and through vibrations (Trivedi et al., 2019), both are sensitive to partially overlapping frequencies, and are processed in common brain regions (Auer et al., 2007), it is reasonable to assume that coupling them will also embellish subjective experience (Melo et al., 2020). Moreover, recent evidence stresses the benefits of multisensory stimulation (audio and tactile) over audio alone, in learning and sensory enhancement by healthy participants (Ciesla et al., 2019;Ciesla et al., 2022), in enriching subjective experience in terms of emotional valence, arousal, and enjoyment (Turchet et al., 2020;Garcia Lopez et al., 2022) and yielding a greater sense of embodiment, engagement and agency (Leonardis et al., 2014). Tactile technologies in themselves have been developed to impact anxiety (Faerman et al., 2022; Goncu-Berk et al., 2020). These nonetheless consider the impact of tactile experience on self ownership and self-agency that have been proposed as a basis for the sense of embodiment (Shimada, 2022). Going back to the example of a concert, the multisensory experience of the sound of one’s favorite music, synchronized with the vibrating ground can thus enhance the emotional experience. VR concerts have become increasingly popular and provide more immersive experiences than simply watching on screen, however, emphasis is usually given to vision and audition, neglecting the importance of synchronised haptics (Venkatesan and Wang, 2023), which play an important role in both the perception and the enjoyment of music (Merchel and Altinsoy, 2020). Supporting evidence suggests that listening to recorded music without vibrotactile feedback is less perceptually rich and immersive (Ideguchi and Muranaka, 2007). In their book Musical Haptics, Pappeti & Saitis (2018) thoroughly explain why haptic technology is the most suitable to communicate musical information and enrich the experience of music listening. Despite the attention musical haptics have recently received, user experience evaluation methods have not been established, and healthcare applications have not been sufficiently utilized (Remache-Vinueza et al., 2021). Frontiers in Virtual Reality frontiersin.org02 Schwartz et al. 10.3389/frvir.2025.1592652 Taken together, the current study features a novel tactile device developed by our lab in the form of two handheld balls, along with a novel algorithm developed by one of the authors of the manuscript (A.S.) dedicated to transforming musical information to touch. The algorithm both transforms frequency content as well as emphasizes various audio features in order to better replicate the sound content to perceptual features of the tactile modality. To further strengthen the experience, actuators with matching content are also placed on the chest and heels. The devices use wide frequency response actuators on par with some of the most popular VR/gaming sets’ controllers (i.e., Oculus Pro and PS5). The system was tested for user experience by tens of users (both healthy and with various mental conditions), with overall positive subjective responses, yet the current study is the first to test the device in an experimental setting. We use this unique setup to perform a proof of concept examination of multisensory music and its impact on one’s emotional experience and state anxiety markers. We hypothesize that audio-tactile music will enhance the regulative qualities of music on emotional state to a greater extent than the same music experienced through only the auditory modality. Based on the reviewed literature we expect to find increased valence, decreased arousal measured by Self Manikin Assessment (SAM; Bradley and Lang, 1994) and lower STAI state anxiety scores (Spielberger, 1983). We further hypothesize that the effects of audio-tactile musical experiences will be more pronounced when participants are able to select their music of choice, due to its added emotional relevance (Fuentes-Sánchez et al., 2022). Methods Participants: Thirty two healthy individuals participated in this proof of concept study (20 females and 11 males; M age = 22, SD = 3.8). Participants were not native English-speakers but were fluent in English. Each provided an informed consent and they were not paid for participation. Participants were recruited online via the Reichman University Sona Systems which facilitates student participation in psychology and neuroscience studies, as approved by the Institutional Review Board (IRB) of Reichman University (P_2023192). Preparation of stimuli We used CrowdAI, released by Spotify (Brost et al., 2019), which contains approximately 4 million tracks in order to choose one instrumental musical track for the experiment. By using an instrumental track as an experimental stimulus, we sought to study the effect of music alone, as opposed to the combined effect of musical and lyrical content on emotions (Barradas and Sakka, 2022). This data set includes eight audio features (Instrumentalness, Acousticness, Speechiness, Liveness, Tempo, Energy, Valence and Danceability). We used the Python package GSA (Heggli, 2020 GitHub Repos) and excluded tracks that are very popular or completely unknown (including only tracks that have between 50 to 500 followers), to prevent familiarity (Ward et al., 2014). Moreover, we included only instrumental tracks (Instrumentalness >0.8) with high valence (>50%) and high BPM (>120 BPM) to keep participants aroused without triggering negative experiences (Pyrovolakis et al., 2022). We ended up with a list of 50 tracks and excluded those shorter than 7 min. We then selected one track randomly: “Café Del Mar” Michael Woods Remix by Energy 52. The track’s features are shown in Table 1 (track features taken from: Spotify for Developers, Get Track’s Audio Features (v1), n.d; Retrieved 18 October 2023 https:// developer.spotify.com/documentation/web-api/reference/getaudio-features). Experimental setup Each participant received the music through headphones (BOSE QC35 IIA), as well as separately transformed left-right channels of audio to vibration on the palms, the chest and the ankles. The tactile devices and conversion algorithm were developed inhouse. The actuators used were voice coil actuators (replacement parts for playstation 5 controllers) with a full response range rated to be between 10 and 1,050 Hz (maximum acceleration frequency rated at approximately 65 Hz., tested for peak-to-peak 0.85 m/s 2 ; for further details see Jagt et al., 2024). These were each driven by a 3W Class D amplifier. Robust actuator response was found to lose intensity above approximately 300 Hz. We thus corrected the response curve using a cascaded filter, resulting in a flat response up to 300 Hz and perceivable actuation up to ~500 Hz. The algorithm converting musical content to tactile actuation (see further below) was thus calibrated in consideration of this frequency range (i.e. 10-500Hz). The vibration emitted to the hands was made into the form of 3D printed balls with a diameter of 2.8cm (standard stress ball size), while the ankle and chest actuators were placed onto the participants’body using velcro straps (see Figure 1). The experimental setup included a PC for music playback and algorithmic processing, to which an 8 output channel sound card was connected (ESIGIGABPORT eX). Output channels 1–2 were mapped to audio, while consecutive pairs were mapped to haptics as such: channels 3–4 (hands; left and right respectively), 5–6 (ankles; left and right respectively) and 7 (chest; software based mixdown of the stereo signal to a mono actuator). All left sided haptic content was identical, as was between right sided actuators. Each side was transformed as a separate algorithmic conversion stream, corresponding directly between the stereo audio output and the side of the haptics on the body. We targeted the chest, palms, and ankles because together they span three somatotopic locations repeatedly highlighted in bodily-mapping research (see discussion for more details). Audio-to-tactile conversion was processed using an inhouse algorithm which transforms audio signal into information perceivable by the tactile sense using a combination of frequency compression and feature detection meant to enhance certain qualities of the musical content. The algorithm performed audioto-tactile conversion of the streamed audio using low latency means. The conversion algorithm was designed to transform information within the most prominent range of frequencies used in music (mainly between 20 and 6,000 Hz) and convert it in bands, via octave based transformation, to a range of vibrations robustly perceivable by the human tactile system (up to 700 Hz; see Merchel and Altinsoy, 2020). When taking into account the ideal range of the Frontiers in Virtual Reality frontiersin.org03 Schwartz et al. 10.3389/frvir.2025.1592652 TABLE 1 Analysis of audio features profile of “Café Del Mar”were derived from Spotify for Developers. Musical Characteristics of “Café Del Mar” Name Length Instrumentalness Speechiness Liveness Tempo Energy Valence Danceab ility Café Del Mar 10:23 0.97 3.72% 37.60% 130 89.00% 79.10% 26.80% FIGURE 1 Actuator placement on the body and step-by-step flow of the study (a) Participants were seated and equipped with noise canceling headphones and five identical voice coil actuators. Two were installed within ergonomically sized plastic spheres to be held in the hands. The remaining three actuators were attached via straps to the ankles and the front center of the chest. The music was converted algorithmically to match somatosensory properties, taking into consideration stereophonic separation of left and right for the hands and ankles, and a monophonic mixdown for the chest. (b) The procedure included three experiential conditions: 1. sound-only (normal music listening through headphones), 2. multisensory (both audio through headphones and vibration actuators simultaneously), 3. participant-choice (free choice of music and of whether to experience it as sound only or multisensory). STAI and SAM questionnaires were given prior to and between all conditions. 32 participants took part in the study. Frontiers in Virtual Reality frontiersin.org04 Schwartz et al. 10.3389/frvir.2025.1592652 actuators selected for the current project, the resulting haptic output was configured between the frequencies of 10–500 Hz. Peak detection and their enhancement were used following frequency transformation to further emphasize moments of attack and rhythmic features (see Figure S1) (The precise algorithm is currently patent pending. For further information contact the author A.S.). Online algorithmic transformation and channel mixing was processed within a Max MSP patcher (cycling ‘74, version 8.5.5) which received the musical output from Spotify. Blackhole (Existential Audio, Canada, Public License version 3.0 (GPLv3) was used as a virtual soundcard for passing the audio between the two applications. Procedure Upon arrival each participant filled a consent form then the experimenter placed actuators on their chest, ankles and hands (in the form of handheld haptic spheres). Subsequently, participants rated baseline measurements. At the start of each session, participants listened to calibration tones and felt a reference vibration from the haptic actuators; each individual adjusted audio volume and vibration amplitude until the stimulus felt clearly perceptible yet comfortable. The chosen audio level was then locked for that participant and remained constant across all experimental conditions. Participants then listened to the same song twice (Café Del Mar 10 min) under two sensory conditions, counterbalanced across participants. The two conditions consisted of: 1) soundonly, which included listening to music over headphones while the tactile devices were deactivated, and 2) multisensory, during which participants experienced music in headphones along with vibration emitted by the tactile devices. The study employed a within-subject experimental design where each participant underwent both conditions while experiencing the same musical piece. Prior to and following every one of the test conditions, participants rated their current emotional valence and arousal, each on a scale from 0 to 100 using Self Assessment Manikins (SAM; Bradley and Lang, 1994), and STAI-state questionnaire (Spilberger, 1983). Upon completion of both conditions, the same participants were also asked to fill out the Aesthetic Emotional Scale questionnaire (Schindler et al., 2017). These participants were then asked to select one more songoftheirownchoosing,aswellastheconditiontheywould prefer to experience it under (sound-only/multisensory). Following this added experience, each participant rated one’s current emotional experience using STAI and SAM as before (see Figure 1). Analysis To define the number of participants needed for the study, a power analysis was conducted using G*Power software (Faul et al., 2007). To determine the required sample size for a within-subjects design with a medium effect size (Cohen’sd= 0.5), 80% power, and a significance level of α= 0.05, a sample size of 32 participants was found to be needed for a significant effect. We tested the normality assumption using Shapiro-Wilks that was found violated (i.e., the data was not normally distributed). Thus to test differences in emotional valence, arousal and state anxiety scores under the different conditions, we conducted a Friedman test for nonparametric repeated measures comparison. To calculate significance between conditions, Wilcoxon signed ranks tests were performed. We compare emotional change (relative to baseline) following multisensory music and following auditory music. To calculate effect size for nonparametric analysis we used Kedall’s Tau and Kendall’s W tests. All p values were corrected for multiple comparisons using Bonferroni corrections. All statistical tests were performed using R (Mangiafico, 2016). Results Emotional valence is higher following the multisensory condition Comparing emotional valence ratings at baseline (Mean = 71.6 ± 16; Median = 78, IQR = 21), following unisensory “CaféDel Mar” (Mean = 69.9 ± 21; Median = 77.5, IQR = 30) and multisensory “CaféDel Mar”(Mean = 76.5 ± 16; Median = 81.5, IQR = 29) yielded a significant Friedman repeated measures test (X 2 r (2,32) = 7.14, p = 0.02, Kendall’s W = 0.112). To understand the simple effect driving the results we conducted a Wilcoxon signed-ranks test which indicated significantly higher valence following the multisensory condition relative to baseline (W = 137.5, z = 1.72, p = 0.04, with moderate effect size Kendall’sτ= 0.30) while the sound-only condition did not significantly differ from baseline valence (W = 174, z = −0.03, p = 0.4; Figure 2). Same examination of emotional arousal did not yield a significant difference ((X 2 r (2,32) = 2.29, p = 0.31) between emotional arousal at baseline (Mean = 43.9 ± 24; Median = 49.4, IQR = 42) following sound-only (Mean = 36.4 ± 24.9; Median = 35, IQR = 40.7) and multisensory experiences (Mean = 38.7 ± 27.1; Median = 34.5, IQR = 36; W = 192.5, z = 0.07, p = 0.09). State anxiety decreases following only the multisensory condition Comparing STAI baseline scores (Mean = 33.7 ± 9; Median = 34, IQR = 13) to STAI scores following the sound-only condition (Mean = 33.1 ± 12; Median = 29, IQR = 12) and multisensory condition (Mean = 29.7 ± 7; Median = 28, IQR = 14) yielded a significant Friedman repeated measures test (X 2 r (2,32) = 8.31, p = 0.015, Kendall’s W = 0.13). Pairwise comparison of music experience conditions to baseline using Wilcoxon signed-ranks test revealed lower state anxiety score following the multisensory condition (W = 251, z = 2.55, p = 0.02) yet not following the sound-only condition (W = 178.5, z = 1.7, p = 0.17; See Figure 3a). The STAI scoring system ranges between a minimum score of 20 to a maximum score of 80. The full range is commonly divided into three classifications: low anxiety (20–37), moderate anxiety (38–44), and high anxiety (45–80; Kayikcioglu et al., 2017). Eight participants within our sample scored a baseline state anxiety within the moderate to high range. Their STAI scores have been plotted Frontiers in Virtual Reality frontiersin.org05 Schwartz et al. 10.3389/frvir.2025.1592652 below and show a common tendency of decreasing following multisensory music (See Figure 3b). Preference of multisensory experience and associations between subjective responses and valence After experiencing both conditions, participants were asked to select a condition of choice (audio or multisensory) for their next song. Twenty eight (88%) of participants chose to experience their song in a multisensory fashion, while only four participants chose the sound-only condition. Examining relations between emotional ratings and Aesthetic Emotional Scale after the condition participants chose, revealed a small to medium correlation with valence, but not arousal ratings. Significant correlations between valence and specific items included: “The experience fascinated me” (r = 0.42, p = 0.02), “Felt something wonderful”(r = 0.41, p = 0.02), “Was impressed”(r = 0.44, p = 0.01) and “Made me happy”(r = 0.38, p = 0.03). Significantly enhanced effects when participants select their music of choice Participants were given the choice to experience any song they’d like under their condition of choice (sound only/ multisensory). Emotional ratings following participants’ experience of choice were compared to baseline to measure emotional change by participants’song of choice. Then emotional change by participants’song was compared to their experience with “Café Del Mar”under the same condition. A Wilcoxon signed-ranks test yielded a significant difference in valence change following Cafe Del Mar (Mean = 5.8 ± 13; Median = 8, IQR = 11) relative to participants’choice (Mean = 15.3 ± 17; Median = 14, IQR = 19) showing greater efficacy for participants’choice (W = 411.5, z = 3.67, p = 0.005, τ= 0.65). Changes to arousal were not found to be significantly different (W = 335.5, z = 1.32, p = 0.36) (see Figure 4;fordetails regarding participants’song of choice see also Table S1). Discussion Audio-tactile enhancement of valence and state anxiety but not arousal The current experiment set out to explore the potential effect of a multisensory (audio-tactile) experience on one’semotional state during music listening. We investigated this using our tactile setup and novel algorithm, which uses actuators with a wide frequency response and is designed especially for transforming musical content to be perceivable through touch (patent pending). Originally, we hypothesised that a combined audio-tactile musical experience would show added effects on both valence and arousal, as well as on state anxiety markers, this in comparison to auditory-only music listening. We found that the addition of our haptic device indeed had a significant effect on FIGURE 2 Valence scores between conditions. Group change to valence ratings in each of the music conditions (sound only and multisensory) in relation to baseline. Changes to valence were found to be significantly different between conditions, suggesting that while multisensory experience of music increases emotional valence, the sound-only condition slightly decreases valence (N= 32; Wilcoxon signed-ranks test; ppp<0.01, pp<0.05); Bonferroni corrections were applied to all pvalues). Frontiers in Virtual Reality frontiersin.org06 Schwartz et al. 10.3389/frvir.2025.1592652 valence, as well as state anxiety self-report measures, but did not affect emotional arousal (see Figures 2,3). We further hypothesised that participants would experience added effects when selecting the musical content themselves. The effect of self choice was indeed found to be significant, in particular when considering valence measures, which further highlights the additive impact of combining tactile information within auditory musical experiences. Below we discuss our findings in light of the existing literature, while considering a number of influential factors, including: multisensory perception and its neural substrates, features of tactile awareness and embodiment, and the effects of agency in musical selection. We consider all of these in terms of their connection to our findings pertaining to emotional states. Perceptual and attention enhancement as a result of combined audio-tactile sensory stimulation Firstly, we discuss the enhancement of the experience caused by multisensory integration of two modalities in tandem (both auditory and tactile information) as opposed to only one. The enhancement of distinct sensory stimuli is a key feature of multisensory integration (Diederich and Colonius, 2004;Peiffer et al., 2007; Khan et al., 2022). Specifically, combining sensory inputs can amplify or suppress incoming signals, when both salient and weak perceptual stimuli are presented, the more salient stimuli are enhanced while the weaker signals are suppressed (Bauer et al., 2015). This mechanism enables reducing perceptual FIGURE 3 State Anxiety (STAI) scores between conditions. (a) State Anxiety (STAI) scores during baseline and following the sound-only and multisensory conditions (N = 32; Friedman test for repeated measures/Wilcoxon signed-ranks tests; ppp<0.01, pp<0.05). Bonferroni corrections were applied to all p values). (b) Scores of the eight participants with the highest STAI baseline scores, showing lower STAI ratings following the multisensory condition. Frontiers in Virtual Reality frontiersin.org07 Schwartz et al. 10.3389/frvir.2025.1592652 uncertainty and enabling the brain to make more reliable decisions by matching information from various senses (Shams et al., 2011;Lu et al., 2016). The use of auditory and tactile stimuli in particular (as opposed to audio-visual for instance) may be further strengthening the multisensory effect. A few papers have now demonstrated various immediate effects on auditory experiences and enhanced auditory capabilities induced by simultaneous tactile stimulation (for more details see the introduction section of this manuscript). These include among others the enhancement of music perception (Russo et al., 2012;Young et al., 2016), as well as speech perception (Ciesla et al., 2019;Ciesla et al., 2022;Fletcher et al., 2020) and even auditory localisation (Snir et al., 2024;Snir et al., 2024;Fletcher, 2021) and instant audio-tactile binding capabilities within complex environments (Snir et al., 2025). Other similarities have been demonstrated, showing that musicians which, similar to their known enhanced auditory detection abilities, show better detection of emotion and frequency through touch when compared to non-musician controls (Sharp et al., 2019;Sharp et al., 2019b). Auditory stimulation has also been shown to have a biasing effect on tactile perception of sinusoidal “sweeps”(Landelle et al., 2023). The current findings support this point of view, showing multisensory integration of audio-tactile music (relative to unisensory) enhances upregulation of positive emotions and downregulates state anxiety. Unlike unisensory music studies where both emotional valence and arousal mediate emotion regulation and pleasure (Salimpoor et al., 2009), here we see that multisensory stimulation had an effect on valence measures yet none were seen for arousal. Effects on valence were expected, as music is known to improve mood and multisensory experiences of music in particular (Yu et al., 2019). Surprisingly, arousal was not significantly different, this in contradiction to prior findings regarding novel sensory experiences, showing that unfamiliar music increases electrodermal activity responses which are strongly related to emotional arousal (Van Den Bosch et al., 2013). Nevertheless, the relaxing content of the music may have reduced arousal to some degree (Baccarani et al., 2023). This may also explain the lack of change in arousal following the participant’s song of choice relative to baseline. Accordingly, the novelty of haptics and the multisensory experience of music offered in the current experiment might have increased the general arousal beyond familiarity of music (Park et al., 2019;Baccarani et al., 2023). On the other hand arousal ratings were not found to be different. Vibrotactile enhancement was strong in the latent dimension of ‘musical engagement’, encompassing the sense of being a part of the music, arousal, and groove. These findings highlight the potential of vibrotactile cues for creating intensive musical experiences (Siedenburg et al., 2024). Effects on emotion have also been demonstrated in audio-visual processing. In fact, enhanced processing of audio visual emotional stimuli show multisensory integration contributes to a coherent sense of self and an intensified sense of presence (Yu et al., 2019). Moreover, it has been suggested that emotional stimuli may sensitize all sensory modalities (Sharvit et al., 2019), by modification of early sensory responses, across vision, auditory and somatosensory modalities, controlled by supramodal networks for emotional regulation of perception and attention (Domínguez-Borràs et al., 2017;Vuilleumier, 2005;Domínguez-Borràs and Vuilleumier, 2013). This further strengthens the view that multimodal stimulation may be key towards enhancing sensory experiences and their potential effect on emotional regulation. FIGURE 4 Valence rating between experimenter and participant selected track. Valence ratings following the participants’song of choice in comparison to “Cafe Del Mar”(a standard playlist intended for relaxation purposes) (N= 32; Wilcoxon signed-ranks tests; pppp<0.001, ppp<0.01, pp<0.05; Bonferroni corrections were applied to all pvalues). Frontiers in Virtual Reality frontiersin.org08 Schwartz et al. 10.3389/frvir.2025.1592652 Neural substrates of auditory and tactile processing Multisensory processing of auditory-visual perception has been much more extensively studied in comparison to auditory-tactile. The expanding research on auditory-tactile processing nonetheless, shows both neural and psychophysical reasons for the immediacy and affective qualities of such integration (Merchel and Altisony, 2020). These effects may in fact be due to the similarities among the two senses as well as their vicinity in terms of where this information would be processed in the brain (Wu et al., 2015). Both the auditory and somatosensory systems are known to have for instance the ability of encoding frequencies, at a partially overlapping frequency range (Pongrac, 2008;Merchel and Altinsoy, 2020;Bolanowski et al., 1988). Yet this may have deeper reasons relating to central nervous system processing. Some authors claim that interactions between the two systems may be inherent (Bernard, 2022). A number of studies have demonstrated the convergence of the auditory and tactile neural pathways in the brain using MRI. Such studies demonstrated that activation in the auditory cortex can be caused by vibrotactile stimulation (Auer et al., 2007;Levänen et al., 1998; Schürmann et al., 2006). Furthermore, multisensory areas, originally understood to be responsive to audio and visual stimulation, have been shown to respond to tactile stimulation as well (Beauchamp et al., 2008). Other work has used texture, an object feature which uses auditory, somatosensory and motor cues, to show shared activity in primary auditory and somatosensory areas for effective discrimination (Landelle et al., 2023). Tactile emotional enhancement due to embodiment via tactile stimulation Tactile experiences play a crucial role in how individuals perceive and interact with their surroundings, both in physical and virtual environments. Touch provides critical feedback for body ownership and agency, shaping our perception of reality (Aoyagi et al., 2021;Cui and Mousas, 2023). Self-ownership and self-agency are both bodily sensory-motor processing, and constitute the embodied self - the self that is situated in “here and now” (Gallagher, 2000). Self ownership is arising from bodily multisensory integration, whereas the self agency is arising from sensorimotor integration regarding intentional movement (Shimada, 2022). This perspective suggests that the use of tactile stimulation, particularly when placed at various positions along the body, may impact embodiment of the musical experience by manipulating self awareness towards different body locations, immersion, and body sensations (Putkinen et al., 2024;Cavdir, 2024). Hence, in the current study body locations were chosen by high sensitivity to emotional impact. Specifically, the chest is sensitive to every high-arousal emotion, reflecting cardiorespiratory changes that listeners consistently feel in the heart–lung area (Nummenmaa et al., 2014; Nummenmaa et al., 2018; Putkinen et al., 2024). The palms, which are among the most touchsensitive surfaces of the body—making them ideal for conveying the energizing aspects of music-evoked affect (Ackerley et al., 2014). Finally the ankles, which extend stimulation to the lower-limb circuitry that lights up for approach-oriented and rhythmentrained states (e.g., dancing), allowing us to engage locomotor embodiment without obstructing other sensors (Hove and Risen, 2009). Stimulating this triad therefore covers core interoceptive arousal, upper-extremity action, and lower-extremity movement, giving a balanced yet practical embodiment profile for modulating mood (Putkinen et al., 2024). Indeed, research into VR concert experiences indicates that tactile stimulation increases the sense of presence, connection, and immersive musical experiences (Venkatesan and Wang, 2023). Neuroscientific investigation suggests that when extrinsic body information (derived primarily from visual and auditory inputs) and intrinsic body information (derived from sensory motor inputs) are consistent, the brain recognizes the body as self, generating a sense of ownership and a sense of agency (Shimada, 2022). In fact, an increased sense of self agency created by audio-tactile music experiences shows that many genres of music activate the supplementary motor area (SMA) in the brain, which controls internally generated movements, thus illustrating the involuntarily planned motion caused by a musical piece (Wu et al., 2015), even in absence of overt movements (Zatorre et al., 2007;Maes et al., 2014). This has been shown to further promote brain plasticity and connectivity in structures related to sensory processing, memory formation, motor function and stimulating complex cognition and multisensory integration, demonstrating that music listening involves somatosensory processing (Wu et al., 2019). Furthermore, motor responses such as dancing or tapping of rhythm are associated directly with music, as an automatic response to auditory processing (Repp and Penel, 2004). In fact, for people who are deaf, body vibration is the only means for experiencing music, and many of them are capable of enjoying music and dancing to it using only such cues (Tranchant et al., 2017). Such findings indicate that our results which show enhanced effects on emotion caused by the added tactile actuation may be influenced by enhanced embodiment of the music. Finally, embodiment theories’approach the perception of music (or any other sensory salient stimuli) as always being in relation to the “self”, taking one’s body as a reference frame while considering all sensory, motor and affective processes (Tajadura-Jiménez, 2008;Damassio, 1999;Niedenthal, 2007). Considering this perspective, we suggest that the enhancement of emotional response resulting from audiotactile music may be due to enhanced embodiment, caused by changing the frame of reference via the use of haptics. Enhanced effects when the music is chosen by the participants Emotional responses to music largely depend on the music, the listener and the situation, hence there is no universal music or sound that will trigger the same emotional reaction in everyone (TajaduraJiménez, 2008). Through enabling free choice of any musical piece with instantaneous tactile feedback, the current setup provided the opportunity to compare the impact of musical selection on valence and arousal. The current study shows that multisensory music enhances emotional valence (but not arousal), and this emotional enhancement is even greater when participants choose the music they would like to experience. As such, there seems to be an additive effect on valence caused by the combination of multisensory Frontiers in Virtual Reality frontiersin.org09 Schwartz et al. 10.3389/frvir.2025.1592652