Comparison of Hearing Device Benefit in Free Conversation and in Speech Tests in Virtual Reality

Comparison of Hearing Device Benefit in Free Conversation and in Speech Tests in Virtual Reality Giso Grimm Angelika Kothe Volker Hohmann December 1, 2025 Abstract Background: The benefit provided by hearing device algorithms often differs between laboratory evaluations and real-world communication situations. This is partly due to the low signal-to-noise ratio (SNR) achieved in adaptive speech tests, which are commonly used to evaluate hearing devices. Furthermore, the tasks performed often differ considerably from real-world communication situations. Hearing aid users often exhibit head movement behaviour in typical laboratory experiments that is not characteristic of conversation situations. These observations suggest that the current evaluation methods may have limited ecological validity. Aim: The primary objective of this study was twofold: first, to investigate the extent to which the benefit provided by hearing devices can be measured during free conversation in an audiovisual virtual reality telepresence environment; and second, to compare this benefit and that measured using a speech matrix test with multiple spatially separated talkers in the same environment. Methods: The study employed two experimental conditions. Condition 1 involved triadic interactive conversations conducted in a virtual reality (VR) setting using telepresence technology. In this condition, the interlocutors were visually represented by animated virtual characters in a noisy pub environment. Condition 2 utilised a speech test in which sentence test material was presented by male and female avatars in different locations. This condition maintained the same spatial configuration and background environment as Condition 1. A directional filter, which was controlled by head movements, was used to provide signal enhancement with varying levels of spatial selectivity. The algorithm was simulated based on ground truth information to ensure it worked independently of the SNR. The time course of the short-time speech and noise levels as well as the signal-to-noise ratio (SNR) during the conversation and the speech test was tracked. The speech test was adapted to achieve 80% word intelligibility (SRT80). Results: The benefit of spatial filtering was found to be higher when measured as an improvement in signal-to-noise ratio (SNR) at the better ear in free triadic conversations than when measured as speech reception threshold (SRT80) in conversation-like two-speaker speech tests. Additionally, we found that the user’s task affects movement behaviour. In free conversation subjects moved their head closer to the active speakers than in the speech test with the same spatial setup. This effect impacts the benefit of the directional filter algorithm. Furthermore, the SNR in free conversation was closer to SNRs typical for conversation environments than in the speech test. Conclusions: This study suggests that the benefit of hearing devices can be assessed through free, interactive conversations in virtual reality using telepresence technology at more realistic signal-to-noise ratios (SNR). Consequently, this approach may improve the ecological validity of research outcomes on the effectiveness of hearing aids. 1 1 Introduction The effectiveness of modern directional hearing aids is closely linked to the head movement behaviour of their users (Hendrikse et al., 2020a). Recent advances in hearing aid technology have introduced novel algorithms that are controlled by both gaze and head movements (Hart et al., 2009; Best et al., 2017; Grimm et al., 2018). But also more conventional algorithms like spatial filtering can be influenced by head movement behaviour, due to misalignment or mal-adaptation (Hendrikse et al., 2020a). With increasing spatial selectivitiy of algorithms as for example provided by recent machine learning approaches (e.g., Westhausen et al., 2024), misalignment may play an increasingly important role. This interaction between hearing aid performance and movement behaviour highlights the need for users to behave naturally in terms of communication-related gaze and head movements during the evaluation process. Ecological validity in hearing research describes the degree to which the outcome of studies is representative of performance in real life (Keidser et al., 2020). This means that a traditional assessment of hearing device benefit, e.g., using speech reception thresholds in simple spatial configurations, may be ecologically valid if the hearing aid user behave in a way that the benefit provided by a device is the same during the assessment as in real life. To achieve natural communication behaviour, interactivity is required (Schilbach et al., 2013; Hadley et al., 2019; Hartwig et al., 2021), e.g., in the form of interactive free conversation. Turn take timing (Hadley and Culling, 2022; Heldner and Edlund, 2010), gaze behaviour (Vertegaal et al., 2001; Holler and Kendrick, 2015) and movement behaviour (Grimm et al., 2020) are critical aspects of natural communication that must be preserved during evaluations. In order to accurately assess the performance of advanced hearing devices in a controlled laboratory environment, such as behaviour-controlled or machine-learning-based devices, it is necessary to replicate real-world communication scenarios (Hohmann et al., 2020). This involves creating settings in which users engage in interactive conversations that allow for a natural flow of communication behaviours, such as eye gaze and head movement, but also speech production. The present study aims to refine evaluation methods along these lines, ensuring they correctly quantify device performance while preserving the naturalness of underlying communication behaviours. Most hearing aid benefit studies rely on speech audiometry, such as sentence tests in noise, to derive a single metric, like the speech reception threshold (SRT). Our central research question is: Can free conversation be used as a valid test paradigm for measuring hearing aid benefit while preserving the natural communication behaviour of their users? In addressing this question, we aim to bridge the gap between conventional, highly controlled audiometric procedures and increasingly complex, behaviour-dependent algorithms. This may enhance the ecological validity of laboratory-based hearing aid assessments. To answer our research question, we engage test subjects in triadic conversations with two confederates using virtual reality and telepresence technology. The confederates were presented to the subjects by their avatars in the form of virtual animated characters. Lip movements were animated in real time with low delay using a speech-driven lip animation method (Llorach et al., 2016), and the avatars’ head movements were controlled by the real head movements of the confederates they represented (Kothe et al., 2025). Free, interactive conversations were interleaved with speech tests in which the avatars reproduced the test’s speech material, using the same spatial geometry and avatar animations as in the conversations. By using virtual acoustic simulation, access to all audio signals separately is possible, and it is possible to measure the signal-to-noise ratio while engaging in interactive conversations. 2 2 Methods 2.1 Experimental paradigm Figure 1 shows an overview of the setup. The test subjects were placed in a virtual pub environment, with two avatars positioned in front of them. Depending on the experimental condition, the avatars were either controlled by the acoustic output of a speech test with two speakers, or by the voices of two confederates using telepresence technology, respectively. One of the two confederates was the experimenter; she was female, and positioned on the left side in front of the test subject. The other confederate was male, on the right side in front of the test subject. Figure 1: Setup of the free conversation condition. The test participant (upper panel) was surrounded by 45 loudspeakers and a projection screen displaying an animated image of a virtual pub environment. This environment included two avatars representing the experimenter and another confederate. The experimenter and confederate were placed in a remote office (bottom panel) and their voices were transmitted to the reproduction system. The bottom picture has been flipped horizontally for a better overview. The test subjects had to perform two different tasks. In one task, they engaged in triadic 3 conversations about casual topics with two confederates. These confederates (bottom panel of Figure 1) were displayed in the laboratory (top panel of Figure 1, where they were represented by avatars with speech-driven lip animation (Llorach et al., 2016), and their real head movements were transmitted in real time to the avatars (Kothe et al., 2025), creating a low-delay telepresence. The other task was a sentence test with two speakers, a female and a male, with interleaved tracks (OLSA; Wagener et al., 1999; Wagener and Brand, 2005). The OLSA speakers were represented by the avatars, with speech-driven lip animation (Llorach et al., 2016) and automated head movement towards the active speaker. This automated head movement also included the test subject; for example, when they started to speak or repeat the understood words, the avatars turned their heads towards them. Additionally, in the speech test, the upcoming speaker turned their head towards the test subject 0.7 seconds before the beginning of the sentence. The nontarget avatar turned the head towards the target avatar at the onset of the sentence, and towards the subject at their speech onset when they repeated the understood words. A directional filter was applied to the signals presented to the test subject. This filter was simulated in the virtual acoustic rendering engine based on the actual head orientation of the subject. The spatial selectivity of the directional filter could be changed, see Section 2.3 for details. 2.2 Virtual environment This study was performed in a virtual pub environment with ambient noise. The audiovisual simulation was modeled after an existing location, the ‘OLs Brauhaus’ in Oldenburg (Grimm et al., 2021; van de Par et al., 2022). Early reflections of the walls, the ceiling, the floor and one table were simulated using a geometric image source model, together with late reverberation generated by a feedback-delay-network. The room acoustic model was rendered using TASCAR (Grimm et al., 2019b), see Section 2.5 for details. The positions of the target speakers, together with the test subject, formed an equilateral triangle with an edge length of 1.5 m. From the subject’s perspective, the female avatar was on the left side, and the male avatar on the right side. The head translation of the subjects was used to calculate a parallax effect to increase the level of immersion, by shifting the virtual camera in the game engine as well as the receiver position in the acoustic simulation, as described in Hendrikse et al. (2019). Subjects were exposed to two different noise conditions: a quiet environment with a background noise level of approximately 40 dB SPL(A) by a fridge to mask the noise produced by the video projectors in the laboratory, and a noisy environment with a combined sound intensity of 66 dB SPL(A), consisting of mostly diffuse babble noise where intelligible portions were removed (Grimm et al., 2019a), overlaid with music reproduced from two simulated loudspeakers in the virtual environment. 2.3 Signal enhancement A spatial filtering algorithm was simulated in rendering system. Different directivity patterns were compared. A directivity of 0 corresponds to an omni-directional setting, a directivity of 1 represents a cardioid directivity pattern. A directivity of 2 results in a narrower beam pattern. The attenuation of the signal was limited to 12 dB. The beam patterns for the settings used in this study are shown in Figure 2. The steering direction of the signal enhancement was controlled by the head movement of the subject. The directivity was achieved by applying an attenuation depending on the angular distance between the steering direction (i.e., the head orientation) and the incidence direction of the sound sources. The diffuse sound field, rendered as a first-order ambisonics signal, was filtered using modal beamforming. For a directivity greater than 1, an 4 additional attenuation of the diffuse sound field was applied, to compensate for the limited spatial resolution of the first-order Ambisonics signal. This method of generating spatial filtering made it possible to achieve the desired spatial selectivity independently of the SNR. This independence from the SNR is important here for statistical analysis in order to separate the effects of the task from the algorithm and potential SNR dependence of the algorithm. 0° 30° 60° 90° 120° 150° 180° 210° 240° 270° 300° 330° 0 0.2 0.4 0.6 0.8 1 dir 0 dir 1 dir 2 Figure 2: Beam pattern of the simulated signal enhancement algorithm. Algorithm ‘dir 0’ is omnidirectional, ‘dir 1’ is similar to a cardioid directionality pattern, and ‘dir 2’ has a higher spatial selectivity. 2.4 Measures and sensors The dependent variables comprised acoustic and behavioural measures obtained under experimental conditions described in Section 2.6. The speech level at the source was recorded using a head-mounted microphone positioned near the speaker’s mouth. To compensate the near-field effect of the microphone, a second order highpass filter with a cut-off frequency of 180 Hz was applied to the signal before further processing. As the microphone moves with the head, this level reflects the sound pressure level of the target speech, regardless of subsequent head translation or rotation. To estimate the sound pressure level at the listener’s ears, a parametric head-related transfer function (HRTF) model (Schwark et al., 2022; Ewert et al., 2021) was embedded in the virtual acoustic scene. This model generated binaural signals for the target speech Sand background noise Nat the listening position, while explicitly accounting for the listener’s head movements in terms of translation and rotation. The binaural signals for Sand Nwere computed both unprocessed (without signal enhancement), labelled with ‘u’, and processed, labelled with ‘p’, using the simulated directional algorithm. In other words, a set of four binaural signals was 5 generated: X=nx(t)l,r S,u, x(t)l,r N,u, x(t)l,r S,p, x(t)l,r N,po(1) It is important to note that the unprocessed signals xuare an estimate of the signals picked up by a hearing device, whereas the processed signals xpare an estimate of the signals heard by the subjects. A set of short-term levels in 10 ms windows L=nLl,r S,u, . . .owas then calculated from the set of signals Xas the 10 log10 of the mean-square of each 10 ms segment, in logarithmic representation in dB SPL. To get an estimate of the speech and noise level at the listening position, the average of the left and right signal intensity was taken for each segment. An estimate of the signal-to-noise ratio (SNR) at the subject’s left and right ear is calculated as SNRl,r p=Ll,r S,p −Ll,r N,p (2) and accordingly for SNRl,r u. Humans typically use their better ear for listening, which means they select the ear with the better SNR. This is commonly done also for the evaluation of hearing device algorithms (Jespersen et al., 2021; Hendrikse et al., 2022). To account for this effect, here the better ear was identified within each 10 ms time segment. This was based on the processed signals, and the same selection was used when calculating the unprocessed SNR: SNRp=SNRl pSNRl p>SNRr p SNRr potherwise (3) SNRu=SNRl uSNRl p>SNRr p SNRr uotherwise (4) For the unprocessed SNR, SNRu, the criterion depended also on SNRp, because the subjects perceived only the processed signals. The device benefit measured in SNR, ∆SNR, is the difference between those: ∆SNR = SNRp−SNRu(5) For further analysis of these measures, L, SNRp, SNRuand ∆SNR, the measures were calculated segment by segment, and the median across all segments in a condition was taken, resulting in a single value per subject and condition. Speech intelligibility was quantified using the speech-reception threshold (SRT), which was measured via an adaptive procedure that varied the speech level while maintaining a constant noise level (Wagener and Brand, 2005). The method was configured to converge at the speech level at which the participant correctly identified 80% of words. A psychometric function was fitted to the data. The SRT is the SNR at which the psychometric function reaches 80% (SRT80). In this study, the control of the sentence test was based on the raw speech material and an assumed stationary noise, with regard to which all SNRs and the SRT are calibrated. Therefore, the absolute SRT had an arbitrary offset, which was corrected by adjusting the median SRT pooled across all speech test conditions and subjects to the median SNR pooled across the last 10 sentences of each list and the same conditions. This approach resulted in a single calibration offset for the whole experiment and therefore maintained all individual differences and variance across conditions. The device benefit measured in SRT, ∆SRT, is the difference between the SRT in a condition with processing (ST-Cn,1, ST-Cn,2) and the SRT in the condition without processing ST-Cn,0. 6 2.5 Apparatus In the experiment, the test subjects were located in a virtual reality laboratory surrounded by 45 loudspeakers and a video projection with a field of view of 300°on a cylindrical screen; see also Hohmann et al. (2020). The Genelec 8020 loudspeakers were arranged in two rings of 16 speakers each: one at ear level and one at +15°elevation; two rings of six speakers at −30°and +40°elevations; and one speaker above the listener. The interlocutors were located in a separate room. Audio signals as well as head movement and gaze data (not used here) were transmitted to the laboratory via the internet with low latency using the OVBOX system (Grimm, 2024). In the virtual reality laboratory, the virtual acoustic environment was rendered using version 0.229.2.58-6a04339 of TASCAR (Grimm et al., 2019b). The direct sound paths of the two remote interlocutors and the background music, as well as the early reflections of the subject’s voice and all the other sources in the virtual environment, were rendered using VBAP (Pulkki, 1997). Late reverberation, babble background noise and the fridge sound were reproduced as first-order ambisonics signals using a maxrE decoder with decorrelation of the loudspeaker signals to maintain diffuseness and avoid colouration artefacts. The visual environment was rendered using the Blender game engine (version 2.79c). The visual model of the room was taken from Grimm et al. (2021). The avatars of the confederates as well as guests in the background were simulated as virtual animated characters. Lip movements of the avatars were animated in real time with low delay using a speech-driven lip animation method (Llorach et al., 2016), and the avatars’ head movements were controlled by the real head movements of the confederates they represented (Kothe et al., 2025). The gaze direction of the avatars was controlled by the head movement; it was either directed to the test subject or other confederate, whatever was closer to the head orientation. Gaze changes were instantaneous. Eye blinks were simulated at random times, and a slow torso movement simulating breathing was applied to the avatars. The head movements of the subjects were recorded using a Qualisys Miqus M3 optical markercrown tracking system comprising six cameras. Head orientation controlled simulated spatial filtering, and the virtual receivers for estimation of the sound levels at the listening position. Head movements of the confederates were recorded with a IMU sensor attached to the headsets, consisting of a MPU6050 motion sensor and processor, with an ESP8266 Wifi micro-controller to send the data as OSC messages (Grimm, 2025). The speech level was recorded with a head-attached microphone for the subject, and with the headset microphones for the confederates. All data were recorded in a central data logging implemented in TASCAR. 2.6 Experimental conditions The subjects had to complete two tasks. One task involved free conversation (‘FC’) in a triadic setting between the subject and two confederates. Although no topics were provided, the confederates initiated the conversation with casual topics such as food preferences, travel, and vacation plans. The other task was a speech matrix test, performed in a complex virtual acoustic setting (‘ST-C’), using the Oldenburg Sentence Test (OLSA) speech material Wagener and Brand (2005). Two test lists with 20 sentences each, one list with a male voice and one with a female voice, were randomly interleaved. The acoustic source positions were the same as in the triadic conversation. The lip movements of the avatars were animated based on the speech material. See Table 1 for a list of conditions. Each condition took approximately 5 minutes. The conditions were pseudo-randomized; not more than three identical tasks were presented in a sequence. The whole experiment lasted for approximately 1 hour per subject. 7 Table 1: List of conditions and independent variables. label task noise algorithm noise earlevel FCq,0 FC 40 dB SPL(A) dir 0 45.2 dB SPL(C) FCn,0 FC 66 dB SPL(A) dir 0 69.1 dB SPL(C) FCn,1 FC 66 dB SPL(A) dir 1 64.6 dB SPL(C) FCn,2 FC 66 dB SPL(A) dir 2 58.6 dB SPL(C) ST-Cn,0 ST-C 66 dB SPL(A) dir 0 69.1 dB SPL(C) ST-Cn,1 ST-C 66 dB SPL(A) dir 1 64.6 dB SPL(C) ST-Cn,2 ST-C 66 dB SPL(A) dir 2 58.6 dB SPL(C) 2.7 Hypotheses and statistical analysis For an overview of the hypotheses and variables, see Table 2. Our primary hypothesis of this study is: The benefit provided by directional filtering, as measured by the speech-reception threshold (SRT) in a speech test, is comparable to the benefit measured by the signal-to-noise ratio (SNR) in free, interactive conversation (H1). A two (task) by two (algorithm) repeated measures analysis of variance (ANOVA) was used for analysis of main effects and interaction effects. Pairwise differences were tested using Tukey’s honest significant difference test to avoid inflating the family-wise error rate. Furthermore, based on previous observartions, we assume that the head movement differs between free conversation and speech tests (H2). Here, a two (task) by two (algorithm) repeated measures analysis of variance (ANOVA) was used for analysis of main effects and interaction effects. We expect that evaluating hearing devices using free conversation will result in a more typical signal-to-noise ratio (SNR), thereby increasing the ecological validity of device evaluation outcomes (H3). However, ecological validity cannot be directly quantified. Therefore, we compare our results with data from the literature on typical SNRs (Smeds et al., 2015). Similarly, we investigate the interaction between algorithm and the vocal effort, expressed in speech level (H4). The internal validity of the paradigm and the apparatus was tested based on the Lombard effect (Brumm and Zollinger, 2011; Garnier and Henrich, 2014), i.e., the effect of noise level on speech production, here measured in terms of broadband speech level. This effect should be consistent for all speakers, regardless of the reproduction method or their role. Table 2: List of hypotheses and comparisons. aim dependent var. independent var. conditions H1 hearing device benefit ∆SNR, task ×algorithm FCn,0-2, ST-Cn,0-2 SRT versus SNR ∆SRT H2 head movement angular distance task ×algorithm FCn,0-2, ST-Cn,0-2 FC versus ST-C to talker H3 ecological validity: SNRutask ×algorithm FCn,0-2, ST-Cn,0-2 typical SNR H4 ecological validity: speech level algorithm FCn,0-2 vocal effort ·/·internal validity: speech level noise ×interlocutor FCq,0, FCn,0 Lombard effect 8 2.8 Test subjects A total of 11 young test subjects were recruited, with a mean age of 25.1 years (standard deviation 5.2 years). All subjects were native German speakers who self-reported normal hearing. None of the subjects knew the experimenters beforehand. All subjects provided written consent after being fully informed about the study. The study was approved by the ethics committee of the Carl von Ossietzky Universit¨at Oldenburg (Drs.EK/2021/068). 3 Results 3.1 Hearing device benefit Figure 3 shows the SNR without (label ‘u’) and with (label ‘p’) signal enhancement in the different tasks ‘FC’ (top left panel) and ‘ST-C’ (top middle panel) as well as the SRT (top right panel). The corresponding improvement by the signal enhancement is shown in the bottom panels of the same figure. The improvement achieved through directional filtering is approximately 4–5 dB for ‘dir 1’ and 7–8 dB for ‘dir 2’; however, the benefit is about 1 dB lower when measured in SRT compared to SNR. For the statistical analysis, three comparisons were performed. First, the device benefit, measured as SRT in a speech test (bottom right panel of Figure 3), was compared to the device benefit, measured as SNR in a free conversation (bottom left panel). Analysis of variances (see Table 3a for details) revealed main effect of ‘task’ (F= 25.86, p= 0) and ‘algorithm’ (F= 127.51, p= 0). The Tukey honest significant difference test was used for the post hoc analysis. On average, this test yielded a benefit that was 2.8 dB larger with ‘dir2’ than with ‘dir1’ (p < 1e−4), and 1.2 dB larger in ‘FC’ than in ‘ST-C’ (p < 1e−4); see Table 4a. The analysis of variances for the comparison of the benefit in ‘FC’ (bottom left panel in Figure 3) with benefit in ‘ST-C’ (bottom middle panel), both measured in terms of SNR, is shown in Table 3b. The analysis revealed significant main effects of ‘task’ (F= 8.21, p= 0.0066) and ‘algorithm’ (F= 383.93, p= 0), but no interaction effects. The benefit was 0.4 dB smaller in ‘ST-C’ than in ‘FC’, and 2.7 dB larger for ‘dir2’ than for ‘dir1’; see Table 4b. A direct comparison of the benefit measured in SNR with the benefit measured in SRT, both in the speech test, showed significant main effect of ‘measure’ (F= 14.97, p= 0.0004) and ‘algorithm’ (F= 157.35, p= 0), but no interaction effect, see Table 3c. The benefit was 0.8 dB larger when measured in terms of ‘SNR’, compared to the measurement in terms of ‘SRT’. The algorithm ‘dir2’ performed 2.7 dB better than ‘dir1’; see Table 4c. 3.2 Unprocessed SNR Figure 3, top left and top middle panel, shows the SNR before any spatial processing (data points with label ‘u’). During free conversation, it is around −6 dB (left panel). In the speech test (middle panel), the SNR depends on the level of signal enhancement, due to the adaptive nature of the speech test. Without spatial filtering (‘dir0’), it is around −6 dB; with ‘dir1’ it is −9.5 dB; and with ‘dir2’ it is around −11.5 dB. ANOVA revealed main effects of task (F= 84.92, p= 0) and algorithm (F= 25.45, p= 0), as well as an interaction effect (F= 30.77, p= 0). See Table 3e for details. The average SNR in quiet was 10.9 dB (standard deviation 0.7 dB). In free conversation, the SNRp, i.e., after signal enhancement, was increasing with increasing spatial selectivity (top left panel). In the speech test, there was also a slight increase in SNRp with increasing selectivity, which indicates an adverse effect of selectivity on SRT. If selectivity 9 SNR benefit must be determined using the method described in Hagerman and Olofsson (2004). This can be achieved post hoc by recording the audio signals of all interlocutors and re-rendering the conversation, including the recorded head movements. By rendering and processing the scene with alternating signs of the noise signal, the processed speech signal and noise signal can be estimated separately. This approach was applied by Hendrikse et al. (2020b), therefore this is not a fundamental limitation of this study. Another potential limitation is how natural the conversation feels in virtual reality. In our study, the interlocutors were represented by avatars in the form of virtual animated characters with a limited set of animations. Kothe et al. (2025) showed that a high level of co-presence can be achieved in such a setup; however, the avatars did not display any facial expressions, which are essential for expressing emotions in conversations (e.g., Ruusuvuori, 2012). One way to overcome these limitations is to estimate SNR based on acoustic simulation using near-mouth recordings of face-to-face conversations. For example, this could involve combining conversation speech and behaviour data, such as that published by Hinrichs et al. (2025), with the re-rendering approach of Hendrikse et al. (2020b). Evaluating hearing aids in free conversation in virtual reality instead of using conventional speech audiometry has implications for clinical tests. Here, the duration per condition was approximately five minutes, similar to the duration of a speech matrix test. One limitation is the requirement for confederates as interlocutors, which could be problematic in a clinical context. However, this could be overcome by using conversational agents, as advancements in machine learning methods mean they may achieve a level of naturalness similar to that of unknown interlocutors represented by their avatars. Significant changes in the complexity of hearing aid algorithms, such as the incorporation of machine learning algorithms and multimodal signal processing, necessitate a substantial enhancement in the naturalness of behaviour during hearing aid algorithm assessment in the near future. 5 Conclusions This study demonstrates that the benefit of hearing devices can be assessed in free conversation using telepresence technology and virtual reality. Using free conversation as a task allows for more natural signal-to-noise ratios to be achieved and for test subjects to exhibit more natural movement behaviour. Similarly, speech tests embedded in a virtual environment with distributed speakers can elicit movement behaviour similar to that observed in conversation, through visual cues and interactive elements; however, significant differences in behavior persist, resulting in changes to the measured algorithm benefit. This methodology can be applied to simulated idealised algorithms, as well as to more complex nonlinear and black-box processing. It provides a robust framework for evaluating hearing aid performance in more realistic conditions, ultimately increasing the ecological validity of hearing device evaluation. Funding This work was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation), Project-ID 352015383 – SFB 1330 project B1. References V. Best, E. Roverud, T. Streeter, C. R. Mason, and G. Kidd. The benefit of a visually guided beamformer in a dynamic speech task. Trends in Hearing, 21:1–11, 2017. ISSN 2331-2165. 16 doi: 10.1177/2331216517722304. H. Brumm and S. A. Zollinger. The evolution of the Lombard effect: 100 years of psychoacoustic research. Behaviour, 148(11-13):1173–1198, 2011. ISSN 0005-7959, 1568539X. doi: 10.1163/000579511X605759. URL https://brill.com/view/journals/beh/148/ 11-13/article-p1173_1.xml. S. D. Ewert, O. Buttler, and H. Hu. Computationally efficient parametric filter approximations for sound-source directivity and head-related impulse responses. pages 1–6. IEEE, 9 2021. ISBN 978-1-6654-0998-8. doi: 10.1109/I3DA48870.2021.9610923. URL https://ieeexplore. ieee.org/document/9610923/. M. Garnier and N. Henrich. Speaking in noise: How does the lombard effect improve acoustic contrasts between speech and ambient noise? Computer Speech & Language, 28(2):580–597, mar 2014. ISSN 0885-2308. doi: 10.1016/j.csl.2013.07.005. G. Grimm. Interactive low delay music and speech communication via network connections (OVBOX). Acta Acoustica, 8:1–7, Apr. 2024. doi: 10.1051/aacus/2024011. G. Grimm. head tracker arduino code. https://github.com/gisogrimm/headtracker, 2025. G. Grimm, H. Kayser, M. M. E. Hendrikse, and V. Hohmann. A gaze-based attention model for spatially-aware hearing aids. Number 282 in ITG-Fachbericht, pages 231–235, Berlin, 2018. VDE Verlag GmbH Berlin, Offenbach. ISBN 9783800747672. URL https://ieeexplore. ieee.org/document/8578029. G. Grimm, A. Kothe, and V. Hohmann. First order ambisonics field recordings for use in virtual acoustic environments in the context of audiology. doi: 10.5281/ZENODO.13341921, 2019a. G. Grimm, J. Luberadzka, and V. Hohmann. A toolbox for rendering virtual acoustic environments in the context of audiology. Acta Acustica united with Acustica, 105(3):566–578, May 2019b. ISSN 1861-9959. doi: 10.3813/AAA.919337. G. Grimm, M. M. Hendrikse, and V. Hohmann. Review of self-motion in the context of hearing and hearing device research. Ear and hearing, 41:48S–55S, 11 2020. ISSN 1538-4667. doi: 10.1097/AUD.0000000000000940. G. Grimm, M. Hendrikse, and V. Hohmann. Pub environment. doi: 10.5281/ZENODO.5886987, 2021. L. V. Hadley and J. F. Culling. Timing of head turns to upcoming talkers in triadic conversation: Evidence for prediction of turn ends and interruptions. Frontiers in Psychology, 13, Dec. 2022. ISSN 1664-1078. doi: 10.3389/fpsyg.2022.1061582. L. V. Hadley, W. O. Brimijoin, and W. M. Whitmer. Speech, movement, and gaze behaviours during dyadic conversation in noise. Scientific reports, 9:10451, 2019. doi: 10.1038/s41598-019-46416-0. B. Hagerman and ˚ A. Olofsson. A method to measure the effect of noise reduction algorithms using simultaneous speech and noise. Acta Acustica united with Acustica, 90:356–361, 2004. ISSN 1610-1928. 17 J. Hart, D. Onceanu, C. Sohn, D. Wightman, and R. Vertegaal. The attentive hearing aid: Eye selection of auditory sources for hearing impaired users. In Human-Computer Interaction – INTERACT 2009, volume 5726 LNCS, pages 19–35. Springer Berlin Heidelberg, 2009. ISBN 3642036546. doi: 10.1007/978-3-642-03655-2 4. M. Hartwig, V. Hohmann, and G. Grimm. Speaking with avatars - influence of social interaction on movement behavior in interactive hearing experiments. pages 94–98, 2021. doi: 10.1109/ VRW52623.2021.00025. URL https://conferences.computer.org/vrpub/#!/toc/2. M. Heldner and J. Edlund. Pauses, gaps and overlaps in conversations. Journal of Phonetics, 38:555–568, Oct. 2010. ISSN 0095-4470. doi: 10.1016/j.wocn.2010.08.002. M. M. E. Hendrikse, G. Llorach, V. Hohmann, and G. Grimm. Movement and gaze behavior in virtual audiovisual listening environments resembling everyday life. Trends in Hearing, 23: 1–29, 1 2019. ISSN 2331-2165. doi: 10.1177/2331216519872362. M. M. E. Hendrikse, G. Grimm, and V. Hohmann. Evaluation of the influence of head movement on hearing aid algorithm performance using acoustic simulations. Trends in Hearing, 24:1–20, Jan. 2020a. ISSN 2331-2165. doi: 10.1177/2331216520916682. M. M. E. Hendrikse, K. Schwarte, G. Grimm, and V. Hohmann. Generating hearing aid microphone recordings including head movement in virtual acoustic environments resembling everyday life. doi: 10.5281/ZENODO.3905920, 2020b. M. M. E. Hendrikse, T. Eichler, V. Hohmann, and G. Grimm. Self-motion with hearing impairment and (directional) hearing aids. Trends in Hearing, 26, Jan. 2022. ISSN 2331-2165. doi: 10.1177/23312165221078707. P. Hinrichs, V. Hohmann, and G. Grimm. Database of gaze, movement and communication behaviour in free triadic conversations. doi: 10.5281/ZENODO.17342367, 2025. M. Hodgson, G. Steininger, and Z. Razavi. Measurement and prediction of speech and noise levels and the lombard effect in eating establishments. The Journal of the Acoustical Society of America, 121(4):2023–2033, Apr. 2007. ISSN 1520-8524. doi: 10.1121/1.2535571. V. Hohmann, R. Paluch, M. Krueger, M. Meis, and G. Grimm. The virtual reality lab: Realization and application of virtual sound environments. Ear & Hearing, 41:31S–38S, 11 2020. ISSN 0196-0202. doi: 10.1097/AUD.0000000000000945. J. Holler and K. H. Kendrick. Unaddressed participants’ gaze in multi-person interaction: optimizing recipiency. Frontiers in Psychology, 6, Feb. 2015. ISSN 1664-1078. doi: 10.3389/fpsyg.2015.00098. C. T. Jespersen, B. C. Kirkwood, and J. Groth. Increasing the effectiveness of hearing aid directional microphones. Seminars in Hearing, 42(03):224–236, Aug. 2021. ISSN 1098-8955. doi: 10.1055/s-0041-1735131. G. Keidser, G. Naylor, D. S. Brungart, A. Caduff, J. Campos, S. Carlile, M. G. Carpenter, G. Grimm, V. Hohmann, I. Holube, S. Launer, T. Lunner, R. Mehra, F. Rapport, M. Slaney, and K. Smeds. The quest for ecological validity in hearing science: What it is, why it matters, and how to advance it. Ear & Hearing, 41:5S–19S, Nov. 2020. ISSN 0196-0202. doi: 10.1097/ AUD.0000000000000944. 18 A. Kothe, V. Hohmann, and G. Grimm. Effect of avatar head movement on communication behaviour, experience of presence and conversation success in triadic conversations. 2025. doi: 10.48550/ARXIV.2504.20844. G. Llorach, A. Evans, J. Blat, G. Grimm, and V. Hohmann. Web-based live speech-driven lip-sync. 2016. ISBN 9781509027224. doi: 10.1109/VS-GAMES.2016.7590381. R. Patel and K. W. Schell. The influence of linguistic content on the lombard effect. Journal of Speech, Language, and Hearing Research, 51(1):209–220, Feb. 2008. ISSN 1558-9102. doi: 10.1044/1092-4388(2008/016). V. Pulkki. Virtual sound source positioning using vector base amplitude panning. J. Audio Eng. Soc, 45:456–466, 1997. J. Ruusuvuori. The Handbook of Conversation Analysis, chapter Emotion, Affect and Conversation, pages 330–349. Wiley, Aug. 2012. ISBN 9781118325001. doi: 10.1002/9781118325001. ch16. L. Schilbach, B. Timmermans, V. Reddy, A. Costall, G. Bente, T. Schlicht, and K. Vogeley. Toward a second-person neuroscience. Behavioral and Brain Sciences, 36(4):393–414, July 2013. ISSN 1469-1825. doi: 10.1017/s0140525x12000660. F. Schwark, M. R. Sch¨adler, and G. Grimm. Data-driven optimization of parametric filters for simulating head-related transfer functions in real-time rendering systems. pages 1–10, 2022. K. Smeds, F. Wolters, and M. Rung. Estimation of signal-to-noise ratios in realistic sound scenarios. Journal of the American Academy of Audiology, 26(2):183–196, Feb. 2015. ISSN 1050-0545. doi: 10.3766/jaaa.26.2.7. R. S. Tweedy and J. F. Culling. Does the signal-to-noise ratio of an interlocutor influence a speaker’s vocal intensity? Computer Speech & Language, 28(2):572–579, Mar. 2014. ISSN 0885-2308. doi: 10.1016/j.csl.2013.06.005. S. van de Par, S. D. Ewert, L. Hladek, C. Kirsch, J. Sch¨utze, J. Llorca-Bof´ı, G. Grimm, M. M. Hendrikse, B. Kollmeier, and B. U. Seeber. Auditory-visual scenes for hearing research. Acta Acustica, 6:55, 2022. ISSN 2681-4617. doi: 10.1051/aacus/ 2022032. URL https://acta-acustica.edpsciences.org/articles/aacus/full_html/ 2022/01/aacus210093/aacus210093.html. R. Vertegaal, R. Slagter, G. van der Veer, and A. Nijholt. Eye gaze patterns in conversations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 301–308. ACM Press, Mar. 2001. doi: 10.1145/365024.365119. K. C. Wagener and T. Brand. Sentence intelligibility in noise for listeners with normal hearing and hearing impairment: Influence of measurement procedure and masking parameters. International Journal of Audiology, 44(3):144–156, Mar. 2005. ISSN 1708-8186. doi: 10.1080/14992020500057517. K. C. Wagener, V. K¨uhnel, T. Brand, and B. Kollmeier. Entwicklung und Evaluation eines Satztests f¨ur die deutsche Sprache Teil I–III (development and evaluation of a german sentence test). Zeitschrift f¨ur Audiologie, 38, 1999. 19 N. L. Westhausen, H. Kayser, T. Jansen, and B. T. Meyer. Real-time multichannel deep speech enhancement in hearing aids: Comparing monaural and binaural processing in complex acoustic scenarios. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32:4596– 4606, 2024. ISSN 2329-9304. doi: 10.1109/taslp.2024.3473315. F. Wolters, K. Smeds, E. Schmidt, E. K. Christensen, and C. Norup. Common sound scenarios: A context-driven categorization of everyday sound environments for application in hearingdevice research. Journal of the American Academy of Audiology, 27:527–540, 2016. ISSN 2157-3107. doi: 10.3766/jaaa.15105. 20