Between Bouba and Kiki Lies Grougi: Modelling the Rough Middle Ground of Sound–Shape Correspondences

Between Bouba and Kiki lies Grougi: Modelling the Rough Middle Ground of Sound–Shape Correspondences Asterios Zacharakis1[0000000320246924], Konstantinos Velenis1[0009000672994763],and Emilios Cambouropoulos1[0000000268874287] Aristotle University, Thessaloniki, Greece [email protected] Abstract. Cross-modal correspondences between audition and vision have been extensively studied in relation to pitch and loudness, but timbre remains more challenging to capture empirically and model acoustically due to its multidimensional nature. Nevertheless, recent research has shown that timbral qualities such as brightness, roughness, and clarity can be reflected in the visual properties of images. Building on the well-known Bouba/Kiki effect, the present study extends this framework to musical timbres and investigates visual forms that span a continuum ranging from full roundness to high angularity. We hypothesised not only that visually induced roughness would occupy the perceptual middle ground between these two extremes, but also that this categorisation would be meaningfully reflected in auditory perception. To test this, 27 participants were asked to associate 30 short sound stimuli with visual shapes using a morphing interface that enabled continuous transformation from a circle to a jagged, starlike form. The results revealed strong, non-random associations between sounds and the three shape categories (smooth, rough, and sharp). Acoustic modelling further supported these distinctions, showing that established models of auditory roughness and sharpness effectively predicted participants’ assignments of the sound stimuli within these categories. Together, these findings offer empirical backing for perceptually grounded mappings between sound and image, with potential applications in artistic creation, education, and interactive media. Keywords: timbre ·roughness ·sharpness ·cross-modal correspondences ·shapes. All rights remain with the authors under the Creative Commons Attribution 4.0 International License (CC BY 4.0). Proc. of the 17th Int. Symposium on Computer Music Multidisciplinary Research, London, United Kingdom, 2025 Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 93 A. Zacharakis et al. 1Introduction Musical sounds —even when experienced in isolation from their original context— can convey extra-musical meaning [15,8], often by evoking qualities that extend beyond audition in the strict sense [15]. Growing interest in timbre semantics over the past fifteen years has advanced our understanding of how such qualities are cognitively organised and what acoustic features underlie them (for an overview of the domain until 2019 see [33,25]). Some of the most notable contributions have organised timbre descriptions into semantic categories, many of which involve cross-modal loans. As early as 1941, Lichte identified brightness and roughness as important dimensions in the description of timbre —qualities that draw from the visual and tactile domains, respectively [12]. Since then, the prominence of brightness and roughness dimensions (and their variations) within timbral semantics has been reinforced by numerous studies across diverse linguistic backgrounds [5,4,27,2,39,23,22]. At the same time, it should not be overlooked that research has also revealed nuanced variations within these primary dimensions, as well additional semantic categories altogether such as, for example, full,woody,brassy/metallic,hollow,open,percussive,etc.[37,20,19]. Given that a considerable portion of timbral semantics originates from crossmodal correspondences [30], it has been argued that systematically exploring such associations offers a fruitful path for future research on extra-musical meaning conveyed by sound [25]. Indeed, a growing body of recent work has focused on potential links between timbral and visual or tactile qualities [24,32,31,34,11,21] and even between timbral and olfactory qualities [35,36]. While brightness is arguably the most well-established semantic dimension of timbre, the tactile-related quality of roughness has drawn scholarly attention since the time of Helmholtz [7], and consistently emerges as the most widely agreed-upon attribute across linguistic groups and levels of musical expertise [39,37,32,38,22]. Roughness constitutes one of the principal qualities of surfaces and is primarily assessed through touch. That said, certain aspects of roughness can also be evaluated visually [6]. With this in mind, we recently explored potential relationships between various graphical manifestations of roughness (i.e., roughness, noisiness, granularity, and sharpness) and sound stimuli specifically designed to convey these qualities in the auditory domain [40] using a fivealternative forced-choice (5AFC) experimental design. Our data showed that the audio-image correspondences were above chance overall. At the same time, the rough and granular visual categories could be acoustically differentiated using an audio feature originating from the Modulation Power Spectrum (MPS), known to predict auditory roughness [3,22] and Zwicker’s sharpness model [41]. Emboldened by these preliminary results, we hereby proceed to a simplified variation of this experiment that is partly inspired by the Bouba/Kiki effect [9,17,16], which has demonstrated the systematic mapping between words and shapes. A bit more than a decade ago, Adeli and colleagues [1] also expanded the Bouba/Kiki paradigm from word-shape to sound-shape associations. Their work demonstrated that softer timbres tended to be associated with rounder, Bouba-style forms, while harsher ones were linked to spikier, Kiki-style shapes. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 94 Bouba-Grougi-Kiki However, this work identified associations between sound and shapes primarily at the source-cause level (e.g., piano, saxophone, gong) rather than with specific acoustic correlates. In the current experiment, participants were given the option to adjust the properties of a shape rather than choosing from a predefined set of images. Thus, using a single slider, it was possible to morph a full circle into a spikier shape composed of 20 trapezoids, which could gradually evolve into triangles by reducing the length of their shorter side (see Figure 1). The underlying assumption was that this continuous visual transformation would progress from smooth to sharp, passing through an intermediate stage perceived as rough —much like how the lexical blend Grougi,positionedbetweenKiki and Bouba, evokes a rough quality. This design provided a neat and efficient way to collect data on all three perceptual qualities (smoothness, roughness and sharpness) using a single, intuitive interface, described in greater detail in the Method section. These investigations into audio-visual correspondences are part of a broader research project entitled Soundsketcher [29], which aims to automatically translate audio into perceptually meaningful visual representations for the creation of aural scores in both creative and educational contexts. 2Method 2.1 Sound stimuli The experimental dataset consisted of 30 short audio stimuli (1.5–3 seconds), drawn from various sound libraries (e.g., Freesound, MUMS [14]), adapted from past studies [13], or designed from scratch to equally represent the three sonic qualities of interest (smooth, rough, and sharp; 10 stimuli per category). To construct this set, the research team, together with undergraduate students from amusiccognitionmodule,compiledabroaderpoolofcandidatesounds,which were then discussed and evaluated through informal listening across the three semantic categories, ultimately yielding the final 30 stimuli. The selected stimuli were subsequently loudness equalised through an iterative listening process conducted within the research team. 2.2 Visual variable The visual response element was a shape-morphing interface presented in the form of a continuous slider (Figure 1). The base shape was a perfect circle. As participants moved the slider to the right, the shape gradually morphed into increasingly jagged, star-like forms with higher angular modulation. This transformation was continuous and allowed for fine-grained adjustments across the full range of possible shapes. The slider corresponded to a hidden scale ranging from 0 to 20 that was not revealed to the participants. This morphing continuum was designed to provide participants with a straightforward and intuitive way to explore the transition from round to angular shapes. The basic assumption here was that the midpoint between complete roundness (i.e., smoothness) and full angularity (i.e., sharpness) would signify roughness. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 95 A. Zacharakis et al. Fig. 1. Visual shape continuum reflecting textural morphing from smooth to rough and finally sharp. Participants adjusted a slider to select the shape that best represented the sound they heard. The shape gradually transformed from a smooth circle (left), to rough undulating contours (center), and finally to sharp, pointed spikes (right). The numbers shown beneath each image represent the corresponding slider value, noting that this was not visible to participants. 2.3 Participants A total of 27 participants took part in the study. The participant pool was diverse in terms of gender and musical background. Most participants identified as male (19), followed by female (8), with one participant choosing not to disclose their gender. Musical experience levels varied considerably: the sample included 9 professional musicians, 5 semi-professionals, 7 amateurs and 6 non-musicians. The majority of participants reported playing a musical instrument, with instrument types including piano, guitar, and electric bass, to more specialised ones such as the double bass. Regarding listening equipment, most participants used closed-back (overear) headphones, while others used on-ear headphones or studio monitors. All participants reported normal hearing and normal or corrected-to-normal vision. Participation was voluntary and no compensation was provided. 2.4 Experimental procedure The experiment was conducted online via a browser-based interactive interface. In each trial, participants were presented with an initial circle shape representing the smoothness and a slider beneath it at value 0. Their task was to adjust the shape of the image by moving the slider until they felt it best represented the quality of each sound stimulus. They could trigger audio playback by clicking abuttonlabeled“Press"A"orclicktoplaythesound”.Atthebeginningof the session, participants were presented with three practice sounds to familiarize themselves with the procedure and the interface. These practice trials were excluded from the data analysis. In the main experiment, the presentation order of the 30 audio stimuli was randomised for each participant to counterbalance potential order effects. There were no restrictions on how many times participants Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 96 Bouba-Grougi-Kiki could listen to a sound before making their judgment. Once they were satisfied with their choice, they clicked the “Next Sound” button to proceed. This process was repeated for all 30 randomised trials following the initial practice phase. Throughout the experiment, the system recorded the slider position (which corresponded to a specific shape configuration), the associated sound ID, and the timestamp of the selection. The full set of audio stimuli, along with the shape generation code used in the experiment, is publicly available on GitHub. 3Results 3.1 Analysis of responses The primary goal of the present analysis was to determine whether the behavioural responses exhibited systematic trends. To this end, the original continuous variable (ranging from 0 to 20) was transformed into a categorical variable by dividing the scale into three regions: [0–5] for smooth, [6–15] for rough, and [16–20] for sharp. This categorisation was informed by both visual inspection of the stimuli’s shape transformations (see Figure 1) and by the goal of optimising Cramér’s V and the fit of the acoustic models (see below). Subsequently, a Chi-square on the contingency table of responses (Table 1) test revealed a significant deviation from uniformity in category choices across sound stimuli, 2(58) = 478.89,p<0.00001,indicatingasystematic,nonrandom pattern in how sounds were attributed to the three categories. The effect size, measured by Cramér’s V (V=0.544), reflects a relatively strong structure in these category assignments [18]. Table 1. Contingency table showing participant selections (counts) by semantic category for each stimulus. Stimulus Smooth Rough Sharp Stimulus Smooth Rough Sharp moog-noise 4 15 8 sharp 1 5 21 quaver-pokes 1 6 20 tuba-rough 3 15 9 granulator 7 18 2 ebass 23 3 1 dissonant-harm 2 7 18 fly 4 17 6 rain-noise 6 10 10 glass 21 3 3 storm-noise 4 17 6 guitar-rough 2 16 9 anchor-chain 2 17 8 guitar-smooth 23 2 2 scary-rasp 3 2 22 harp 1 9 17 granular 6 15 6 saxophone 2 19 6 bowed-cymbal 2 7 18 sharp-saw 6 5 16 cello 21 4 2 sharp-kirk 8 8 11 flute 17 6 4 smooth-bass 24 2 1 hammond 19 5 3 smooth-sine 23 3 1 pan-flute 17 7 3 smooth2 20 3 4 SP1Désintégrations 5 5 17 steelsea-sharp 7 13 7 Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 97 A. Zacharakis et al. At the individual level, Figure 2 illustrates the strength of associations between each sound and the three response categories. Subfigure 2a presents the significant uncorrected standardised residuals from the Chi-square test of independence, while Subfigure 2b shows the significant standardised residuals after False Discovery Rate (Benjamini-Hochberg) correction, which enhances sensitivity to meaningful relationships while controlling false discoveries. The uncorrected coefficients reveal 27 out of 30 significant associations, while even the stricter corrected coefficients still indicate significant associations for 24 out of 30 sounds with one of the visual categories. Smooth Rough Sharp Categories moog-noise quaver-pokes granulator dissonant-harm rain-noise storm-noise anchor-chain scary-rasp granular bowed-cymbal cello flute hammond pan-flute sharp tuba-rough ebass fly glass guitar-rough guitar-smooth harp saxophone sharp-saw sharp-kirk smooth-bass smooth-sine smooth2 steelsea-sharp Sounds Uncorrected Standardised Residuals 3.742 2.443 3.093 2.443 4.392 3.742 4.392 4.717 4.392 3.418 2.085 3.096 2.759 2.759 2.085 2.085 2.759 2.422 3.433 3.825 3.147 4.503 3.147 2.809 4.164 2.809 2.47 -2.754 -2.429 -2.429 -2.104 -2.429 -2.754 -2.104 -2.429 -2.754 -2.429 -2.295 -2.295 -2.295 -2.274 -2.274 -2.613 -2.274 -2.613 -2.613 -3 -2 -1 0 1 2 3 [] (a) Smooth Rough Sharp Categories moog-noise quaver-pokes granulator dissonant-harm rain-noise storm-noise anchor-chain scary-rasp granular bowed-cymbal cello flute hammond pan-flute sharp tuba-rough ebass fly glass guitar-rough guitar-smooth harp saxophone sharp-saw sharp-kirk smooth-bass smooth-sine smooth2 steelsea-sharp Sounds FDR-Corrected Standardised Residuals 3.742 2.443 3.093 2.443 4.392 3.742 4.392 4.717 4.392 3.418 3.096 2.759 2.759 2.759 2.422 3.433 3.825 3.147 4.503 3.147 2.809 4.164 2.809 2.47 -2.754 -2.429 -2.429 -2.429 -2.754 -2.429 -2.754 -2.429 -2.613 -2.613 -2.613 -3 -2 -1 0 1 2 3 Not Significant (b) Fig. 2. Heatmap of the standardised residuals from the Chi-square test of independence, with blue indicating negative values and yellow representing positive values, where non-significant values are shown in white. (a) Significant matchings for the uncorrected standardised residuals. (b) Significant matchings after False Discovery Rate (Benjamini-Hochberg) correction. 3.2 Auditory modelling After establishing a strong systematic relationship between sounds and visual categories, we sought to interpret these relationships through acoustic predictors. To this end, we computed conditional probabilities reflecting the likelihood of each visual category being chosen given a specific sound stimulus, P(Visual | Auditory). Responses were aggregated into a 30×3contingencytable,capturingthe distribution of selections across conditions. A multinomial regression was then performed on this contingency structure to assess the extent to which acoustic predictors systematically influenced visual choices. The modelling explored some audio features that have been linked with auditory roughness and sharpness. The roughness models by Vassilakis [28] and Sethares [26] were implemented using the MIR Toolbox [10] (window size: 40ms, 50% overlap). Additionally, the roughness and sharpness models based on the work by Fastl & Zwicker [41] were computed using the MATLAB functions acousticRoughness and acousticSharpness. Furthermore, a metric based on the Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 98 Bouba-Grougi-Kiki Modulation Power Spectrum (MPS), known to predict auditory roughness [3,22], was also calculated. The MPS is the 2D-Fourier transform of a spectrogram that quantifies the distribution of modulation energy across the temporal and spectral dimensions. In this study, the 2D Fourier transform was applied to a loudness spectrogram estimated using MATLAB’s acousticLoudness function (window size: 50 ms, hop size: 2ms), which implements Zwicker’s method for loudness estimation (ISO 532-1) [41], rather than a spectrogram derived from a standard Fourier transform. MPS-roughness was calculated as the ratio of energy within the 30–150 Hz range of the temporal modulation axis to the total positive modulation energy, using a slight variation of the metric proposed by Rosi et al. [22]. Amultinomialregressionmodelwasthenemployedtopredictvisualcategory probabilities through the audio features. A log transformation was applied to the predictor table X,Xlog = log10(X+"),with "= 104to suppress positive outliers in the audio features and improve the stability and interpretability of regression coefficients. Standardisation (z-scoring) was subsequently performed to ensure comparability across features. Model evaluation relied on log-likelihoodbased metrics, including McFadden’s R2and Nagelkerke’s R2,toassessexplanatory power. Rather than relying on discrete classifications, predictions were derived from estimated probability distributions over the visual categories, providing a more continuous interpretation of the relationship between predictors and response tendencies. Table 2. Multinomial regression results predicting the selection of the shape categories smooth and rough (vs. sharp) based on acoustic features. The sign and magnitude of the beta coefficients indicate how the probability of selecting a given category is influenced by each feature, accompanied by 95% confidence intervals and significance levels. Model fit: Log-Likelihood (L)=-12.77;McFadden’sR2= 0.61, Nagelkerke’s R2= 0.83; Prediction Accuracy = 100%; Log Loss = 11.18. These metrics reflect the model’s goodness-of-fit, classification performance, and uncertainty in predictions. Likelihood Ratio Test (LRT) p-values: Vassilakis’ roughness = 0.0001, Zwicker’s sharpness = 0.0000, indicating that both predictors are statistically significant. Predictors Betas p-Value 95% CI Cat. 1: [Smooth] Intercept -337.7 <.0001 [-343.05 , -332.38] Vassilakis’ roughness -845.3 <.0001 [-849.5,-841.09] Zwicker’s sharpness -2608 <.0001 [-2616.3, -2599.6 ] Cat. 2: [Rough] Intercept 775.7 <.0001 [770.67 , 780.76] Vassilakis’ roughness 189.6 <.0001 [185.66,193.6] Zwicker’s sharpness -1120.5 <.0001 [-1127 , -1114] Several pairwise combinations of the five examined descriptors resulted in significant and comparable models, all of which featured one of the roughness descriptors as the first predictor and Zwicker’s sharpness as the second. The Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 99 A. Zacharakis et al. model reported here (Table 2) yielded the highest McFadden’s R2(.612) and features Vassilakis roughness as the first predictor and Zwicker’s sharpness as the second. This model also demonstrated meaningful interpretability, as all of the betas were statistically significant. The beta coefficients presented in Table 2 reflect how the two auditory predictors influence the likelihood of a shape falling into the first two categories (i.e., smooth and rough), relative to the reference category (sharp). According to the model, the likelihood of a stimulus falling into the smooth category over the sharp category increases substantially as Vassilakis’ roughness decreases, and even more so as Zwicker’s sharpness decreases. In contrast, while higher Vassilakis’ roughness increases the likelihood of the rough category relative to sharp, lower Zwicker’s sharpness continues to increase the likelihood of rough over sharp. -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 Vassilakis' roughness -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 Zwicker's Sharpnes Class Separation in Log-Transformed Feature Space moog-noise quaver-pokes granulator dissonant-harm rain-noise storm-noise anchor-chain scary-rasp granular bowed-cymbal cello flute hammond pan-flute sharp tuba-rough ebass fly glass guitar-rough guitar-smooth harp saxophone sharp-saw sharp-kirk smooth-bass smooth-sine smooth2 steelsea-sharp Smooth Rough Sharp Fig. 3. Separation of visual categories in the feature space defined by Vassilakis’ roughness and Zwicker’s sharpness (log-transformed). Each point represents a stimulus, colored and shaped according to the most probable visual category (smooth, rough, or sharp) based on participant responses. The clear separation between regions illustrates the strong contribution of these auditory features to the categorisation of shape responses. While the magnitude of the model’s beta coefficients and the perfect classification accuracy might raise concerns about potential overfitting, the twodimensional feature space defined by the log-transformed Vassilakis’ roughness and Zwicker’s sharpness (Figure 3) shows a clear separation between the visual categories. The fact that stimuli associated with each category —based on the most frequent visual assignment by participants— occupy distinct regions in the feature space supports the conclusion that these auditory features play a meanProc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 100 Bouba-Grougi-Kiki ingful role in shaping perceptual categorisation. Nonetheless, the robustness of this model should ideally be confirmed through independent validation. Across multiple 5-fold cross-validation runs, the multinomial regression model trained on auditory features achieved mean classification accuracies ranging from 76.7% to 86.7%, with an overall average of approximately 80.0% (±11.4%). The corresponding mean log loss values averaged around 9.6 (±1.9), indicating relatively low predictive confidence despite high classification accuracy. This suggests that the auditory features offer meaningful discriminatory power for modelling sound–shape correspondences, although the elevated log loss points to limited certainty in the probability estimates, likely a consequence of the small dataset size. 4Discussion These preliminary findings offer promising insights into cross-modal correspondences between vision and audition. In particular, our hypothesis of a visual continuum linking smoothness to sharpness via roughness appears to be supported by the categorisation and analysis of our visual variable. As mentioned in the Results section, the discretisation of the continuous visual variable into three distinct regions (smooth,rough,andsharp)wasguidedbyacombination of visual inspection of the generated shapes and iterative refinement based on optimisation of the Cramér’s V values (here quantifying the strength of association between visual categories and the sound stimuli) and goodness-of-fit measures from the subsequent acoustic modelling. A Chi-square test of independence identified audio-visual associations, which exceeded chance levels both overall and across individual stimuli. Indeed, the distribution of our 30 audio stimuli across the three visual categories was approximately even (see Figure 2) and broadly aligned with our initial assumptions on perceived smoothness, roughness and sharpness. Thus, our results extend the well-known Bouba/Kiki effect from speech-based sounds to musical timbres, in agreement with previous research [1], while also revealing an intermediate perceptual category linked to roughness. Based on this finding, we subsequently sought to achieve an acoustic modelling of the visual categories employing a number of established models of auditory roughness and sharpness. The modelling further strengthened our assumptions on the distinctions achieved between smoothness, roughness and sharpness. Sounds with higher roughness values, as measured by Vassilakis’ model [28], were more likely to be assigned to the rough category and less likely to be assigned to smooth,relativetosharp. Additionally, higher values of Zwicker’s sharpness [41] reduced the likelihood of assignment to both smooth and rough,againrelative to sharp. This pattern is clearly illustrated in the scatterplot of Figure 3, where stimuli with higher probabilities of being rated as sharp (green diamonds) tend to cluster in the upper-right region of the plot, characterised by both high Zwicker’s sharpness and high Vassilakis’ roughness. Stimuli with a higher likelihood of being categorised as rough (red squares) appear just below and further to the right, defined by lower acoustic sharpness but even higher acoustic roughProc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 101