Interactive machine learning for music classification

MASTER’S THESIS Interactive machine learning for music classification Author: Danila Alexandrovich Danilin u254724 [email protected] Supervisors: Dmitry Bogdanov Music Technology Group (UPF) dmitry.bogdano[email protected] Pablo Alonso Jimenez Music Technology Group (UPF) [email protected] Music Technology Group, Universitat Pompeu Fabra, Barcelona August 31, 2025 Abstract Audio embeddings are a promising approach to music representation, in part thanks to their ability to extract complex patterns from audio data; the predictive power of audio embeddings is utilized for a semantically meaningful, two-dimensional visualization of music data in a user interface (UI) which has been developed as part of this thesis research. As a contribution to ongoing research on the intersection between music information retrieval (MIR) and interactive machine learning (IML), the UI allows users to iteratively train a classifier for numerous audio classification tasks. As part of this research, the certainty-based class prediction uncertainty (CPU) heuristic, and the dataset coverage (DC) heuristic are proposed; these heuristics are shown to identify informative samples in music collections, and their efficiency is objectively evaluated by means of simulated, iterative active learning (AL) classification tasks for 6 different embedding-dataset pairs. The objective evaluations have shown promising results, in which high classification accuracies are shown to be achieved in fewer iterations in AL classification tasks. 1 Acknowledgments I would like to thank my family, friends, thesis supervisors, professors and academic peers for abundantly providing motivation, support and lots of beautiful moments during the next, insightful and musical step in my academic journey, I am very grateful. I would also like to thank my teammates from the AI Song Contest in 2022 for introducing me to the field of computation musicology, for that I am also very grateful. The fruits of the music-human symbiosis are profound, and its echoes continue to inspire humankind to connect, find meaning, and do good. In part thanks to this symbiosis, we have been able to reach the current levels of societal and technological development. A guiding star and a faithful companion in our journey through life, it continues to patiently nudge humankind, to stay on track and be a meaningful part of this reality; that is the beauty of music. 2 Table of Contents 1 Introduction 4 2 Background 5 2.1 Audioembeddings ...................................... 5 2.1.1 Overview of existing audio embedding models . . . . . . . . . . . . . . . . . . . 5 2.1.2 Latentspace ..................................... 6 2.2 Human-in-the-loop machine learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2.1 Activelearning(AL)................................. 7 2.2.2 Interactive machine learning (IML) . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.3 Existing work on AL/IML and audio . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3 Heuristics for (re-)annotation and classification tasks 9 3.1 Class prediction uncertainty (CPU) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.2 DatasetCoverage(DC) ................................... 9 4 User Interface 10 4.1 DashPlotly.......................................... 10 4.2 SettinguptheUI....................................... 11 4.3 UI fundamentals: a basic music classification loop . . . . . . . . . . . . . . . . . . . . . 12 5 Methods 15 5.1 Objectiveevaluation ..................................... 15 5.1.1 Audioembeddings .................................. 15 5.1.2 Datasets........................................ 16 5.1.3 ALstrategies..................................... 17 5.2 Subjectiveuserevaluation.................................. 18 5.2.1 Questionnaire..................................... 19 5.2.2 Participants...................................... 19 6 Results 20 6.1 Objective evaluation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 6.1.1 Overview ....................................... 20 6.1.2 MAESTandGTZAN ................................ 21 6.1.3 MAEST and Moods MIREX . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 6.1.4 MAEST and Freesound Loop Dataset . . . . . . . . . . . . . . . . . . . . . . . 23 6.1.5 CLAPandGTZAN ................................. 24 6.1.6 CLAPandMoodsMIREX ............................. 25 6.1.7 CLAP and Freesound Loop Dataset . . . . . . . . . . . . . . . . . . . . . . . . 26 6.2 Subjectiveevaluations .................................... 27 6.2.1 Participant1 ..................................... 27 6.2.2 Participant2 ..................................... 27 6.2.3 Participant3 ..................................... 28 6.2.4 Participant4 ..................................... 29 7 Discussion and future work 30 7.1 CPUandDCheuristics ................................... 30 7.2 UserInterface......................................... 30 7.3 Other directions for future research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 8 References 32 ‘ 3 1 Introduction In his manifesto on the strengths and limitations of contemporary music information retrieval (MIR) research, Widmer made an interesting point: computers cannot distinguish between songs that an individual might find boring or interesting (Widmer, 2016). Understanding individual preferences, which is necessary for answering questions of a nature similar to the one posed by Widmer in 2016, requires the consideration of a very wide range of factors, including cultural, educational, regional and social background, age, and even personality traits (Bogdanov, 2013; Matrosova, 2024), but also the highly dynamic nature of listener context (Ben Sassi & Ben Yahia, 2020). In their works from the 2010s, both Bogdanov and Widmer have foreseen that computational algorithms will have a deeper understanding of music content and individual (contextual) music preferences (Bogdanov, 2013; Widmer, 2016). In line with this foresight, there currently exist deep learning-based methods which automatically process audio signals into semantically meaningful, lower-dimensional representations (Zaman et al., 2023); commonly referred to as audio embeddings (Alonso-Jim´enez et al., 2023). Audio embedding models have the ability to extract highly complex patterns from audio data, and yield higher accuracies in classification tasks compared to traditional classification methods (Zaman et al., 2023). The exceptional performance of audio embeddings in classification tasks form the core foundation on which this research is built; in this research, an interactive user interface (UI) is introduced, in which users are able to explore and annotate their music collections. The predictive power of audio embeddings is utilized for a semantically meaningful, two-dimensional visualization of music data, and it forms the basis for two ML-based heuristics which are proposed as part of this research; the certainty-based class prediction uncertainty (CPU) heuristic, and the dataset coverage (DC) heuristic. In the UI, users can iteratively train a classifier which can return class predictions for audio files. Consistent with the active learning (AL) paradigm, the proposed heuristics highlight informative tracks which are suitable candidates for user annotation in order to train the classifier efficiently. By annotating informative data points, as opposed to random data points, high classification accuracies can be achieved within fewer training iterations (Joshi et al., 2012; Konyushkova et al., 2017). The efficiency of the CPU and DC heuristics in highlighting informative data points is reflected in promising results from AL simulations with a number of embedding-dataset pairs, carried out as part of this research. By allowing the user to steer the classifier training process in accordance with their own personal taxonomies, this UI is a contribution to ongoing research on the intersection between MIR and interactive machine learning (IML). Besides its contribution towards a better understanding of music collections, the identification of informative samples in music collections has the potential to advance musicological research through efficient dataset creation, among other useful applications. Furthermore, the UI is indifferent to both chosen audio embedding and provided music collection alike, granting users the freedom to inspect their music collections from different perspectives, while simultaneously accounting for future efforts made towards the development of general-purpose audio embeddings which achieve high accuracies in classification tasks (Alonso-Jim´enez et al., 2023). In this thesis document, the advantages and potential drawbacks of the proposed approach to music classification are discussed in detail, with the aim of shining a light on possible directions for future research in computational musicology. Lastly, there is an important nuance to point out; in this research, certainly not all of the musical understanding is offloaded to computational models and algorithms. You, the reader, have a crucial role to play as well. Therefore, I invite you to interact with the environment1. I wish you a pleasant reading, and a fruitful journey in music exploration and understanding. 1https://github.com/danilkaru/interactive-ML-music-classifier 4 2 Background 2.1 Audio embeddings One of the major challenges in computational musicology is the development of data structures which appropriately represent music, and, more abstractly, encode musical meaning (Volk et al., 2011). A long-standing, related challenge in scientific literature has been the attempt to encode musicological and semantic information about audio files in descriptive representations (Serra et al., 2013). Even though audio files2accurately capture the linear progression of music pieces in the temporal domain, they do not implicitly encode abstract musical features such as genre, key, tempo and mood (Serra et al., 2013). Therefore, a common approach in the field of MIR has been the extraction of musical features at different levels of abstraction with the use of computational algorithms (Serra et al., 2013). With the recent advancements in the field of deep learning (DL), a new, compact representation for audio has been widely researched in scientific literature, commonly referred to as audio embeddings. Audio embeddings are representations of audio signals with a relatively low dimensionality (Alonso-Jim´enez et al., 2023), and they are generated from neural networks which are trained on audio datasets (Zaman et al., 2023), consequently capturing a multitude of complex musical features which are stored in a compact, vector-based form. Figure 1: Part of a CLAP audio embedding (Chen et al., 2022; Elizalde et al., 2022; Wu et al., 2024) representing the reggae.00007 audio file from the GTZAN (Tzanetakis & Cook, 2002) dataset. The full embedding vector has a dimensionality of (1024,). Audio embeddings have proven to be a very significant contribution to the field of MIR, in part due to their state-of-the-art performance in audio classification tasks (Zaman et al., 2023; Zhang et al., 2025). Audio embeddings outperform traditional computational classification models, which typically rely on an intermediate manual or algorithmic music feature extraction step; audio embedding models, however, do feature extraction automatically, and are often able to capture more abstract patterns which can help with distinguishing between classes more accurately (Schmid et al., 2023; Zaman et al., 2023). The performance of audio embedding models in classification tasks is dependent on the datasets which they are trained on; to ensure that high classification accuracies are achieved, audio embedding models must be trained on very large datasets, requiring significant computational resources and human annotation efforts (Zaman et al., 2023). However, after the model has been trained, class predictions for new input data can be retrieved with relatively simple classifiers such as shallow multilayer perceptrons (MLPs) (Alonso-Jim´enez et al., 2023; Schmid et al., 2023). Using knowledge from pre-trained models for downstream classification tasks is a concept commonly referred to as transfer learning (Van Den Oord et al., 2014). 2.1.1 Overview of existing audio embedding models Current audio embedding models are generally part of one of two broad categories: task-specific audio embeddings, and general-purpose audio embeddings. A further distinction can be made with respect 2.wav, .mp3, .ogg etc. 5 to the underlying neural network architectures in the embedding models; the majority of embedding models are based on convolutional neural networks (CNNs), recurrent neural networks (RNNs), autoencoders, transformers, and hybrid architectures which integrate one or more aforementioned approaches (Zaman et al., 2023). Task-specific audio embeddings are typically developed with the aim to achieve high accuracies in specific classification tasks; commonly researched directions include genre classification, mood classification, speech recognition, environmental sound classification, and more (Zaman et al., 2023). Examples of embedding models which achieved state-of-the-art classification accuracies in genre classification tasks include the CNN-based musicnn (Pons & Serra, 2019), and the more recent, transformer-based MAEST (Alonso-Jim´enez et al., 2023) models. Both models were trained on large and annotated datasets under the supervised learning paradigm. Inspired by language learning in humans, Baevski et al. have attempted to create a model which can extract complex features from small datasets; the resulting, self-supervised wav2vec 2.0 model achieved state-of-the art performance in speech recognition classification tasks (Baevski et al., 2020). A significant amount of research is also being dedicated to the development of general-purpose audio embedding models which can simultaneously address speech, audio event and audio tasks (AlonsoJim´enez et al., 2023). Measuring the performance of general-purpose audio embedding models is commonly done with the HEAR (Turian et al., 2022) and HARES (Wang et al., 2021) benchmarks (Alonso-Jim´enez et al., 2023; Schmid et al., 2023), which cover a variety of audio classification tasks for speech, music and environmental sound (Schmid et al., 2023). Examples of general audio embedding models include CLAP, introduced by Elizalde et al. in 2022, which is a model which produces embeddings from audio and text input; a key property of the CLAP model is its ability to understand audio through natural language, and vice versa (Elizalde et al., 2022). The CLAP model achieved state-of-the-art performance in 16 downstream tasks across 8 different audio domains (Elizalde et al., 2022). Efforts have also been made to fine-tune task-specific embedding models to achieve high performances in domains beyond the one which the model was originally trained for; in their 2022 research, Ragano et al. have successfully fine-tuned the wav2vec 2.0 model to achieve high classification accuracies for music classification tasks (Ragano et al., 2022). More recently, Schmid et al. have introduced the mn01,mn10 and mn30 models, and succeeded in reducing computational resource demands compared to earlier general audio embedding models, while simultaneously achieving state-of-the-art performance in HEAR benchmark classification tasks (Schmid et al., 2023). 2.1.2 Latent space A powerful property of audio embeddings is that their vectors can be seen as coordinates in a multidimensional, semantically meaningful latent space; embedding vectors of ’similar’ music pieces have coordinates which are close to each other. Similarity of embedding vectors in the latent space is dependent on the chosen dataset and target classification task (Tahiro˘glu & Wyse, 2024); for instance, if the model was trained to predict genres, then similarity in the latent space will mostly be related to genre. The latent space of audio embeddings has a number of useful applications. One such application is the visualization of music; in 2022, Tovstogan et al. have introduced a user interface in which audio embeddings are reduced in dimensionality to a two-dimensional, semantically meaningful representation, allowing for exploration and rediscovery within music collections (Tovstogan et al., 2022). Similarly, Lanzend¨orfer et al. have introduced Audio Atlas, an interactive web application for the visualization of audio data using CLAP (Elizalde et al., 2022) embeddings (Lanzend¨orfer et al., 2024). Furthermore, the latent space can also be used for creative applications; in their 2024 research, Tahiro˘glu & Wyse have outlined the potential of latent spaces as a tool for computational music creativity, showing the possibilities for the exploration and subsequent sonification of regions in (con6 tinuous) latent spaces (Tahiro˘glu & Wyse, 2024). Consistent with ongoing efforts for understanding the inner workings of ML models, Zhang et al. have recently published a work on interpretability in audio embedding models; in their 2025 work, Zhang et al. aim to understand the semantics of audio data which is encoded in CLAP (Elizalde et al., 2022) embeddings (Zhang et al., 2025). Earlier works on audio interpretability have approached interpretability by highlighting features in spectrograms that significantly contribute to model decisions, exemplified by the 2019 research by Won et al., and observing the prediction output of ML models by passing large numbers of input data (Zhang et al., 2025). 2.2 Human-in-the-loop machine learning A growing amount of research in the domain of machine learning is being carried out on the concept of human-in-the-loop machine learning, a subfield which is centered around the idea of human users interactively training machine learning algorithms (Mosqueira-Rey et al., 2022). The goal of humanin-the-loop machine learning is to integrate human knowledge, emotional state and practical capability with ML algorithms, such that algorithms are trained as quickly and effectively as possible, while still achieving good classification accuracies (Joshi et al., 2012; Konyushkova et al., 2017). 2.2.1 Active learning (AL) Active learning (AL) is an approach by which annotation costs can be potentially reduced effectively (Wang et al., 2019). By means of selecting candidate data points for the iterative expansion of the training set which are most informative (Maia et al., 2024), active learning allows the training an algorithm as quickly and effectively as possible while still achieving good classification accuracies (Joshi et al., 2012; Konyushkova et al., 2017). To achieve this goal, active learning systems allow human ’oracles’ to improve the learning process by providing labels to unannotated data in real time (Saras´ua et al., 2012; Mosquiera-Rey et al., 2022). Algorithm 1 contains a general outline of an active learning loop, adapted from the 2007 research by Schein & Ungar and slightly adjusted: Algorithm 1 Active learning (AL) loop Require: Dataset consisting of labeled and unlabeled data (X,Y), human annotator 1: Initialize training set (Xtrain, Ytrain) 2: while Target classsifier accuracy not reached do 3: Calculate AL heuristic values for all unlabeled samples 4: Highlight top ksamples as candidates for annotation in accordance with heuristic (Xk) 5: Let human annotator annotate the highlighted ksamples (Yk) 6: Add annotated samples to the training set Xtrain ←Xtrain +Xk, Ytrain ←Ytrain +Yk 7: Train classifier on Xtrain, Ytrain 8: end while An important premise of the AL paradigm is that, rather than augmenting the training set of the classifier with random samples, the classifier will reach some acceptable accuracy faster by selecting the most informative samples at every training iteration i, and adding them to the training set of the classifier at the next iteration i+ 1 (Schein & Ungar, 2007). The informativeness of samples in AL is often quantified through heuristics; samples which are highlighted by a heuristic are considered to be suitable candidates for the expansion of the training set in future AL iterations (Konyushkova et al., 2017). One of the most common AL heuristics is the minimization of class prediction uncertainty in classifiers; samples which have a high class prediction uncertainty get highlighted as candidates for addition to the training set in future iterations (Konyushkova et al., 2017). This approach to AL is referred to as certainty-based AL (Shuyang et 7 al., 2017). In their 2012 research on AL for personalized music emotion recognition, Su & Fung have implemented a variation of this heuristic, in which the training set is extended with instances which have a class posterior probability closest to 0.5 (Su & Fung, 2012). Similarly, Du et al. have connected informativeness with prediction uncertainty in their 2015 paper on strategies for improving AL algorithms (Du et al., 2015). In their paper, Du et al. refer to the best-versus-second-best (BvSB) method, in which for every unannotated data point, the delta between the two highest posterior probabilities of a binary/multi-label classifier prediction is considered as a measure of uncertainty (Du et al., 2015). Thus, the smaller the delta between the two highest posterior probabilities, the higher the prediction uncertainty for a given data point can be considered to be. The BvSB measure tends to select candidate points which have the property of being close to the hyperplane which distinguishes two or more classes (Du et al., 2015). Additionally, several AL heuristics which are not certainty-based have been proposed in literature; in their 2012 research, Joshi et al. have proposed a heuristic which maximizes coverage of training points in a dataset, ensuring that all points have a nearest labeled data point within some bounded distance (Joshi et al., 2012). Moreover, data-driven approaches to AL have been proposed, in which candidate sample selection strategies are learned based on experience from previous AL outcomes; in their 2017 research, Konyushkova et al. have trained a regression model which for data points can predict the expected error reduction at a particular iteration in an active learning loop (Konyushkova et al., 2017). 2.2.2 Interactive machine learning (IML) Interactive machine learning (IML) is another approach towards human-in-the-loop machine learning (Mosquiera-Rey et al., 2022). The IML approach builds upon the principles of AL, but is a more human-centered approach; in the AL paradigm, the human annotator needs to label highlighted samples, and repeat this task for a prolonged time, as a result of which human factors such as distraction and fatigue can come into play (Mosquiera-Rey et al., 2022). In IML environments, however, the human annotator is not necessarily required to only annotate highlighted data points, and human annotators can interact with the environment beyond the annotation loop, in a freer and less structured manner (Mosqueira-Rey, 2022). 2.3 Existing work on AL/IML and audio The potential of applying AL for MIR tasks has been identified as early as 2012 by Saras´ua et al., in which they used active learning techniques for music mood classification tasks (Saras´ua et al., 2012). Saras´ua et al. have employed different selection strategies, and their findings included the importance of choosing data points such that the whole dataset space is covered (Saras´ua et al., 2012), an idea which has been also central to the coverage-based heuristic as proposed by Joshi et al. in their 2012 research (Joshi et al., 2012). In 2019, Wang et al. have developed a sound classification model to detect artifact noise in sound recordings from the Sounds of New York City (SONYC) project, an initiative for mitigating urban noise pollution in New York City (Wang et al., 2019). The aim was the identification of an sound artifacts, in which 15 human annotators were helped by certainty-based AL heuristics (Wang et al., 2019). In their 2021 research, Hilasaca et al. have proposed a visual framework for data annotation; in the visual framework, audio files are clustered based on extracted numerical features, and subsequently presented to the user in an interactive 2D space, allowing the user to explore and label soundscape ecology3data, as part of a long-term ecological research project in the Cantareira-Mantiqueira corridor in Brazil (Hilasaca et al., 2021). 3The academic study of acoustical patterns which are emanated from landscapes (Pijanowski et al. 2011) 8 Audio previewing An important addition to the UI is the implementation of audio playback on hover; by hovering over the data points in the 2D embedding space, users can playback the audio files which are part of the uploaded music collection. This can be done by providing the local folder path containing the audio files which have the same filename (without extension, e.g. .mp3/.wav) as the names stored in the id fields in the uploaded embedding file. Because the matching happens on id-strings and not on file paths, in principle, the user is free to pass a local folder path which contains alternative versions of the audio (e.g. full version, snippets, alternative mixes/masters, etc.). The audio playback on hover functionality has been proposed in earlier research, including the 2020 research by Tovstogan et al. on the the exploration of latent spaces of music collections (Tovstogan et al., 2020). 5 Methods To quantify the relevance of the user interface, as well as the relevance of the proposed heuristics for improved music classification, objective and subjective evaluations are carried out as part of this research. In the objective evaluations, the heuristics are tested by means of simulated active learning loops. In the subjective evaluations, 4 participants are asked to interact with the user interface, and to answer a number of questions related to the user interface. 5.1 Objective evaluation With the aim of quantifying the relevance of the heuristics which have been introduced in this research, an experimental setup is proposed in which active learning (AL) classification accuracies are measured for 3 different active learning strategies; expanding the training set with random samples, expanding the training set with candidates highlighted by the CPU heuristic, and expanding the training set with candidates highlighted by the DC heuristic. 5.1.1 Audio embeddings The active learning strategies are simulated for 2 different audio embeddings; •MAEST (Alonso-Jim´enez et al., 2023); •CLAP (Elizalde et al., 2022; Chen et al., 2022; Wu et al., 2024). MAEST The MAEST (Alonso-Jim´enez et al., 2023) audio embeddings have been generated with the associated Github repository6. For this audio embedding, all audio files have been converted to mono format, with a sample rate of 16 kHz. The MAEST model which has been used for embedding generation is discogs-maest-10s-pw-129e, and it requires an audio file to have a duration of at least 10 seconds. Furthermore, consistent with the findings by Alonso-Jim´enez et al. that the middle blocks of the transformer in the MAEST model feature the best representation for downstream classification tasks (Alonso-Jim´enez et al., 2023), the 6th transformer block is used. The discogs-maest-10s-pw-129e MAEST model generates an embedding vector for every 10 seconds of audio. For audio files which are longer than 10 seconds, the embedding vectors are averaged out to yield one embedding vector with a dimensionality of (768,). 6https://github.com/palonso/MAEST 15 CLAP The CLAP (Elizalde et al., 2022) audio embeddings have been generated with LAION-CLAP, a Github repository associated with 2022 research by Chen et al., and 2024 research by Wu et al.7. For this audio embedding, all audio files have been converted to a sample rate of 48 kHz. LAION-CLAP offers the possibility to choose different pretrained checkpoints, however, within the scope of this research, the default pretrained checkpoint has been used. For every audio file, the LAION-CLAP model generates embeddings with a dimensionality of (1024,). Embedding file structure The embedding vectors are encapsulated in a .json file, as outlined in section 4.3. Besides the embeddings, an ID, genre/label/mood (descriptor) and a BPM is stored for every song. The ID and descriptor are retrieved from a .csv file which contains all song IDs and descriptors, and depending on the dataset this file can be generated programatically or manually. 5.1.2 Datasets The active learning strategies are applied on 3 different classification tasks with the following datasets; •GTZAN (Tzanetakis & Cook, 2002); •Moods MIREX (Hu & Downie, 2007); •Freesound Loop Dataset (Ramires et al., 2020). GTZAN The GTZAN dataset consists of 1000 tracks with a duration of 30 seconds, and the dataset has been created for genre classification tasks. The number of unique classes in the GTZAN dataset is 10. Moods MIREX The Moods MIREX dataset consists of 269 tracks, and the dataset has been created for mood classification tasks. The number of unique classes in the Moods MIREX dataset is 5. Freesound Loop Dataset The Freesound Loop Dataset consists of 9455 loops which have been retrieved from Freesound (Font et al., 2013). The number of unique classes in the dataset is 6. Before generating the embeddings for the Freesound Loop Dataset, a few data preprocessing steps have been applied. First of all, all tracks which have more than one label have been filtered out. After this step, only 28 loops with the class “vocal” remained, thus this class has been removed from the data. Next of all, all loops which have a duration of less than 10 seconds have been filtered out. After this step, the class with the least amount of remaining loops was “bass” with a loop count of 82. Therefore, the decision has been made to create a subset of 400 loops which are randomly sampled from the pool of 842 suitable loops. The resulting subset consists of 400 samples; 78 “percussion” loops, 89 “fx” loops, 70 “bass” loops, 76 “melody” loops, and 87 “chords” loops. 7https://github.com/LAION-AI/CLAP 16 5.1.3 AL strategies Six different active learning (AL) strategies are evaluated; in each strategy, unannotated data points, represented by audio embedding vectors which have been extracted from the associated audio files, are either randomly chosen, or chosen in accordance with the class prediction uncertainty (CPU) or dataset coverage (DC) heuristic, and subsequently added to an expanding training set in order to iteratively train a multilayer perceptron (MLP) classifier. Within the scope of this research, the AL strategy with random sample selection is referred to as random baseline (RB). For each chosen data point, the associated class label is extracted from the dataset, and subsequently the embedding-label pair gets added to the training set as training data. Thus, the class labels in the objective evaluation simulations are equivalent to the class labels which are found in the evaluated dataset. The algorithm for the RB active learning strategy is as follows: Algorithm 2 Random candidate sample selection Require: Labeled embedding data (X, Y ) 1: Let the user (re)label an initial subset (Xinit, Yinit) 2: Initialize Xacc ←Xinit, Yacc ←Yinit 3: while unlabeled samples X\Xacc remain do ▷All samples in X that are not in Xacc 4: Select krandom samples ▷ k is user-defined 5: Add selected samples to (Xacc, Yacc) 6: Train classifier on (Xacc, Yacc) 7: Return label predictions for X\Xacc 8: end while 17 The algorithm for the CPU active learning strategy is as follows: Algorithm 3 Candidate selection based on minimization of class prediction uncertainty (CPU) Require: Labeled embedding data (X, Y ) 1: Let the user (re)label an initial subset (Xinit, Yinit) 2: Initialize Xacc ←Xinit, Yacc ←Yinit 3: while unlabeled samples X\Xacc remain do ▷All samples in X that are not in Xacc 4: for all xi∈X\Xacc do 5: p←predict proba(xi) 6: δi←|pmax−p2nd| p2nd ×100 ▷Class prediction uncertainty 7: end for 8: Select ksamples with the smallest δi▷ k is user-defined 9: Add selected samples to (Xacc, Yacc) 10: Train classifier on (Xacc, Yacc) 11: Return label predictions for X\Xacc 12: end while The algorithm for the DC active learning strategy is as follows: Algorithm 4 Candidate selection based on maximization of dataset coverage (DC) Require: Labeled embedding data (X, Y ) 1: Let the user (re)label an initial subset (Xinit, Yinit) 2: Initialize Xacc ←Xinit, Yacc ←Yinit 3: while unlabeled samples X\Xacc remain do ▷All samples in X that are not in Xacc 4: for all xi∈X\Xacc do 5: Compute cosine distances to each xj∈Xacc 6: di←min (cosine distance(xi, xj)) ▷Dataset coverage 7: end for 8: Select ksamples with the largest di▷ k is user-defined 9: Add selected samples to (Xacc, Yacc) 10: Train classifier on (Xacc, Yacc) 11: Return label predictions for X\Xacc 12: end while In the CPU, DC and RB strategies, the training set is randomly initialized, and it is not guaranteed that all class labels occur in the initial training set. Therefore, 3 additional AL strategies are evaluated in which the initial training set of the CPU, DC and RB strategies are initialized with stratified sampling, ensuring that every class label from the dataset occurs at least once in the initial training set. Within the scope of this research, these strategies are referred to as RB (S), CPU (S) and DC (S). Additionally, in the objective evaluation of CLAP and the Freesound Loop Dataset, a theoretical approximation of an upper bound of classification accuracy is calculated, in this research referred to as the hybrid-one-look-ahead (HOLA) algorithm. At every iteration of the objective evaluation simulation, the HOLA algorithm looks ahead one iteration, and simulates the classification accuracies which are returned if candidates are chosen in the next iteration based on the CPU or the DC heuristic; the aim of the HOLA algorithm is to maximize the classification accuracy in a greedy bestfirst approach. The accuracies returned by the HOLA algorithm represent the potential classification accuracy improvement which is feasible to achieve, under the condition that at every iteration, the ’right’ candidate samples are added to the training set. 5.2 Subjective user evaluation Within the scope of this research, subjective evaluations of the UI are carried out, both for providing insights into the user-centeredness of the current implementation of the UI, as well as for answering the proposed hypotheses in this research with more data at hand. In the subjective evaluations in this research, 4 participants interact with the UI to complete some amount of iterations for different 18 classification tasks, after which they are asked to evaluate the process and the resulting classifications by means of a Likert scale questionnaire. The music collections are chosen by the participants, and they are assisted in creating .json audio embedding files. 5.2.1 Questionnaire Range Step size Questions about the user interface Q1: Interacting with the user interface was enjoyable; [1, 7] 1 Q2: Interacting with the user interface was easy; [1, 7] 1 Q3: The buttons in the user interface have a clear purpose; [1, 7] 1 Q4: The datatables were a useful addition to the user interface; [1, 7] 1 Questions about the music embedding space Q5: Similar music pieces are close to each other in the interactive music data ’cloud’; [1, 7] 1 Questions about the classifier and heuristics Q6: Musically speaking, the classes predicted by the classifier for unannotated data make sense to me; [1,7] 1 Q7: Musically speaking, in the classifier probability distribution, the classes predicted as second and third most likely class make sense to me; [1,7] 1 Q8: The tracks highlighted by the heuristics matched your definition of difficult tracks; [1, 7] 1 Q9: How many iterations did it require to reach an acceptable classifier performance? [1,∞) 1 Table 1: Questions used for hypothesis testing When participants answer 1 (strongly disagree) or 7 (strongly agree), they are asked to provide an additional elaboration for why they chose the rating. The questions as outlined in table 1 are grouped by category, and the ordering of the questions may differ in the interview. 5.2.2 Participants Participant 1: Predicting subjective emotional depth in music Participant 1 is a male participant from The Netherlands in the age group 18-24. The participant has proposed a classification task in which the classifier attempts to recognize nuances in perceived emotional depth in 98 songs from his music collection. Along with the list of songs, he provided annotations, in which every song is labeled as being part of class ED1, ED2, ED3, ED4, and ED5, where class ED1 describes songs which are perceived to be the least emotionally deep, and class ED5 describes songs which are perceived to be the most emotionally deep. Participant 2: Predicting music preference Participant 2 is a male participant from The Netherlands in the age group 25-34. The participant has proposed a classification task in which the classifier attempts to recognize music preference, described 19 by grades as class labels. The music collection consisted of 55 songs, and a file with song names and corresponding grades was created. In the interaction with the UI, songs were labeled with 3, 4, 5, 6, 7, 8 or 9 as classes by the participant. Participant 3: Distinguishing between Slavic languages Participant 3 is a male participant from The Netherlands in the age group 18-24. The participant has chosen a classification task in which the classifier attempts to distinguish between 7 different Slavic languages. The audio collection contains excerpts from weather forecasts in Belarusian, Czech, Polish, Russian, Serbian, Slovak and Ukrainian, split into 10 audio chunks of approximately 4 or 5 seconds for each language. Participant 4: Classifying personal music collection Participant 4 is a male participant from The Netherlands in the age group 13-17. The participant has chosen a classification task in which the classifier attempts to distinguish between 5 different music labels; pop, indie, minecraft, synthwave and 1980. The music collection consists of 80 audio files from the personal collection of the participant. 6 Results 6.1 Objective evaluation results 6.1.1 Overview In this subsection, the results for the objective evaluation are outlined. For every embedding-dataset pair, 25 simulations are carried for every active learning strategy (i.e. random, CPU, DC), and the results are averaged out. The results for every embedding-dataset pair are presented by means of a table which contains classification accuracies at various iterations, and a graph which shows the progression of average classification accuracies over the course of iterations. For simulations with the GTZAN dataset, the training set is expanded with 20 data points at every iteration, for the Moods MIREX dataset the training set is iteratively expanded with 5 data points, and for the Freesound Loop Dataset the training set is iteratively expanded with 8 data points. The GTZAN training set is expanded over the course of 40 iterations, the Moods MIREX training set is expanded over the course of 44 iterations, and the Freesound Loop Dataset training set is also expanded over the course of 40 iterations. The HOLA/theoretical upper bound accuracy curve, as introduced in section 5.1.3, has been computed in the CLAP-Freesound Loop Dataset simulations, and has been mentioned in the table and visualized in the graph in section 6.1.7. [Thesis document continues on the next page] 20 6.1.2 MAEST and GTZAN In the objective evaluation for GTZAN and MAEST, the training set was expanded over the course of 40 iterations, and at each iteration, the training set was expanded with 20 data points. Algo. (↓) ; Iter. (→) 1 3 5 7 10 15 20 25 30 35 40 Non-stratified RB 0.597 0.766 0.792 0.806 0.829 0.857 0.873 0.876 0.886 0.890 0.888 DC 0.597 0.762 0.807 0.837 0.869 0.879 0.897 0.901 0.901 0.899 0.889 CPU 0.597 0.769 0.808 0.841 0.863 0.870 0.885 0.888 0.891 0.894 0.888 Stratified CPU (S) 0.614 0.766 0.808 0.838 0.861 0.873 0.879 0.884 0.891 0.899 0.890 DC (S) 0.614 0.766 0.796 0.824 0.855 0.875 0.894 0.898 0.897 0.897 0.890 RB (S) 0.614 0.763 0.794 0.805 0.836 0.854 0.872 0.876 0.888 0.894 0.894 Table 2: Classification accuracy progress for MAEST-GTZAN over the course of 40 iterations The following graph shows the progression of classification accuracies for the algorithms without stratified sampling: Figure 9: MAEST-GTZAN accuracy curve (non-stratified) And the following graph shows the progression of classification accuracies for the algorithms with stratified sampling: Figure 10: MAEST-GTZAN accuracy curve (stratified) 21 6.1.3 MAEST and Moods MIREX Algo. (↓) ; Iter. (→) 1 3 5 7 10 15 20 25 30 35 40 44 Non-stratified RB 0.236 0.241 0.256 0.273 0.273 0.273 0.275 0.294 0.313 0.310 0.319 0.319 DC 0.236 0.254 0.251 0.248 0.259 0.274 0.256 0.283 0.297 0.310 0.329 0.315 CPU 0.236 0.259 0.257 0.272 0.300 0.297 0.296 0.319 0.312 0.312 0.306 0.318 Stratified CPU (S) 0.241 0.238 0.262 0.274 0.291 0.292 0.306 0.320 0.319 0.317 0.319 0.321 DC (S) 0.241 0.245 0.245 0.232 0.263 0.263 0.277 0.295 0.298 0.316 0.325 0.318 RB (S) 0.241 0.240 0.260 0.269 0.282 0.282 0.318 0.313 0.318 0.315 0.309 0.322 Table 3: Classification accuracy progress for MAEST-Moods MIREX over the course of 44 iterations The following graph shows the progression of classification accuracies for the algorithms without stratified sampling: Figure 11: MAEST-Moods MIREX accuracy curve (non-stratified) And the following graph shows the progression of classification accuracies for the algorithms with stratified sampling: Figure 12: MAEST-Moods MIREX accuracy curve (stratified) 22 6.1.4 MAEST and Freesound Loop Dataset Algo. (↓) ; Iter. (→) 1 3 5 7 10 15 20 25 30 35 40 Non-stratified RB 0.228 0.226 0.240 0.245 0.230 0.245 0.240 0.246 0.246 0.250 0.243 DC 0.228 0.225 0.228 0.228 0.252 0.249 0.249 0.254 0.243 0.239 0.238 CPU 0.228 0.236 0.236 0.246 0.252 0.256 0.246 0.242 0.248 0.238 0.240 Stratified RB (S) 0.230 0.221 0.233 0.236 0.238 0.237 0.239 0.238 0.231 0.242 0.240 DC (S) 0.230 0.252 0.234 0.240 0.264 0.248 0.247 0.239 0.250 0.241 0.241 CPU (S) 0.230 0.243 0.246 0.240 0.244 0.241 0.242 0.243 0.242 0.236 0.242 Table 4: Classification accuracy progress for MAEST-Freesound Loop Dataset over the course of 40 iterations The following graph shows the progression of classification accuracies for the algorithms without stratified sampling: Figure 13: MAEST-Freesound Loop Dataset accuracy curve (non-stratified) And the following graph shows the progression of classification accuracies for the algorithms with stratified sampling: Figure 14: MAEST-Freesound Loop Dataset accuracy curve (stratified) 23 6.1.5 CLAP and GTZAN Algo. (↓) ; Iter. (→) 1 3 5 7 10 15 20 25 30 35 40 Non-stratified RB 0.606 0.710 0.734 0.754 0.778 0.798 0.800 0.806 0.816 0.820 0.829 DC 0.606 0.693 0.733 0.771 0.787 0.810 0.818 0.816 0.824 0.826 0.828 CPU 0.606 0.728 0.754 0.776 0.795 0.803 0.812 0.813 0.814 0.822 0.829 Stratified CPU (S) 0.628 0.726 0.752 0.765 0.783 0.796 0.802 0.811 0.815 0.821 0.829 DC (S) 0.628 0.693 0.735 0.755 0.794 0.800 0.812 0.817 0.821 0.826 0.829 RB (S) 0.628 0.690 0.727 0.747 0.763 0.780 0.798 0.808 0.820 0.820 0.829 Table 5: Classification accuracy progress for CLAP-GTZAN over the course of 40 iterations The following graph shows the progression of classification accuracies for the algorithms without stratified sampling: Figure 15: CLAP-GTZAN accuracy curve (non-stratified) And the following graph shows the progression of classification accuracies for the algorithms with stratified sampling: Figure 16: CLAP-GTZAN accuracy curve (stratified) 24 the semantic meaning of sound can be effectively communicated to the user through additional visual cues such as the video thumbnail and images retrieved from audio-to-image algorithms (Ishibashi et al., 2020). Furthermore, in the current implementation of the UI, all music embedding representations are shown as unique data points, which may lead to a cluttered visualization of data in the case of large music databases. One idea could be the aggregation of data clouds into combined ’bubbles’, which can be clicked, in order to subsequently zoom in on a segment of the data. An interesting example of this idea is already implemented in the 2025 version of geotag-based sound exploration on Freesound (Font et al., 2013). Figure 21: Freesound Map of Sounds (2025). Retrieved from https://freesound.org/browse/geotags/ Other possible features which could be added to the UI in future updates include: data point removal, allowing the user to save logs which show a list of performed actions with timestamp, saving user annotations as a ‘checkpoint’, allowing the user to choose different values for displayed problematic tracks, per heuristic, and more. More directions can be found in the “Additional contributions by the participant” subsections, in the results of the objective evaluations (section 6). 7.3 Other directions for future research N400 is a brain wave response traditionally considered to be related to language-semantic processing, in which the maximum amplitude in brain activity is reached after about 400 ms after the onset of a word stimulus which is incongruent with the preceding sequence of words (Gazzaniga et al., 2018). In 2009, Daltrozzo & Sch¨on sought to find out if the N400 effect is also elicited by musical stimuli (Daltrozzo & Sch¨on, 2009). The research by Daltrozzo & Sch¨on yielded the first evidence that musical information can also elicit N400 responses, given that a N400 brain response was observed when 1second, subjectively semantically unrelated music excerpts were played back-to-back (Koelsch, 2011). Thus, the N400 brain response, elicited by instant audio previews which are subjectively inconsistent with personal taxonomies, could possibly function as a powerful catalyst for quickly finding outliers or candidates for (re-)annotation. Although out of the scope of this research, examining the relationship between audio previewing and the N400 might be an interesting direction for future research. To my knowledge, the connection between the N400 effect and audio previewing in interactive user interfaces has neither been made or explored in other research yet. 31 8 References •Alonso-Jim´enez, P., Bogdanov, D., Pons, J., & Serra, X. (2020). TensorFlow Audio Models in Essentia. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2003.07393 •Alonso-Jim´enez, P., Serra, X., & Bogdanov, D. (2023). Efficient supervised training of audio transformers for music representation learning. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2309.16418 •Ben Sassi, I., & Ben Yahia, S. (2020). How does context influence music preferences: a userbased study of the effects of contextual information on users’ preferred music. Multimedia Systems, 27(2), 143–160. https://doi.org/10.1007/s00530-020-00717-x •Bogdanov, D. (2013). From music similarity to music recommendation: computational approaches based on audio features and metadata. http://mtg.upf.edu/node/2817 •Bogdanov, D., Lizarraga-Seijas, X., Alonso-Jim´enez, P., & Serra X. (2022). MusAV: A dataset of relative arousal-valence annotations for validation of audio models. International Society for Music Information Retrieval Conference (ISMIR 2022). http://hdl.handle.net/10230/54181 •Chen, K., Du, X., Zhu, B., Ma, Z., Berg-Kirkpatrick, T., & Dubnov, S. (2022). HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection. https://doi.org/10.48550/arXiv.2202.00874 •Daltrozzo, J., & Sch¨on, D. (2009). Is conceptual processing in music automatic? An electrophysiological approach. Brain Research, 1270, 88–94. https://doi.org/10.1016/j.brainres.2009.03.019 •Du, B., Wang, Z., Zhang, L., Zhang, L., Liu, W., Shen, J., & Tao, D. (2015). Exploring representativeness and informativeness for active learning. IEEE Transactions on Cybernetics, 47(1), 14–26. https://doi.org/10.1109/tcyb.2015.2496974 •Elizalde, B., Deshmukh, S., Al Ismail, M., & Wang, H. (2022). CLAP: Learning Audio Concepts From Natural Language Supervision. https://doi.org/10.48550/arXiv.2206.04769 •Font, F., Roma, G., & Serra, X. (2013). Freesound technical demo. MM’13. Proceedings of the 21st ACM International Conference on Multimedia; 2013 Oct 21-25; Barcelona, Spain. https://doi.org/10.1145/2502081.2502245 •Gazzaniga, M., Ivry, R. B., & Mangun, G. R. (2018). Cognitive Neuroscience: Fifth International Student Edition. W.W. Norton & Company. •Hilasaca, L. H., Ribeiro, M. C., & Minghim, R. (2021). Visual Active Learning for Labeling: A case for Soundscape Ecology data. Information, 12(7), 265. https://doi.org/10.3390/info12070265 •Hu, X. & Downie, J. S. (2007). “Exploring mood metadata: relationships with genre, artist and usage metadata”, Proceedings of the 8th International Conference on Music Information Retrieval, ISMIR’07, Vienna, Austria, 2007. •Huzaifah, M. & Wyse, L. (2021). Deep Generative Models for Musical Audio Synthesis. In: Miranda, E.R. (eds) Handbook of Artificial Intelligence for Music. Springer, Cham. https://doi.org/10.1007/978-3-030-72116-9 22 •Ishibashi, T., Nakao, Y., & Sugano, Y. (2020). Investigating audio data visualization for interactive sound recognition. IUI ’20: Proceedings of the 25th International Conference on Intelligent User Interfaces. https://doi.org/10.1145/3377325.3377483 •Joshi, A. J., Porikli, F., & Papanikolopoulos, N. (2012). Coverage optimized active learning for k - NN classifiers. 2012 IEEE International Conference on Robotics and Automation. https://doi.org/10.1109/icra.2012.6225054 32 •Koelsch, S. (2011). Toward a neural basis of music perception – a review and updated model. Frontiers in Psychology, 2. https://doi.org/10.3389/fpsyg.2011.00110 •Konyushkova, K., Sznitman, R., & Fua, P. (2017). Learning Active Learning from Data. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1703.03365 •Lanzend¨orfer, L. A., Gr¨otschla, F., Valizada, U., & Wattenhofer, R. (2024). Audio Atlas: Visualizing and Exploring Audio Datasets. Extended Abstracts for the Late-Breaking Demo Session of the 25th Int. Society for Music Information Retrieval Conf., San Francisco, United States, 2024. https://doi.org/10.48550/arxiv.2412.00591 •Maia, L. S., Rocamora, M., Biscainho, L. W. P., & Fuentes, M. (2024). Selective annotation of few data for beat tracking of Latin American music using rhythmic features. Transactions of the International Society for Music Information Retrieval, 7(1), 99–112. https://doi.org/10.5334/tismir.170 •Matrosova, K. (2024). Modeling and Influencing Music Preferences on Streaming Platforms. Computer Science [cs]. Universit´e Sorbonne Paris Nord, 2024. https://hal.science/tel-04865002v2 •McInnes, L., & Healy, J. (2018). UMAP: uniform manifold approximation and projection for dimension reduction. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1802.03426 •Mosqueira-Rey, E., Hern´andez-Pereira, E., Alonso-R´ıos, D., Bobes-Bascar´an, J., & Fern´andezLeal, ´ A. (2022). Human-in-the-loop machine learning: a state of the art. Artificial Intelligence Review, 56(4), 3005–3054. https://doi.org/10.1007/s10462-022-10246-w •North, A. C., & Hargreaves, D. J. (1996). Situational influences on reported musical preference. Psychomusicology Music Mind and Brain, 15(1–2), 30–45. https://doi.org/10.1037/h0094081 •Pijanowski, B. C., Villanueva-Rivera, L. J., Dumyahn, S. L., Farina, A., Krause, B. L., Napoletano, B. M., Gage, S. H., & Pieretti, N. (2011). Soundscape Ecology: The science of sound in the landscape. BioScience, 61(3), 203–216. https://doi.org/10.1525/bio.2011.61.3.6 •Plotly Technologies Inc. (2025). Basic Dash Callbacks |Dash for Python Documentation | Plotly. Retrieved from https://dash.plotly.com/basic-callbacks •Plotly Technologies Inc. (2025). Dash Core Components |Dash for Python Documentation | Plotly. Retrieved from https://dash.plotly.com/dash-core-components •Plotly Technologies Inc. (2025). Layout |Dash for Python Documentation |Plotly. Retrieved from https://dash.plotly.com/layout •Pons, J., & Serra, X. (2019). musicnn: Pre-trained convolutional neural networks for music audio tagging. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1909.06654 •Ragano, A., Benetos, E., & Hines, A. (2022). Learning Music Representations with wav2vec 2.0. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2210.15310 •Ramires, A., Font, F., Bogdanov, D., Smith, J. B. L., Yang, Y., Ching, J., Chen, B., Wu, Y., Wei-Han, H., & Serra, X. (2020). The Freesound Loop Dataset and Annotation Tool. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2008.11507 •Saras´ua, ´ A., Laurier, C., & Herrera, P. (2012). Support Vector Machine Active Learning for Music Mood Tagging. 9th International Symposium on Computer Music Modeling and Retrieval (CMMR). https://mtg.upf.edu/system/files/publications/saras%C3%BAa-CMMR-2012.pdf •Schein, A. I., & Ungar, L. H. (2007). Active learning for logistic regression: an evaluation. Machine Learning, 68(3), 235–265. https://doi.org/10.1007/s10994-007-5019-5 33 •Schmid, F., Koutini, K., & Widmer, G. (2023). Low-Complexity audio embedding extractors. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2303.01879 •Serra, X., Magas M., Benetos E., Chudy M., Dixon S., Flexer A., G´omez E., Gouyon F., Herrera P., Jord`a S., Paytuvi O., Peeters G., Schl¨uter J., Vinet H., & Widmer G., “Roadmap for Music Information” ReSearch, Peeters G. (editor), 2013, Creative Commons BY-NC-ND 3.0 license, ISBN: 978-2-9540351-1-6 •Shuyang, Z., Heittola, T., & Virtanen, T. (2017). Active learning for sound event classification by clustering unlabeled data. ICASSP 2017 - 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 751–755. https://doi.org/10.1109/icassp.2017.7952256 •Su, D., & Fung, P. (2012). Personalized music emotion classification via active learning. MIRUM ’12: Proceedings of the Second International ACM Workshop on Music Information Retrieval With User-centered and Multimodal Strategies, 57–62. https://doi.org/10.1145/2390848.2390864 •Tahiro˘glu, K., & Wyse, L. (2024). Latent Spaces as Platforms for Sonic Creativity. International Conference on Computational Creativity (ICCC’24). https://www.researchgate.net/publication/382947199 Latent Spaces as Platforms for Sonic Creativity •Tovstogan, P., Serra, X., & Bogdanov, D. (2020). Web Interface for Exploration of Latent and Tag Spaces in Music Auto-Tagging. Proceedings of the 37th International Conference on Machine Learning, Vienna, Austria, PMLR 108, 2020. https://repositori.upf.edu/handle/10230/45186 •Tovstogan, P., Serra, X., & Bogdanov, D. (2022). Visualization of Deep Audio Embeddings for Music Exploration and Rediscovery. Proceedings of the 19th Sound and Music Computing Conference, June 5-12th, 2022, Saint-´ Etienne (France). http://hdl.handle.net/10230/53710 •Turian, J., Shier, J., Khan, H. R., Raj, B., Schuller, B. W., Steinmetz, C. J., Malloy, C., Tzanetakis, G., Velarde, G., McNally, K., Henry, M., Pinto, N., Noufi, C., Clough, C., Herremans, D., Fonseca, E., Engel, J., Salamon, J., Esling, P., Manocha, P., Watanabe, S., Jin, Z., Bisk, Y. (2022). HEAR: Holistic Evaluation of Audio Representations. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2203.03022 •Tzanetakis, G. & Cook, P. (2002). Musical Genre Classification of Audio Signals. IEEE Transactions on Speech and Audio Processing. 10. 293 - 302. https://doi.org/10.1109/TSA.2002.800560 •Van Den Oord, A., Dieleman, S., & Schrauwen, B. (2014). Transfer learning by supervised pretraining for audio-based music classification. International Symposium/Conference on Music Information Retrieval, 29–34. https://biblio.ugent.be/publication/5973853 •Volk, A., Wiering, F., & Van Kranenburg, P. (2011). Unfolding the potential of computational musicology. International Conference on Informatics and Semiotics in Organisations, 137–144. https://pure.knaw.nl/ws/files/475092/avolk paper iciso2011.pdf •Volk, A., & Van Kranenburg, P. (2012). Melodic similarity among folk songs: An annotation study on similarity-based categorization in music. Musicae Scientiae, 16(3), 317–339. https://doi.org/10.1177/1029864912448329 •Wang, L., Luc, P., Wu, Y., Recasens, A., Smaira, L., Brock, A., Jaegle, A., Alayrac, J., Dieleman, S., Carreira, J., & Van Den Oord, A. (2021). Towards learning universal audio representations. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2111.12124 •Wang, Y., Mendez, A. E. M., Cartwright, M., & Bello, J. P. (2019). Active Learning for Efficient Audio Annotation and Classification with a Large Amount of Unlabeled Data. ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 880–884. https://doi.org/10.1109/icassp.2019.8683063 34 •Widmer, G. (2016). Getting closer to the essence of music: The Con Espressione Manifesto. ACM Transactions on Intelligent Systems and Technology, 8(2), 1–13. https://doi.org/10.1145/2899004 •Wilkie, K., Holland, S., & Mulholland, P. (2010). What Can the Language of Musicians Tell Us about Music Interaction Design? Computer Music Journal, 34(4), 34–48. https://doi.org/10.1162/comj a 00024 •Won, M., Chun, S., & Serra, X. (2019). Toward Interpretable Music Tagging with Self-Attention. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1906.04972 •Wu, Y., Chen, K., Zhang, T., Hui, Y., Nezhurina, M., Berg-Kirkpatrick, T., & Dubnov, S. (2024). Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keywordto-Caption Augmentation. https://doi.org/10.48550/arXiv.2211.06687 •Zaman, K., Sah, M., Direkoglu, C., & Unoki, M. (2023). A survey of audio classification using deep learning. IEEE Access, 11, 106620–106649. https://doi.org/10.1109/access.2023.3318015 •Zhang, A., Thomaz, E., & Lu, L. (2025). Transformation of audio embeddings into interpretable, concept-based representations. https://doi.org/10.48550/arXiv.2504.14076 35