Toward Musicologically-Informed Retrieval: Enhancing MEI with Computational Metadata

Toward Musicologically-Informed Retrieval: Enhancing MEI with Computational Metadata Nádia Carvalho1,2[0000000168825138] and Gilberto Bernardes1,2[0000000338842687] INESC TEC, Faculty of Engineering, University of Porto, Porto, Portugal {up201208223,gba}@fe.up.pt Abstract. We present a metadata enrichment framework for Music Encoding Initiative (MEI) files, featuring midto higher-level multimodal features to support content-driven (similarity) retrieval with semantic awareness across large collections. While traditional metadata captures basic bibliographic and structural elements, it often lacks the depth required for advanced retrieval tasks that rely on musical phrases, form, key or mode, idiosyncratic patterns, and textual topics. To address this, we propose a system that fosters the computational analysis and edition of MEI encodings at scale. Inserting extended metadata derived from computational analysis and heuristic rules lays the groundwork for more nuanced retrieval tools. A batch environment and a lightweight JavaScript web-based application propose a complementary workflow by offering large-scale annotations and an interactive environment for reviewing, validating, and refining MEI files’ metadata. Development is informed by user-centered methodologies, including consultations with music editors and digital musicologists, and has been co-designed in the context of orally transmitted folk music traditions, ensuring that both the batch processes and interactive tools align with scholarly and domainspecific needs. Keywords: Music Encoding Initiative (MEI) ·Music Segmentation · Key and Mode Detection ·Web Application. 1Introduction As digital music archives expand, richer metadata becomes crucial for advanced applications like scholarly analysis and semantic search. The Music Encoding Initiative (MEI) framework [11] provides a flexible structure for representing musical documents, but existing metadata tools, such as MerMEId [15], often All rights remain with the authors under the Creative Commons Attribution 4.0 International License (CC BY 4.0). Proc. of the 17th Int. Symposium on Computer Music Multidisciplinary Research, London, United Kingdom, 2025 Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 221 N. Carvalho et G. Bernardes remain limited to low-level or bibliographic descriptions [10]. These fail to capture midor high-level semantics, such as phrase structure, modality, and thematic elements, which are essential for content-aware retrieval and musicological research. Recent research projects like Neuma [1], SIMSSA [8], and the Music Encoding and Linked Data framework [21,16] have introduced tools for semantic segmentation and high-level feature extraction in symbolic music. While effective, most focus on isolated problems and offer limited integration with editorial workflows. Especially in folk music traditions, where melodic variation and oral transmission challenge standard theoretical models, structured analytical metadata is essential for enabling content-based retrieval [20]. Our work extends MEI by embedding computationally derived metadata that captures midto high-level multimodal musical features. The framework supports thematic search, corpus comparison, and scholarly annotation by integrating symbolic analysis with domain expertise. It was co-designed with musicologists working on orally transmitted folk music and supports multiple data types, including symbolic scores, lyrics, and bibliographic data. A scalable batch processing pipeline applies rule-based analysis, while a browser-based app enables human-in-the-loop validation. Both tools were developed using user-centered methods to ensure usability and impact. The paper is structured as follows: Section 2 outlines the framework and its integration into scholarly workflows. Section 3 details the metadata enrichment methods. Sections 4 and 5 present the batch processing environment and the interactive tool MeiScribe.Section6showcasesapracticalusecase,andSection7 concludes with key findings and future directions. 2 Metadata Framework for MEI Files Our framework enhances MEI files with semantically rich metadata, bridging symbolic music encodings with higher-level analytical descriptors. Designed with input from the EA-Digifolkproject, the system addresses the needs of educators, archivists, and musicologists working with orally transmitted music. It integrates three main components: – Metadata Schema: Defines extended MEI-compatible structures, covering Title, Publisher, Source, Work Info, Ambitus, Pattern Metadata, and Phrase Segmentation. This ensures interoperability and alignment with scholarly use cases. – Batch Processing Environment: Applies computational methods and rule-based heuristics to extract midand high-level features at scale. Enriched metadata is injected into MEI or exported in linked formats. – Interactive Web Tool: Offers an intuitive browser interface for validating and refining annotations. Users can upload scores in formats like MusicXML and ABC, which are converted to MEI for analysis. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 222 Toward Musicologically-Informed Retrieval This modular architecture balances automation with expert oversight. By grounding the schema in MEI standards—using custom namespace elements and following MEI’s extensibility model while ensuring schema validity—and separating batch from manual tasks, the framework enables scalable and precise metadata enhancement. Figure 1 illustrates the architecture and component interactions. Fig.1: Architecture of the metadata enrichment framework showing three core components: Metadata Schema, Batch Processing, and Interactive Web Tool. The MEI schema includes five modules—Title, Publisher, Source, Work Info, and Score Info—for interoperable metadata. Batch mode handles automatic extraction/injection; the web tool supports validation and refinement. Underlined fields are batch-editable; bold fields can be auto-detected. 3 Metadata Enrichment Through Computational Analysis Techniques We apply a modular pipeline to enrich MEI files with midto high-level musical features derived from computational analysis and heuristics. These include ambitus, tonality and modality, tempo and meter, structural patterns, phrase structure, and textual topics—supporting deeper musicological insight and more effective retrieval across large collections. 3.1 Ambitus We compute pitch range (ambitus) by extracting MIDI pitch values from MEI files using the Verovio Toolkit [5]. The minimum and maximum notes define the ambitus, which is encoded in <scoreDef>using the <ambitus>element. An @automatic flag indicates whether detection was computational. An example of the ambitus annotation in MEI is depicted in Listing 1.1. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 223 N. Carvalho et G. Bernardes <scoreDef meter.count="6" meter.unit="8" key.sig="0"> <staffGrp> <staffDef n="1" xml:id="P1" label="Voice" lines="5" clef. line="2" clef.shape="G" key.sig="0"/> </staffGrp> <ambitus automatic> <ambNote type="lowest" oct="4" pname="g"/> <ambNote type="highest" oct="5" pname="d"/> </ambitus> </scoreDef> Listing 1.1: Example of Ambitus Annotations in MEI The <ambitus>tag contains two <ambNote> elements: one indicating the lowest note in the score (type="lowest"), and the other indicating the highest note (type="highest"). In this example, the pitch range spans from G4 to D5. By placing <ambitus>inside <scoreDef>,theambitusistreatedasglobal metadata for the score or for the specified staff. 3.2 Tonal and Modal Frameworks Key and mode define the tonal and modal framework of a piece, offering insight into its scalar and harmonic structure (e.g., major, minor, dorian, phrygian). This metadata is especially relevant for repertoires beyond common-practice tonality, supporting tasks like tonal comparison, modal classification, and harmonic analysis. It also improves search and recommendation by enabling filtering based on tonal traits. In our framework, tonal and modal information are estimated by matching pitch-class distributions to rotating templates for major, minor, and modal scales (i.e., ionian, dorian, phrygian, lydian, mixolydian, aeolian, and locrian) [2]. The best-matching template defines the key and mode, with confidence scores derived from Pearson correlation [17], based on [18]. MEI stores this in <key>with @mode,@automatic,and@confidence attributes. The example in Listing 1.2 illustrates a typical metadata block in our MEI enrichment framework. In this case, the key is G dorian, and the annotation was generated automatically with 90% confidence, as indicated by the @automatic flag. <work n="1" xml:id="GM001"> <title type="main">No title</title> [...] <key mode="dorian" automatic confidence="90%">G</key> <meter automatic>Binary</meter> <tempo type="indicative" automatic>Allegro</tempo> [...] </work> Listing 1.2: Example of Key, Mode, Meter and Tempo Annotations in MEI Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 224 Toward Musicologically-Informed Retrieval 3.3 Tempo and Meter Meter and tempo are essential temporal features that shape the rhythmic structure and expressive character of a musical work. Detecting and encoding meter provides information about the underlying beat organization, such as duple, triple, or compound meters, while tempo annotations convey the pace at which the music unfolds. Including this data in metadata supports analytical tasks like rhythmic pattern analysis, genre classification, and performance interpretation. Meter is inferred from MEI time signatures and rhythmic patterns (e.g., binary, ternary, polyrhythmic, free). Tempo is estimated from note density and mapped to qualitative (e.g., Allegro) or numeric (e.g., BPM indications) labels. Annotations are added via <meter>and <tempo>with @automatic tags for downstream refinement. As shown in Listing 1.2, the example includes a binary meter and an indicative tempo marked as Allegro.Bothannotationswere generated automatically, as indicated by the @automatic flag. 3.4 Structural Patterns Pattern analysis plays a central role in understanding the internal structure of musical works, especially within oral and traditional repertoires, where repetition, variation, and transformation of small musical units are fundamental [13]. In our framework, we focus on three primary types of motivic features: pitch, intervallic, and rhythmic patterns. To represent motivic content, we use histogram-based descriptors of surface elements [19]. Pitch content is encoded as a 12-bin pitch class vector [18], while intervallic structure is captured in a 24-bin vector spanning the [12,12] range, reflecting interval frequency and directionality [7]. Rhythmic structure is modeled as a variable-length vector based on tactus-level segmentation [12], highlighting idiomatic accents and filtering out surface-level ornamentation—e.g., a flamenco bulería reveals characteristic stress patterns at beats 3, 6, 8, 10, and 12. These compact representations support pattern comparison and classification. They’re stored in MEI under <supplied>elements with @type labels (e.g., @type="pitch pattern"). Only summary vectors are kept in MEI, not listings of note-by-note patterns, as observed in Listing 1.3. <section> <supplied type="pitch pattern" automatic> <histogram total="26" units="count" normalized="false" pc_0="7" pc_1="0" pc_2="4" pc_3="0" pc_4="4" pc_5="3" pc_6="0" pc_7="5" pc_8="0" pc_9="2" pc_10="0" pc_11="1"/> <supplied type="interval pattern" automatic> <histogram intm_-12="0" intm_-11="0" intm_-10="0" intm_-9="0" intm_-8="0" intm_-7="1" intm_-6="0" intm_-5="1" intm_-4="1" intm_-3="1" intm_-2="3" intm_-1="1" intm_0="0" intm_1="1" intm_2="5" Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 225 N. Carvalho et G. Bernardes intm_3="2" intm_4="3" intm_5="1" intm_6="0" intm_7="3" intm_8="0" intm_9="0" intm_10="0" intm_11="0" intm_12="0"/> </supplied> <supplied type="rhythm pattern" automatic> <histogram optimal_resolution="16th" bin_00="10.0" bin_01="2.5" bin_02="8.0" bin_03="2.5" bin_04="7.0" bin_05="2.5" bin_06="8.0" bin_07="2.5" bin_08="10.0" bin_09="2.5" bin_10="8.0" bin_11="2.5" bin_12="7.0" bin_13="2.5" bin_14="8.0" bin_15="2.5"/> </supplied> [...] </section> Listing 1.3: Example of Structural Pattern Annotations in MEI 3.5 Phrase Structure and Formal Boundaries Including phrase structure and formal boundaries in metadata enriches digital music collections by providing insight into a piece’s organization and form. This supports musicological analysis, thematic indexing, and structure-based search, enabling users to identify patterns like A-B-A forms or cadential phrases across works. Such metadata bridges the gap between symbolic encoding and musical interpretation, enhancing both scholarly and practical engagement with digital scores. Phrase segmentation is computed using the Local Boundary Detection Model (LBDM) [3], which identifies local discontinuities based on absolute pitch intervals, inter-onset intervals (e.g., time between note onsets), and rest durations. These factors are normalized, weighted, and combined into a boundary strength score. Peaks in this profile indicate likely phrase boundaries. In our framework, detected boundaries are linked to MEI note identifiers and structured into phrases. These are then compared using the SIAM algorithm [14], which effectively identifies melodic pattern similarities in folk music [9,4]. Phrases are annotated using <phrase>elements with start and end note IDs and semantic labels, grouped within a <supplied type="phrases"> tag. As shown in Listing 1.4, each phrase is assigned a formal label (e.g., “A”, “A2”, “Intro”, “Bridge”) reflecting its structural role, following standard music-analytical conventions for identifying coherent musical units. <supplied type="phrases" automatic> <phrase n="1" startid="#n1bahmqv" endid="#n7kwtt8" type=" Intro"/> <phrase n="2" startid="#n9jqa4o" endid="#n3qsh5e" type="A"/> <phrase n="3" startid="#nbqrooi" endid="#n1bjptp9" type="B"/> <phrase n="4" startid="#n1tjsewn" endid="#nntgr78" type=" Bridge"/> <phrase n="5" startid="#n1iv1k4c" endid="#nvb2i9c" type="A2"/ > Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 226 Toward Musicologically-Informed Retrieval <phrase n="6" startid="#n1q50odr" endid="#nnlv1ke" type="B2"/ > </supplied> Listing 1.4: Example of Phrase Annotations in MEI 3.6 Textual Topics In traditional and oral music, lyrics play a key role in conveying meaning and context. Annotating textual topics helps situate works within their cultural and historical settings while supporting analysis and retrieval. We provide three automated annotations: preprocessed clean lyrics, most frequent unigrams and bigrams, and high-level textual topics derived from thematic clustering. Following the framework in [6], lyrics are tokenized, lemmatized, and cleaned using language-specific stopwords to retain semantic content. From this, we extract: 1) normalized clean lyrics, 2) frequent unigrams and bigrams, and 3) highlevel topics via BERT-based clustering. Bigram analysis captures both individual themes and their contextual relationships within the musical tradition. Textual topics are encoded in the <keywords>element within <work>,using three <term>fields: clean lyrics (preprocessed text), most frequent ngram (unigram and bigram), and high-level topics from semantic clustering (Listing 1.5). Each <term>includes a @type attribute, and topics are separated by semicolons. This compact structure captures lexical and thematic dimensions for retrieval. For example, the Spanish lullaby “Duérmete, niño. . . ” is reduced to “duérmete niño venir coco llevar,” preserving key terms like “duérmete,” which clearly signal its context as a lullaby. <workList> <work xml:id="GM002"> [...] <incip type="lyrics"> <incipText> Duérmete, niño Duérmete ya Que viene el coco Y te llevará </incipText> </incip> <keywords> <term type="clean-lyrics">duérmete niño venir coco llevar</ term> <term type="ngram">duérmete</term> <term type="bigram">venir coco</term> <term type="textual-topic">Lullaby</term> </keywords> [...] </work> Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 227 N. Carvalho et G. Bernardes </workList> Listing 1.5: Example of Textual Topic Annotations in MEI 4BatchProcessingEnvironment In support of large-scale MEI metadata enrichment, we developed a batch processing environment designed for efficient, scalable analysis and annotation of digital music corpora. This component of the framework automates the extraction of midto high-level musical metadata, serving as a foundational layer for subsequent interactive refinement and semantic Detection. The batch system automates MEI enrichment through a modular processing pipeline. It parses MEI input, transforms content into computation-friendly formats, and extracts metadata using the analytical methods from Sections 3. Metadata is injected back into MEI using custom tags within MEI’s extensionfriendly schema. Consistency is preserved through template alignment. To balance automation with editorial control, we distinguish between: Global fields (e.g., publisher, and source info) – batch editable, and Work-specific fields (e.g., ID, title, lyrics, etc ) – refined individually. This distinction informs the interface design, allowing safe, large-scale annotation without overwriting unique content. Batch-editable fields are marked in Figure 1, guiding efficient edits while preserving work-specific data. Fields defined in 3 can be batch computed. By separating shared and unique data, the system ensures integrity and supports scalable annotation, especially for institutional or regional collections with shared contexts and individual editorial needs. 5InteractiveWeb-BasedTool To complement single and batch processing, we developed MeiScribe,1,a lightweight, browser-based tool for reviewing and editing enriched MEI metadata. Fully JavaScript-based and installation-free, it supports standalone use and integration with digital platforms. MeiScribe balances automation with expert oversight. Users can inspect and refine metadata through an intuitive interface without MEI expertise, and perform or rerun automatic analyses on individual or batch files (Section 3). It supports format conversion (MusicXML, ABC, MEI), metadata overlay and editing, batch updates, and score-aligned visualizations of metadata such as phrase boundaries and rhythm patterns. These features support workflows ranging from individual file review to corpus-level curation. 1MeiScribe is available at https://ea-digifolk.github.io/MeiScribe/ (last accessed June 28, 2025). Demonstration videos can be found at https://drive.google.com/drive/ folders/1OxqFZE8HhL_ntS66NUPLVUHpeHK6owPU?usp=sharing. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 228 Toward Musicologically-Informed Retrieval 5.1 Converter from MusicXML and ABC to MEI format MeiScribe converts MusicXML and ABC into MEI using the Verovio Toolkit [5], while MEI files load directly. Unsupported formats (e.g., MIDI) are flagged. Files can be imported via local storage, URLs, or plain-text ABC, and exported individually or as zipped batches. 5.2 Metadata Overlay and Editing Interface A carousel-style interface displays and allows editing of fields from <titleStmt >,<pubStmt>,<source>,and<workList>,alongsidefeatureslikeambitusand phrase structure (Figure 2). Fields are auto-extracted using XPath and updated live to ensure MEI validity. Fig.2: Phrase Structure Interface for Processing Individual MEI Files. Note IDs can be viewed in the score by hovering over each individual note. 5.3 Batch Processing Interface The Batch Interface enables efficient metadata editing across multiple files. Users can activate specific camps (e.g., titleStmt,workList)toapplytargetededits (Figure 3a). Automated analyses (e.g., key, mode, textual topics) are available, with complex features—like ambitus and phrase segmentation—handled Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 229