Monster Voice: Split Tongue

Monster Voice: Split Tongue YOUNGJOO JENNIFER RYU, Korea National University of Arts, Republic of Korea Additional Key Words and Phrases: sound language, wearable instrument, chronic pain disease, pain communication ACM Reference Format: 1 PROGRAM NOTES Monster Voice: Split Tongue is a live recitation composition of a spoken language dedicated to female pain, one that may become a new method of communication for women who hurt. A new sound language based on phonetic symbols, designed using SuperCollider, is read aloud on the custom-built body-worn Arduino instruments. The pain of a hurting woman is difficult to convey to others. Instead of language, the content of pain has been vocalized through whimpers, cries, screams, or silence. Following in the footsteps of generations of women who have attempted to verbalize their pain, the language dismantles the existing [consonant+vowel] language system. Instead, it incorporates the structure of [sound+vowel] to interpret the writings of women who came before us. At the same time, the language is an attempt to vocally communicate the chronic pain of the composer/performer herself, who suffers from fibromyalgia. The process of physically translating the new language, Monster Voice, through the custom-built body-worn instruments revive and give voice to the pain of the composer/performer and the women of previous generations. 2 PROJECT DESCRIPTION (1) Background and Overall Concept Monster Voice: Split Tongue is a project conceived by an artist with chronic pain syndrome who struggled to communicate her pain to others through conventional language. The artist uses body-worn instruments to experiment with a new sound language to communicate and utilize in speech. The artist has fibromyalgia, a chronic disorder with an unknown cause, and her pain is constant and repeating - thus, her unexplainable pain is ironically both cause and symptom. Pain is impossible to share not because of the difference in language between the enunciator and audience but because of the nonexistence of a possible explanatory/symbolic language. Being stripped of language-as-explanation, to speak but not be heard. As a woman in pain, the artist is exiled twice over, once from the power of language and once from the right to speak up against her expulsion[3]. When the artist explores her wish for a common language that can communicate pain, this language does not yet exist and is a communicative method that must be newly invented. Thus, the translation process is not simply substituting text with sound but formulating an entirely new communication system. What was Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Copyright remains with the author(s). Youngjoo Jennifer Ryu. 2024. Monster Voice: Split Tongue. 1, 1 (September 2024), 7 pages. Music Proceedings of the International Conference on New Interfaces for Musical Expression NIME’24, 4–6 September, 2024, Utrecht, The Netherlands 2 Youngjoo Jennifer Ryu required, in short, is not semantic translation but the transfer of the medium itself, and it is done with wearable instruments acting as tactile organs. (2) Sound Language Development: Monster Voice (a) Concept Monster Voice, the new sound language, was created by deconstructing the existing English language system. It was made based on the IPA(The International Phonetic Alphabet) diacritical system to reference how English sounds when pronounced. The conversion from the preexisting language to the new language involves the following process in Figure 1. Fig. 1. Translation of text to new sound language Monster Voice (b) Vowel: Formant Synthesis In the sounds of someone in pain, vowels make up most of their speech, like the concept of Anne Carson’s Ololyga[ 1 ]. (i.e., ouch, oh, ahh) In Monster Voice, vowels play the same role as the conventional English system. The vowel sounds were created in SuperCollider using formant synthesis. Praat, a sound analysis program for linguistics, was used for formant frequencies and bandwidths data for every IPA vowel. (c) Consonant: New Sounds SynthDefs for each IPA consonant phonetic symbol were created on SuperCollider for replacement. The focus was to create a sound that the image of pain or the process of failed communication incorporated into the synthesis, not to mimic existing sounds. Different parameters were used for consonant SynthDefs to make three types of synths for each consonant symbol. These synth types were distributed by the location of the consonant in each word, beginning, middle, and end, to create different sounds in the composition. (3) Wearable devices: Physically Grafted Instrument (a) Concept In Monster Voice: Split Tongue, three body-worn instrument modules were used. The criteria for creating the instruments were to relate to the body as a tangible material and stay consistent with the artist’s gestures in everyday life while playing. The instrument consists of a necklace-like sensor that detects the opening and closing of the mouth, a tension sensor that covers the face and recognizes pulling or flicking by the hand, and a pressure sensor that responds to the pressing of painful areas and acts as an augmented mechanism of conversion for both verbal and non-verbal expressions. The instruments developed in this project should be understood as organs. It may be Monster Voice: Split Tongue 3 described as a wearable device but is more accurately a physically grafted instrument. The act of speech is the active use of this instrument as a replacement for discarded vocal organs, for the signification of sounds that exist outside the realm of indicative signals. (b) Instrument I: Speech Controller Fig. 2. Instrument I: speech controller (i) Description The first instrument module controls the speech itself. It is a necklace with a 3D-printed chin rest supported by the upper chest area. A slide potentiometer functions as an analog trigger input. An extension spring moves a slider when the performer opens their mouth, switching on/off above/below a specific value to detect the act of speaking. Fig. 3. Instrument I: speech control example (ii) Mechanism The sound starts when the mouth is opened and ends when the mouth is closed in a mechanism similar to actual speech. Every speech unit is switched on and off around a specific accent vowel the artist designated in a composition sequence unit. All vowel SynthDefs have the Env.asr gate that starts and stops the sound, controlling the length of the accented vowel. Since the speed of speech is affected by the lengths of the syllables’ vowels, the performer can control the speed of the composition. The entire composition and sequence unit duration depend on how long the performer opens their mouth, making it an on-site composition factor. 4 Youngjoo Jennifer Ryu Fig. 4. Instrument II: vowel effector (c) Instrument II: Vowel Effector The second instrument module intuitively visualizes the sound effects and tactile pain sensations during the performance. A string-attached mask-shaped effector only applies to vowel Synths. Four strands of conductive rubber cord stretch sensors are attached to the face side of the mask and interact with the performer’s hands and facial muscle movements. A filterbank SynthDef created in SuperCollider receives the real-time value of the rubber cords and uses it as the filter frequency. The vowel filter bank creates the intonation of Monster Voice speech. The performer makes various gestures with both hands, including stretching, bouncing, or making multiple elastic bands touch each other. Also, the sensor value of the strings changes according to the performer’s opening/closing and lipsyncing. (d) Instrument III: Speech Interrupter Fig. 5. Instrument III: speech interrupter The pain area module creates the interruption that only applies to consonant Synths by interacting with hand pressure and body posture. This speech interference module comprises a palm-sized pressure-sensitive conductive sheet worn as a belt over the performer’s pain-causing area. As pressure is applied, the interrupter effector created in SuperCollider receives the real-time value of the pressure sensor and proportionally lowers the amplitude of the current playing consonant Synth nodes. The effector also uses the same value to increase the amplitude of a high-pitched sine wave sound, replacing the consonant Synth sounds. The performance Monster Voice: Split Tongue 5 requires the performer to remain in a fixed posture while performing, which causes pain to the artist/performer due to their condition. The performer presses/rubes this interrupter module or does the stretch posture. The usage frequency of this module increases as it reaches the end of the composition. (4) Composition of the Performance: Text Matrix Score (a) Process During the performance, the performer learns the language of sound by weaving together fragments of the writings of ancestral sick women like Theresa Hak Kyung Cha[ 2 ], Nella Larsen[ 4 ], Adrienne Rich[ 7 ], and Clarice Lispector[ 5 ][ 6 ] in the first part with the incomplete version of Monster Voice. In the latter part, the performer reads a poem written based on the interview of the artist done by a text editor, Donghwi Kim, in the complete version of Monster Voice. Fig. 6. Score as a text matrix (b) Content The two-part composition begins with the reinterpretation of a common language for women through the perusal of pieces written by women across many continents and generations. Consonants and vowels are dismantled and replaced by non-phonetic sounds and formant compound vowels, and the script is formed by expanding from phonemes to syllables, words, and sentences. As each step repeats throughout the composition, the sounds flow gradually, mirroring learning a new language. The composition’s second half is a monologue of the artist’s narrative and utilizes the newly familiarized linguistic system, Monster Voice. (c) Underlying Technologies The text matrix acts as a score since the composition goes through ’text-to-sound language’ conversion, as in Figure 1. For the first part, the artist created a SuperCollider sequence manually. For the second part, based on the sequence generation rules formed during the discussion with the Python developer, the SuperCollider sequence was developed. The artist then listened to the result and fine-tuned it manually to make it more musical. 6 Youngjoo Jennifer Ryu 3 PERFORMANCE NOTES If NIME 2024 is hosted in person: • The composer will bring their laptop, their instruments, and one mini monitor(as a score reader) equipped with a clamp. The composer could bring an audio interface if needed. • Requirements for the performance: four loudspeakers, one monitor speaker on stage, an audio interface with four-channel outputs on stage, a projector stand or microphone stand for monitor clamp installation, one table on stage, and one power strip with a minimum of three outlets on stage. •Performer: The composer will perform by themself. • Venue: Tivoli Vredenburg or HKU. (Hybrid performance is also possible. However, the four-channel audio should be mixed down into stereo audio) • (HKU possibilities are not sure because the software part of this project code is in SuperCollider code, so the artist should check the code suitability for WFS.) • Since the sound movements of this piece are for the surround speaker system, the venue, capable of providing four-channel outputs, suits this project. This performance has similar features to a book recitation performance that needs a relatively quiet and concentrated atmosphere. • Feasibility: The composer has had their related compositions performed at high-profile computer music festivals and has multiple experiences working as a sound engineer at computer music festivals. Fig. 7. Technical floor plan of Monster Voice: Split Tongue 4 MEDIA LINKS •Video: https://drive.google.com/file/d/1iprgELAFUpBCNWnkGRIVlVPAeqMacUIu/view?usp=sharing •Audio: https://drive.google.com/file/d/1mPeCBrekA-ROAmqCQlaMcEZENBYRHk3U/view?usp=drive_link ACKNOWLEDGMENTS This work was supported by Seoul and Seoul Foundation for Arts and Culture with project grant at 2023. The authors would like to thank technical supervisor Joosun Hwang and Loksu(Alex Griffiths), text editor Donghwi Kim, developer Jinsu Eun, and translator Soyoung Chong. Monster Voice: Split Tongue 7 ETHICAL STANDARDS Monster Voice: Split Tongue aims to share the un-communicable features of pain to connect people and raise awareness of chronic disease. It was composed and performed in Seoul, Republic of Korea. The artist does not believe there are any potential conflicts of interest, whether financial or non-financial. All research and project participants gave full consent to participate. No animals were involved in this research. The artist has often created varying performance versions as a free public concert. All electronic elements were created using SuperCollider, Arduino, and TouchDesigner. The previous two programs are open-source software, and the TouchDesigner was used only for receiving the Arduino controller’s serial signal and changing it into an OSC signal. REFERENCES [1] Anne Carson. 1995. Glass, Irony, and God. Vol. 808. New Directions Publishing, New York. [2] Theresa Hak Kyung Cha. 2001. Dictee. University of California Press, California. [3] Johanna Hedva. 2016. Sick woman theory. Mask Magazine 24 (2016), 1–15. [4] Nella Larsen. 1986. Quicksand and passing. Rutgers University Press, New Jersey. [5] Clarice Lispector. 1989. The Stream of Life. University of Minnesota Press, Minnesota. [6] Clarice Lispector. 1992. The Hour of the Star. New Directions Publishing, New York. [7] Adrienne Rich. 1978. Origins and history of consciousness. College English 39, 6 (1978), 690–691.