Lietuviško balso sintezės dabartis ir perspektyvos

PIJUS KASPARAITIS, GINTARAS SKERSYS Vilniaus universitetas KEY WORDS: Lithuanian language, voice synthesis, LIEPA project. INTRODUCTION Language is the most natural and acceptable form of communication between humans and computers, so it is not surprising that a lot of effort is being put into improving computers’ ability to speak different languages. Lithuanian language is no exception. Although a quarter of a century has passed since the first attempts to “talk” a computer in Lithuanian, it can be said that the major breakthrough took place only in 2015 after the completion of the project “Lietuvių kalba valdomos paslaugos” (LIEPA). During the LIEPA project, an electronic text reader (synthesiser) that speaks in 4 voices and 7 services in which it is used was developed. For more details see the LIEPA project website (https://www.raštija.lt/liepa). Algorithms used in Lithuanian voice synthesizers, sound bases and achieved results (voice intelligibility and naturalness) are described in more detail in the article (Kasparaitis 2016), and this work will discuss new voice synthesis-based services developed following the activities of the LIEPA project and services that could appear in the near future. 1. EXISTING SYNTHESIZER APPLICATIONS The electronic text reader developed during the LIEPA project is freely distributed along with the source texts, which creates conditions for other people and organizations not related to the LIEPA project to seek new and more diverse possibilities of the synthesizer application. 1.1. The performance Remote Vilnius For the first time, LIEPA synthesizer voices Aistė and Edvardas were used even before they were finished. This happened at the international theatre festival Sirenos 2014, which was Pijus Kasparaitis, Gintaras Skerys. Present and Perspectives of Lithuanian Voice Synthesis |2 BENDRINĖ KALBA 90 (2017) www.bendrinekalba.lt ISSN 2351-7204 presented the performance Remote Vilnius by the German company Rimini Protokoll, created especially for Vilnius. There is no stage or actors in the play. The setting is specially selected Vilnius kitos miesto erdvės, dekoracijos – tai namai, automobiliai, praeiviai, o aktoriai – kartu ir žiūrovai streets or – a group of about 50 people equipped with headphones, in which the synthetic voices of Aistė or Edvardas (in the show Juozas) are heard, trying to gain the listener’s trust, interest and ultimately control this group of people. The show is like a prediction of the future of humanity, when computers will lead people. A frame from the performance is presented in Fig. 1, see the excerpts from the performance. 2014. pp. 1. (PDF) 2014. The performance received a lot of feedback in the press, the list of links can be found in the project LIEPA news (LN1 2014). The performance needed precisely mechanically and unnaturally sounding voices, so the unpolished synthesizer voices were perfectly suited for it. The soundtrack was also enhanced with various sound effects. 1 PAV. Kadras iš spektaklio Remote Vilnius (D. Matvejevo nuotr.) Pijus Kasparaitis, Gintaras Skerys. Present and prospects of Lithuanian voice synthesis |3 BENDRINĖ KALBA 90 (2017) www.bendrinekalba.lt ISSN 2351-7204 1.2. Voice-over of articles on websites Another obvious and therefore abundant group of applications is the voice-over of internet articles, messages, etc. The title of an article in the Lithuanian News (LŽ 2015) perhaps best describes the essence: From now on, “Lietuvos žinios” is not only read, but also listened to. In other words, a voice recording is published alongside the text of the article. The Lithuanian news portal (http://lzinios.lt/) and its mobile version (http://m.lzinios.lt/) started using the synthesizer for this purpose first from September 14, 2015, and from the end of October 2015 – also Ūkininko patarėjas (http://ukininkopatarejas.lt/), from February 2016 the website of the Vilnius City Municipality (http://www.vilnius.lt/) (VLN 2016), from February 2017 – the mobile version of the news portal Delfi 2017). (http://m.delfi.lt/) (DELFI LT), from February 2018 – the website of the Vilnius City Council (http://www.vilnius.lt/) (VLN 2016), from February 2019 – the mobile version of the news portal Delfi (http://m.delfi.lt/) (DELFI LT). It is very important that the synthesizer makes as few errors as kūrėjams sukelia pradinis teksto apdorojimas, t. y. skaičių, datų, santrumpų, specialiųjų ženklų possible. Most difficulty is changing the synthesizer and other non-textual information into text. Different sites may use different rules for text editing, so the text processing block of the synthesizer must be adapted to each site. Voice recordings can be placed next to articles and permanent content sites. The most important difference from the previously described application would be that the number of articles is finite or changes slowly, so it is possible to adapt the text of each article to the synthesiser or in general to prepare two versions of the text: one for the reader, the other for the synthesiser. And the synthesizer can be slightly adapted to the text, for example, supplementing the synthesizer's dictionary of abbreviations. An example of this application is the Universal Lithuanian Encyclopedia (https://www.vle.lt/). The fragment of the chapter in white (VLE 2017) perfectly illustrates how the text is presented to the reader and how it should be rearranged by the synthesizer before reading it: „...western Δ protolanguage separated about 1 t-year a. At the moment, the synthesizer reads the unprepared text, i.e. the synthesizer itself should decide which word to insert instead of the triangle, choose the decimal digit, expand the abbreviations, etc. Pijus Kasparaitis, Gintaras Skerys. Present and prospects of Lithuanian voice synthesis |4 to BENDRINĖ KALBA 90 (2017) www.bendrinekalba.lt ISSN 2351-7204 harmonize the syllables of words, although it is not yet able to do so. There is another problem in this example that is not even noticed by the reader – a hyphen is used instead of a hyphen. 1.3. Voice-over service Text-to-speech services are mainly used by the blind and visually impaired, but they can also be useful for other users, such as students, for voice-overs of lecture notes. One such service is RoboBraille, developed by Synscenter Refsnæs together with Sensus ApS from Denmark (http://robobraille.org/). Lithuanian synthetic voices created in the LIEPA project have been available in RoboBraille since November 2015 (ROBO 2015, LN2 2015). The RoboBraille service allows you to automatically convert text documents into many alternative formats suitable for people with visual or reading disabilities. One of the possible formats is an MP3 audio file. It is a self-service service that operates 24 hours a day, 7 days a week. For individual non-commercial users, the service is free of charge. The service allows you to either select one or several text files (available formats:.DOC,.DOCX,.PDF,.PPT,.PPTX,.TXT,.HTML,.HTM,.RTF,.EPUB,.MOBI, etc.), or specify the Internet address, or simply enter the desired text in the field. Then you can choose one of four Lithuanian voices and one of seven speeds. When the system prepares an MP3 audio file, it sends a link to it to the e-mail address specified by the user. 1.4. Bus station information Another application of the synthesizer: to read messages for passengers in buses, railway stations, airports, etc. UAB Vilkaviškis bus station director Algimantas Ankevičius has already started the process. Voice recordings instead of the live voice of the narrator were used here from 1980 when a clock was connected to the tape recorder, recordings were replaced by a Lithuanian voice synthesizer in 2012, and the voices of the LIEPA project have been sounding in the Vilkaviškis bus station since the second half of 2015. See the video (FLM4 2015) for more details. From 7 a.m. to 7 p.m. passengers are informed about the exact time, arrival and departure of buses and other information. On the platform only announcements are read, and in the passenger hall announcements and radio broadcasts are read. The ad must be read in a minute, Pijus Kasparaitis, Gintaras Skerys. Present and prospects of Lithuanian voice synthesis |5 BENDRINĖ KALBA 90 (2017) www.bendrinekalba.lt ISSN 2351-7204 because in a minute the radio turns on. The advertisements are read by synthesizer voice duos (alternating female and male voices for several phrases), similar to television hosts. 1.5. NAO ROBOTS 2 PAV. Robot NAO (http://old.mii.lt/index.php?siteaction=ne ws_notices.view&id=4009&lang=lt) Probably one of the most interesting applications of the LIEPA project for younger users of modern technologies is the humanoid robot NAO (Figure 2), which speaks Lithuanian and is owned by the Robotics Academy and UAB Baltic Orbis. He invited visitors to visit the Lithuanian stand during the EU OPEN DAY 2016 in Brussels, where the European Union countries were presented (LN3 2016, FLM2 2016). Projects The Lithuanian synthetic voices created by LIEPA were put into the robot and taught the robot to move by Robotics Academy engineer Linas Aidokas, the robot’s performance script was prepared by Prof. Laimutis Telksnys (VU MII), Dr. Rita Miliūnaitė (LKI) and Tautė Bernotaitė. (LKI). NAO robot also taught Lithuanian words in St. Petersburg At the European Day of Languages (FLM3 2016) held at the Hermitage Museum (Lithuania’s stand was visited by almost 3000 visitors), he recited the excerpt of Kristijonas Donelaitis’s Year in the conference “Will robots speak Lithuanian?” held at the Seimas of the Republic of Lithuania. (LN4 2016). Pijus Kasparaitis, Gintaras Skerys. Present and prospects of Lithuanian voice synthesis |6 BENDRINĖ KALBA 90 (2017) www.bendrinekalba.lt ISSN 2351-7204 2. POTENTIAL APPLICATIONS This section will present those potential applications of speech synthesis that have not yet been implemented, but the first steps have already been taken. 2.1. Synthesizer port to OS LINUX The LIEPA project synthesizer was developed for Windows environment. By adapting it to the Linux environment, its number of uses would be greater. In particular, it would be used to voice articles for online newspapers, magazines and other similar websites, as already described 1.2, but on those websites that are based on Linux. Similarly, the synthesizer would also be used to voice websites with more permanent content, such as the planned voice of the encyclopedia Wikipedia (see Chirgwin 2016). 2.2. Virtual assistants With the development of artificial intelligence, virtual assistants are being created that are able to answer questions written by humans in free form. Virtual assistants Martynas and Ieva help visitors of the website of the Migration Department of the Ministry of Interior of the Republic of Lithuania (http://www.migracija.lt/). You can ask them questions and they will answer them or provide a list of links to related pages on the site. For example, the question How much does a passport cost will be answered as follows: State fees for issuing personal documents at a fixed rate:  Passport;  Personal identity card;  provisional citizenship certificate. Exemption from the fee How to pay Atkreipiame Jūsų dėmesį, asmens dokumentus išduoda teritorinių policijos įstaigų migracijos units. Pijus Kasparaitis, Gintaras Skerys. Present and prospects of Lithuanian voice synthesis |7 The BENDRINĖ KALBA 90 (2017) www.bendrinekalba.lt ISSN 2351-7204 page State fees list is still being opened. The same answer is given to the question How much does an identity card cost, i.e. the answers are prepared in advance and somewhat more general than the questions. Incorrect questions are answered by the assistant with the following type of response: Please clarify the question; I will try to help you if you ask short questions; I will try to answer your question according to the competence of the Migration Department. One of the shortcomings of virtual assistants is that their answers are not voiced in Lithuanian. 3 PAV. Virtual assistants Ieva and Martynas (http://www.migracija.lt/) constantly shouting: "Hot chebureks! Toast! ...“ A robot could also Pijus Kasparaitis, Gintaras Skerys. Present and prospects of Lithuanian voice synthesis |8 BENDRINĖ KALBA 90 (2017) www.bendrinekalba.lt ISSN 2351-7204 2.3. MIO MOBILE KIOSK ROBOTS Daugelis poilsiautojų Palangos pajūryje yra matę šaltkrepšiais nešinus jaunuolius, do this work. One of the Lithuanian startups (http://miokiosk.com/) is currently developing an autonomous mobile vending machine named MIO mobile kiosk (Figure 4) (Fuks 2016). It is expected that it will be able to carry and sell snacks and drinks in parks, squares, squares, as well as during various events. It is planned that this robot will be called through a mobile app, and the touch screen will be used to communicate with the buyer. For the convenience of users, the robot could also speak Lithuanian: invite buyers, advertise its goods, provide instructions for use, etc. 4 PAV. MIO mobile kiosk (http://miokiosk.com/) 2.4. Automatic translation software Already several decades ago, the first computer programs capable of translating text from one language to another appeared. Some of them can read the voice translation or already Pijus Kasparaitis, Gintaras Skerys. Present and prospects of Lithuanian voice synthesis |9 BENDRINĖ KALBA 90 (2017) www.bendrinekalba.lt ISSN 2351-7204 translated text. For example, one of the most popular programs Google Translate (http://translate.google.lt/) can read text written in the languages of neighboring countries: Polish, Russian, Latvian. There are at least 4 known translation programs that translate into or from Lithuanian: the already mentioned Google translator, Microsoft translator (http://www.bing.com/translator/), Vytautas Didysis University translator (http://vertimas.vdu.lt/twsas/), Tildė IT and Lithuanian Language Institute translator (https://translate.tilde.com/en/). Unfortunately, this translation program is not able to read Lithuanian text. 2.5. Adapting smartphones for the blind In recent years, many tasks that people were used to doing on computers, such as browsing the web or writing emails, are now done on smartphones. Blind and visually impaired people are no exception, and they are also increasingly using smartphones. As with computers, the answer for the blind is given by voice. To this end, smartphones have built-in voice synthesizers and screen readers for the blind linked to them via a standardised interface. The most popular smartphone operating systems and built-in readers are the following: the open-source Android operating system and TalkBack reader, Apple's iOS and VoiceOver, Microsoft's Windows Mobile and Narrator. Unfortunately, synthesizers in their devices are specially restricted, so at the moment it is possible to speak in nei Apple, nei Microsoft nėra sukūrusios lietuviško sintezatoriaus, o galimybės įdėti savo sukurtą Lithuanian only on Android smartphones. This would involve porting the synthesizer software to Android, creating a standard interface and allowing blind people to use the smartphone with the help of the built-in screen reader TalkBack. Several obstacles associated with the transfer of LIEPA synthesizer to Android system:  Smartphones usually have less memory than computers, so it would be necessary to reduce the sound bases, which may lead to a deterioration of the voice quality. However, in smartphones, a lower signal sampling frequency may be sufficient, which would allow to reduce sound bases by reducing the frequency alone;  In Android, the interface is usually programmed in the Java programming language, which belongs to interpreted languages, and the synthesizer is written in C, which is