Today, an increasing number of texts are translated by computers. This kind of automatic computer translation is most often called machine translation (MT). Machine translation is generally performed to make sense as quickly as possible of the vast stream of information in many languages, and also to disseminate one’s own information as widely as possible. The exchange of electronic information between nations and languages is becoming an important part of everyday life, opening up new opportunities for our language and posing new challenges and problems for it.
The term “machine translation” (English machine translation) was first used in the middle of the last century by Warren Weaver, who proposed adapting the newly emerging computers for translating texts (Weaver 1949: 1). W. Weaver suggested viewing a text written in another language as a cipher that could be decoded. The cryptographic achievements of British and American specialists in the recently ended Second World War inspired optimism. At the time, there was a sincere belief that electronic machines would help overcome the curse of the Tower of Babel within a few years.
Unfortunately, translation progressed slowly, and the quality of machine translation was nowhere near that of human translation. Seventy years later, after undeniable successes and disappointments and much work accomplished, optimism is once again prevailing in the machine-translation community. This time, hopes are pinned on artificial neural networks and rapid progress in developing artificial intelligence.
Machine translation has also reached the Lithuanian language. Today, readily available online machine-translation tools translate from the most widely spoken languages into Lithuanian and vice versa. The translation quality of most tools is still quite poor, and their output must be edited. Only the Google Translate tool stands out for its better qualityGoogle Translate2, in which the use of
1 This article is based on the presentation “Machine Translation for Lithuanian,” delivered in Vilnius at the 24th International Jonas Jablonskis Conference, “Digital Language Resources, Their Development Directions and Potential Uses,” on September 29, 2017.
2 Available online https://translate.google.com.
BENDRINĖ KALBA 90 (2017) www.bendrinekalba.lt ISSN 2351-7204
Page 2
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 2
neural translation technologies. Most sentences translated using this tool retain the meaning they had in the original language.
Machine translation is widely used to disseminate information in other languages. Machine-translation results are often irresponsibly uploaded to the internet completely unedited. The consequences of such actions are already apparent: the internet is awash with poor Lithuanian texts containing distorted information, automatically generated by one machine-translation program or another3. Gradually, such texts may also seep into corpora. Machine translation has already become an everyday reality—sometimes astonishing, sometimes glaringly obvious—that deserves not only closer attention from linguists but also their more substantial involvement in the development of machine translation.
This article briefly presents the history of machine translation (MT) and introduces some ideas that have influenced its development. It presents the results achieved by various translation technologies for Lithuanian. It examines the possibilities of applying MT to Lithuanian using rule-based, statistical, and neural methods. For lack of space, attempts to combine several of these methods in a single translation system are not discussed.
1. ORIGINS OF MACHINE TRANSLATION
Good translation from another language has always been a certain intellectual challenge for people, and translation has never been an easy undertaking. It is no wonder that efforts were made to automate translation somehow. Between 1932 and 1935, the French engineer Georges Artsrouni invented and built a universal mechanical device that he called a mechanical brain (French cerveau mécanique), which, among its other functions, could translate typed text into several languages (Hutchins 2004a: 12). This electromechanical device contained a long (up to 40 metres) memory tape made of flexible cardboard. It held a multilingual dictionary containing as many as several tens of thousands of words. A French word typed on the keyboard was mechanically encoded, and the memory tape rotated to the appropriate position according to that code. After a few seconds, the device displayed the translation of the word entered (Daumas 1965: 294–295). The device was demonstrated at the 1937 Paris World's Fair, attracted great interest, and was awarded the Grand Prize. In fact, it was the world's first
3 Jonathan Swift, in Gulliver’s Travels, sarcastically describes a machine that arranges meaningless combinations of words, which were continually and painstakingly written down in books. Nowadays, the same kind of drivel can be produced much faster...
BENDRINĖ KALBA 90 (2017)
www.bendrinekalba.lt
ISSN 2351–7204
Page 3
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 3
working mechanical dictionary. G. Artsrouni continued to improve his device, but mechanical problems and the outbreak of the Second World War halted the work.
In 1933, in the Soviet Union, Piotr Smirnov-Trojansky received an inventor’s certificate for an electromechanical translation machine4. Although the machine was never built, its inventor formulated many valuable ideas about machine translation that were realized only many years later, with the advent of computers. P. Smirnov-Trojansky spoke Russian and therefore had a good understanding of the characteristics of inflected languages. He proposed dividing the translation process into three stages: in the first stage, a human would reduce the text to lemmas and annotate them with morphological-syntactic tags (the analysis stage); in the second stage, the machine would automatically find the lemmas in one or more target languages and assign them the same tags (the transfer stage); and in the third stage, an editor would use the target-language lemmas and their tags to prepare a fluent translation (the generation stage). P. Smirnov-Trojansky also proposed various methods for translating synonyms, homonyms, and idioms. P. Smirnov-Trojansky’s ideas were not appreciated in his homeland; he did not live to see the computer era5, but the inventor’s ideas were not forgotten and later became known in the West as well.
The era of practical machine translation began with the advent of computers. In 1947, W. Weaver wrote to the brilliant linguist and computer designer Norbert Wiener, asking whether the latest advances in cryptography and the newly developed computers6 could be applied to language translation. W. Weaver argued that any Russian text could be treated as an English message encoded in Cyrillic letters. He thought computers could simply decipher that message. Unfortunately, N. Wiener’s response was skeptical.
In 1949, W. Weaver sent a memorandum to a wider audience, proposing the use of computers to translate texts (Weaver 1949: 7)7. At the time, ideas for simple, word-for-word machine translation prevailed in various American and British laboratories. The scholar was convinced that computers could translate much better. In the memorandum, W. Weaver proposed four approaches to solving and researching translation problems. First of all, he proposed statistically analyzing the immediate context of a word being translated in order to solve problems of polysemy. This insight now forms the basis of statistical machine translation. The second proposal was based on the assumption that in language
4 In the inventor’s certificate, the device is called a “machine for selecting and printing words when translating from one language into another or several languages simultaneously.”
5 P. Smirnov-Trojansky died in 1950.
6 The first British electronic computer, Colossus, began operating in early 1944; the first American electronic digital computer, ENIAC, was built in late 1945. The first Soviet computer, MESM, was built in 1950.
7 The term machine translation (English machine translation) used in W. Weaver’s memorandum soon caught on.
BENDRINĖ KALBA 90 (2017) www.bendrinekalba.lt ISSN 2351–7204
Page 4
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 4
contains logical elements, so using neural network models (McCulloch, Pitts 1943: 115–133), it should be possible to deduce the translation from the assumptions present in the source language. Time has demonstrated the brilliance of this insight—today, the neural machine translation method produces the highest-quality translations. The third approach to solving the problems should consist of applying cryptographic methods to language translation. Forty years later, this was implemented in statistical machine translation algorithms. To illustrate the fourth proposal, W. Weaver drew an allegorical picture in which people were housed in tall, enclosed towers symbolizing different languages. Somehow, people manage to communicate by calling to one another through the thick walls of the towers. Fortunately, the towers share a common foundation and basement; they need only go down there, where communication should be easier. W. Weaver argued that the world’s languages contain a certain common deep structure that still needs to be uncovered. The scientist’s latter proposal prompted the search for and construction of a universal language—the machine translation interlingua8.
Machine translation soon gained momentum after W. Weaver’s memorandum. In 1954, a machine translation system developed through collaboration between IBM and Georgetown University was publicly demonstrated in New York. It translated from Russian into English. The system had only six grammatical rules programmed into it, and approximately 250 words were used for translation (Hutchins 2004b: 102). The system focused on organic chemistry, and the translation examples had been carefully prepared in advance. After the experiment, there was hope that machines would translate perfectly within a few years, and generous funding was allocated to continue the work. MT began to develop rapidly in the United States of America (USA) and the Soviet Union, with the aim of gaining a strategic advantage in the Cold War. The most commonly translated languages were Russian and English.
2. RULE-BASED MACHINE TRANSLATION
In the mid-1950s, machine (computer) translation systems emerged that came to be known as rule-based systems (English rule-based). They were developed on the premise that language could be described using a set of certain rules (including grammatical ones). Under this method, a sentence in the source language is analyzed to a chosen level, its structure is identified, and this is transformed according to established rules into an equivalent structure in the target language, from which a sentence in the target language is generated. Sentence structures in different languages can differ considerably and have various features that must be reflected in the MT
8 It should not be confused with natural-language interlinguas—Esperanto, Ido, Volapük, and so on.
BENDRINĖ KALBA 90 (2017)
www.bendrinekalba.lt
ISSN 2351–7204
Page 5
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 5
system. Daiva Šveikauskienė (2005: 411–417) investigated the problems of representing Lithuanian sentence syntactic structures in a form suitable for machine translation. There are various types of rule-based MT systems, differing in the depth of analysis (Fig. 19). The deeper the sentence analysis, the simpler the transformation. Ideally, if a text is analyzed to the level of an interlingua, no transformation at all would be needed—the interlingual representation of the text should be the same in all languages.
FIG. 1. Vauquois triangle of rule-based machine translation
Attempts to address the problems of rule-based MT have applied both empirical and linguistic methods. Among empiricists, the most popular was the direct translation method. In translation between two languages using this method, only a dictionary and simple programming rules are used; virtually no language analysis or syntactic reorganization is performed (this method corresponds to the lowest level in Fig. 1). In the 1950s in the USA, a group of researchers led by Erwin Reifler developed special dictionaries for machine translation, which included rules for local changes in word order alongside lexical equivalents. They also provided translations of numerous phrases and generalizing concepts (English cover terms) to address problems of polysemy. In 1964, Mark II, a Russian–English direct translation system developed on commission from the U.S. Air Force, began operating. It used E. Reifler’s dictionary, which contained over 170,000 words (Reifler 1960: 312).
Linguistic methods were applied more extensively at Georgetown University, where a large contingent of machine translation researchers gathered and split into several groups applying different methods
9 Such illustrations are often called the Vauquois triangle; this visualization method was proposed by Bernard Vauquois, a French pioneer of machine translation (1976: 335).
BENDRINĖ KALBA 90 (2017)
www.bendrinekalba.lt
ISSN 2351–7204
Page 6
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 6
groups. In the group led by Michael Zarechnak, the Georgetown Automatic Translation (GAT – Georgetown Automatic Translation) system was developed. It performed three levels of analysis: morphological (including the identification of idioms), syntactic, and syntagmatic. The modified GAT system was successfully put into use at Euratom (in Ispra, Italy) in 1963 and at the U.S. Atomic Energy Commission in 1964. Developing an MT system based on an interlingua took much longer. The commercially viable MT system KANT appeared only in the last decade of the twentieth century (Mitamura et al. 1991: 105–118). The system was used to translate some technical texts. An interlingua for expressing general language was never developed.
For the entire decade following the IBM–Georgetown MT demonstration, great efforts were made to substantially improve MT translation quality10, but progress was very limited. In 1966, the ALPAC (Automatic Language Processing Advisory Committee) committee was established in the United States and concluded that MT had no prospects in the near future because its quality was poor and it still had to be edited by a translator, who spent more time doing so than translating without any machines. The ALPAC committee disregarded the fact that machine translation helps translate enormous volumes of information, and that many users are satisfied with poor translations that convey barely the gist. Following the committee’s negative conclusions, funding for MT projects in the United States was practically halted for several decades, and funding also declined in other countries.
Despite the skeptical attitude toward MT, steady progress continued in this field. The rule-based translation system SYSTRAN, whose development began in the United States in 1968 (Toma 1977: 569–581), was continuously improved and later became a commercial product. The system was first used by the U.S. Department of Defense, NASA, Euratom, the European Commission, and many other organizations. Under a commercial agreement in 1975, the European Commission obtained the right to improve the system for its own needs. The system translated texts from certain subject areas into many languages reasonably well. From 2010, SYSTRAN began using hybrid technologies. Other countries also developed fairly effective MT systems: ARIANE (in Grenoble, France), the similar MU system (in Kyoto, Japan), and SUSY (in Saarbrücken, Germany). The European EUROTRA project (1982–1992), which cost more than 50,000,000 ECU11, ended in failure—hundreds of specialists never succeeded in creating a functioning MT system. Thus began a serious crisis in rule-based MT. For many years afterward, progress stagnated.
10 During this period, the United States spent about $20 million on developing and improving MT systems.
11 In a 1994 report by the Commission of the European Communities (CEC – Commission of the European Communities), data on EUROTRA funding were presented: in 1982–1992, the Commission allocated 37.5 million ECU to the project (CEC 1994: 3.10), while the governments of various countries provided more than 20 million ECU in additional funding to the sixteen EUROTRA centers (CEC 1994: A4.2).
BENDRINĖ KALBA 90 (2017) www.bendrinekalba.lt ISSN 2351–7204
Page 7
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 7
Rule-based machine translation appeared in Lithuania only in this century. In 2003–2004, the machine translation system Česilko (Hajič et al. 2000: 10–12) was adapted for Lithuanian (Homola, Rimkutė 2004: 77–81). During that period, Petras Homola and Erika Rimkutė studied the system’s machine translation capabilities between Czech and Lithuanian. From 2005 to 2007, Vytautas Magnus University carried out the project “Online Information Translation Tool,” funded by the European Union Structural Funds. The structure of the planned MT system is described by Vidas Daudaravičius (2006: 7–18). A public online translation tool from English into Lithuanian was developed12. The translation engine was provided by the Russian company PROMT. Some of the system’s linguistic components were prepared in Lithuania. The system translated only certain sentence types very well; good translations were in the minority. In 2007, a linguistic evaluation of this MT system was carried out (Rimkutė, Kovalevskaitė 2008: 257–264). Audronė Daubarienė and Greta Ziezvytė (2013: 55–61) evaluated the conformity of translations produced by the VMU MT system with standards of textuality (De Beaugrande, Dressler 1981: 3). In their study, the authors found that the machine-translated texts in different genres that they examined failed to meet the standards of coherence and some other standards of textuality because of numerous semantic errors; consequently, such translations cannot be acceptable or informative to readers.
As the decades passed and, essentially, progress stagnated, it came to be understood that language is an extremely complex phenomenon that is very difficult to describe with rules. To address problems of ambiguity and anaphora translation, the use of extralinguistic information about the structure of the world and its logic was proposed (Bar-Hillel 1958: 197–207). However, no such systematized information existed, and formally describing the meaning of a text seemed a more difficult task than translation itself. It was clear that machine translation problems required revolutionary solutions.
2. STATISTICAL MACHINE TRANSLATION
In 1988, a group of IBM researchers (Brown et al. 1988: 71–76) proposed using information embedded in parallel corpora for machine translation, searching for the most likely translation alternatives. The proposed translation model is based on the assumption that, in a noisy channel, a sentence in the source language f is transformed into a sentence e in the target language. Noise in the channel statistically distorts the source-language sentence f, so at the channel output we can obtain various sentences e, of every conceivable length, that have one meaning or another in the target language—or no meaning at all. Since the channel strives to translate correctly, correct translation alternatives increasingly make it through the noise.
12 Available online http://vertimas.vdu.lt/twsas/.
BENDRINĖ KALBA 90 (2017) www.bendrinekalba.lt ISSN 2351-7204
Page 8
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 8
In this model, it is assumed that when a sentence f is sent into a channel, the most suitable translation ê among all e is the one whose conditional probability of appearing at the channel output, p(e|f), is greatest, i.e.
ê = argmaxe p(e|f).
Here we encounter the problem that finding the best translation ê requires examining not only all possible sentences in the target language, but also every possible arrangement of words in it. Calculating such a number of probabilities is very difficult. Applying Bayes’ theorem yields an equivalent expression that is easier to calculate (Koehn et al. 2003: 127):
argmaxe p(e|f) = argmaxe p(f|e) p(e)
Ši lygtis yra pamatinė statistinio mašininio vertimo lygtis (Brown ir kt. 1993). Joje sąlyginė tikimybė p(f|e) parodo tikimybę, kad kanalu buvo pasiųstas sakinys f, jei išėjime pasirodė sakinys e. Šis lygties narys vadinamas vertimo modeliu, kuris apskaičiuojamas iš dvikalbių tekstynų. Antrasis lygties narys p(e) parodo sakinio e tikimybę vertimo kalboje. Šis narys aprašo kalbos modelį, kuris apskaičiuojamas iš vienkalbių vertimo kalbos tekstynų13.
FIG. 2. Vauquois triangle of statistical machine translation
The translation method proposed by IBM researchers immediately began competing with rule-based translation in terms of translation quality. Developers of statistical MT systems were attracted by the possibility of translating into many languages without any knowledge of translation or grammar; it was enough to train the systems being developed on available corpora. In classical statistical translation, no
13 The language model reduces the amount of computation because it helps immediately discard sentences not found in the corpora and select the most fluent translation; it also indirectly explains why literary works are better translated by a translator whose target language is their native language.
BENDRINĖ KALBA 90 (2017)
www.bendrinekalba.lt
ISSN 2351–7204
Page 9
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 9
analysis of sentences in the source language or synthesis of sentences in the target language is performed (Fig. 2); translation is based solely on probabilities determined during system training. The five word-alignment models for translation proposed by Peter Brown and his colleagues were not sufficiently flexible: they allowed a source-language word to be translated by one or more words, or not translated at all. In 2003, a more flexible machine translation model capable of translating source-language phrases was proposed (P. Koehn et al. 2003: 127–133).
During this period, methods for automatically evaluating machine translation quality were rapidly improved, since it was very important to quickly determine which version of a machine translation system being tuned translated best. Another aim was for such evaluation to allow objective comparison of the translation quality of different systems. Over time, BLEU (an acronym for bilingual evaluation understudy) became one of the most popular metrics (Papineni et al. 2002: 311–318); it automatically compares how far a machine-translated text diverges from a human translation. This metric does not take into account the comprehensibility or grammatical correctness of the text; it counts only exactly matching words or phrases. In calculating the metric, matching phrases have a relatively greater statistical weight than individual matching words. Depending on the task, other metrics with one desirable property or another are also often used to evaluate MT quality: F-measure, METEOR (Banerjee, Lavie 2005: 62–72), NIST (Doddington 2002: 138–145), TER (Snover et al. 2006: 223–231), and many others.
Despite the popularity of statistical MT, the limitations of this method for translating into morphologically rich (inflectional) languages became increasingly apparent. Parallel corpora for inflectional languages usually do not contain all possible phrase forms or even all forms of less common individual words. If some possible forms were not included in training the MT system, those forms will not be translated. This shortcoming of classical MT is caused by data insufficiency, which results in a sparse translation probability matrix after the system is trained. To overcome this shortcoming, Philip Koehn and Hieu Hoang proposed factored statistical machine translation (Koehn, Hoang 2007a: 868–876). They proposed incorporating additional linguistic information into the translation system by annotating parallel corpora with linguistic analysis tags, which are then used to train the system. In 2007, P. Koehn and colleagues released a complete statistical MT package, Moses, as open-source software for the general public (Koehn et al. 2007b: 177–180). Applying factored statistical MT slightly improved translation quality for inflectional languages, typically by 1–2 BLEU points (Bojar 2007: 235–236; Skadinš et al. 2010: 128–129).
From September 25, 2008, the statistical MT system Google Translate also began translating into Lithuanian. From 2012 to 2014, Vilnius University carried out [projects of] the European Union Structural Funds
BENDRINĖ KALBA 90 (2017)
www.bendrinekalba.lt
ISSN 2351–7204
Page 10
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 10
the funded project “Development of an English–Lithuanian–English and French–Lithuanian–French machine translation system based on statistical methods”. Most of the project work was carried out by the company Tilde. The MT system ALPMAVIS was developed, and a public online statistical MT service became available to the public (https://www.versti.eu/)14. According to tests conducted in 2014, quality scores for translating general texts (on the BLEU metric) more than doubled the results of Lithuanian rule-based translation and were practically on a par with those of the Google Translate translation system. ALPMAVIS is particularly good at translating legal texts.
3. QUALITY OF MACHINE TRANSLATION INTO LITHUANIAN
With the emergence of a new linguistic phenomenon—Lithuanian texts translated by machines—it became interesting to assess the quality of such translation in relation to standard language. A general-purpose text was chosen for evaluation, as it better reflects standard language, and the BLEU metric was used to measure translation quality.
A general-purpose text published online was selected for testing translation quality15.
BLEU scores were calculated on a longer document than the excerpts shown below, since sentence-level BLEU scores correlate poorly with human quality assessments (Kos, Bojar 2009: 140). The information provided will help the reader understand how the percentage scores of BLEU CI (calculated without distinguishing between uppercase and lowercase letters) relate to machine translation quality. On the other hand, even without looking at the scores, readers can form their own opinion of how MT systems based on the models described earlier translate.
The following are translations of the same original text, produced using various translation methods. Before each translated text, the translation method, the date of online translation, and the BLEU CI percentage score are indicated.
Original text in English:
Everyone wants a long, happy and healthy life, and scientists are doing their best to make this possible. They are tackling some of the major current health issues as well as emerging threats such as the increasing impact of Alzheimer’s disease, diabetes and antibiotic-resistant ’superbugs’. Investment in health research and innovation will help us stay active, develop new, safer and more effective treatments and help keep our health and care systems
14 ALPMAVIS is also accessible through the information system for integrated Lithuanian language and writing resources, Raštija.lt (www.raštija.lt).
15 Iš programos HORIZON 2020. Europos Komisijos Mokslinių tyrimų ir inovacijų generalinis direktoratas 2014.
BENDRINĖ KALBA 90 (2017) www.bendrinekalba.lt ISSN 2351–7204
Page 11
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 11
viable. It will give doctors the tools they need for more personalised medicine, and it will step up prevention and treatment of chronic and infectious diseases.
Translator’s translation:
Everyone wants to live a long and happy life and be healthy, and scientists are doing everything they can to make this possible. They are trying to solve some of the most important existing health problems and emerging health threats, such as the growing problems caused by Alzheimer’s disease, diabetes, and antibiotic-resistant microorganisms. Investment in health research and innovation will help us remain active, develop new, safer, and more effective treatments, and keep our healthcare system viable. It will give doctors the tools they need to tailor medicines to individual patients’ needs; moreover, it will contribute to the prevention and treatment of chronic and communicable diseases.
Everyone wants a long, happy, and healthy life, and scientists are trying to make this possible. They are addressing some of the main current health problems, as well as emerging threats such as the increasing impact of Alzheimer’s disease, diabetes, and antibiotic-resistant “superbugs.” Investment in health research and innovation will help us stay active, develop new, safer, and more effective treatments, and help keep our healthcare systems viable. It will give doctors the tools they need for more personalized medicine, and it will improve the prevention and treatment of chronic and infectious diseases.
Everyone wants a long, happy, and healthy life, and scientists are trying to make this possible. They are addressing some of the main health problems and emerging threats, such as the increasing impact of Alzheimer’s disease, diabetes, and antibiotic-resistant “superbugs.” Investment in health research and innovation will help us stay active, develop new, safer, and more effective treatments, and help keep healthcare systems viable. This will give doctors the tools they need for more personalized medicine, and it will promote the prevention and treatment of chronic and infectious diseases.
Google statistical machine translation, 2014 09 19, BLEU – 17.10:
Each of us strives to be healthy and happy for a long time, and scientists are doing their utmost to make this happen. They address some of the main health problems as well as new threats, such as the increasing impact of Alzheimer’s disease, diabetes, and antibiotic-resistant “superbugs.” Investment in health research and innovation will help us stay active, develop new, safer, and more effective treatments, and help support the sustainability of our healthcare systems. This will provide doctors with the tools they need to practice more personalized medicine and make greater efforts to prevent and treat infectious and chronic diseases.
BENDRINĖ KALBA 90 (2017)
www.bendrinekalba.lt
ISSN 2351-7204
Page 12
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 12
Everyone wants a long, happy, and healthy life, and scientists are doing everything they can to make this possible. They are fighting some of today’s major health problems, as well as new threats such as Alzheimer’s disease, diabetes, and the growing impact of antibiotic-resistant superbugs. Investment in health research and innovation will help us remain active, develop new, safer, and more effective treatments, and help keep our health and healthcare systems viable. It provides the tools needed for more personalized medicine and will strengthen the prevention and treatment of chronic and infectious diseases.
The presented machine-translation excerpts reflect the typical quality of machine translation when translating general texts. Much better general-purpose translations are obtained only when the text being translated has at some point been used to train the system. An objective assessment of an MT system’s translation quality is possible only using texts that have never been included in the system’s training data. It must be acknowledged that currently operational statistical and rule-based MT systems still translate general texts into Lithuanian poorly.
4. NEURAL MACHINE TRANSLATION
In 2013, Nal Kalchbrenner and Phil Blunsom proposed using neural networks for machine translation (Kalchbrenner, Blunsom 2013: 1700–1709). The idea of using computer graphics processors for neural-network computation opened up the possibility of tackling real-world problems, including machine translation. Neural MT (NMT) is implemented using millions of artificial neurons. The concept of NMT is fairly simple: the encoder transforms an entire source-language sentence into a so-called thought vector16 in a vector space with several hundred dimensions. The vector consists of a sequence of numbers that represents the meaning of the sentence. In effect, this vector is a vectorized thought of the sentence. The directions of vectors representing related objects and ideas differ only slightly17. The decoder generates a translation based on the thought vector. Both the encoder and the decoder are built on recurrent neural networks (Sutskever et al. 2014: 3104–3112; Cho et al. 2014: 1724–1734).
16 The term was popularized by Geoffrey Hinton, one of the most prominent developers of artificial intelligence.
17 This can be verified, for example, by experimenting with the vectorization tool word2vec (Mikolov et al. 2013: 3111–3119).
BENDRINĖ KALBA 90 (2017) www.bendrinekalba.lt ISSN 2351–7204
Page 13
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 13
FIG. 3. Vauquois triangle for neural machine translation
Figure 3 presents a neural MT diagram based on the Vauquois triangle18. It should be noted that a high level of analysis, approaching interlingua, is achieved automatically. The NMT system is trained using parallel corpora. During training, recurrent neural networks automatically encode the morphological, syntactic, and semantic information in sentences, along with regularities detected to varying degrees. At last, machine translation has become inseparable from the development of artificial intelligence. Neural machine translation is advancing rapidly and produces much better results than rule-based or statistical machine translation.
Exploration of NMT capabilities soon revealed a new phenomenon: neural networks were able to translate between language pairs for which they had not been directly trained. If an NMT system was trained to translate from language A into language B and also from language A into language C, then the system can also translate from language B into C (Johnson 2017: 339–351), although the translation quality is somewhat lower. This phenomenon can be called zero-shot translation (zero-shot translation). This means that, during training, an MT system’s neural network develops its own interlingua, which is understood only by the network itself.
At the end of 2016, Google Translate (GT) began using neural translation. Here is what the neural translation of our illustrative example looks like (Google neural MT 2017 09 14, BLEU CI – 25.94):
18 A depiction of neural machine translation based on the Vauquois triangle was published online by SYSTRAN in 2016 in “Pure Neural Machine Translation.” Available online: http://www.systransoft.com/download/white-papers/systran-white-paper-PNMT-12-2016_2.pdf, (accessed 2017 11 06).
BENDRINĖ KALBA 90 (2017)
www.bendrinekalba.lt
ISSN 2351–7204
Page 14
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 14
Everyone wants a long, happy, and healthy life, and scientists are doing everything they can to make this possible. They are addressing some of today’s most important health problems, as well as emerging threats, such as the increasing impact of Alzheimer’s disease, diabetes, and antibiotic-resistant “superbugs.” Investment in health research and innovation will help us stay active, develop new, safer, and more effective treatments, and keep our health and care systems sustainable. This will give doctors the tools they need for more personalized medicine, thereby improving the prevention and treatment of chronic and infectious diseases.
It must be acknowledged that our example was translated by the neural network incomparably better than by rule-based or statistical machine translation systems. For now, GT does not translate everything equally successfully; for example, when an attempt was made to translate Beauty requires no ornaments, the result was Beauty does not need ornaments. Similar experiments can reveal many cases in which GT translates very poorly, so there is still room for improvement.
In 2018, Vilnius University plans to begin implementing a new-generation neural machine translation system for English, Lithuanian, Polish, French, Russian, and German. The translation system will integrate automatic text-to-speech conversion and the translation of text dictated in Lithuanian.
5. FINAL REMARKS
Today, advances in machine translation are inseparable from progress in the development of artificial intelligence. In terms of translation quality, all classical MT methods need improvement; their potential has not been exhausted, and their limits are unclear. The romantic era of easy gains has ended, and heuristic mathematical or engineering methods do not always help. Children learn language almost automatically, without effort, but their language is later perfected only by learning grammar and studying literature. It seems unlikely that machine translation can be further improved without the substantial involvement of linguists. Such involvement is facilitated by machine translation tools implemented on open-source platforms: for example, Apertium for rule-based translation (Forcada et al. 2011: 127–144); Moses (Koehn et al. 2007b: 177–180), LetsMT! (Vasiljevs et al. 2012: 43–48) for statistical translation; TensorFlow and OpenNMT for neural translation. We can only hope that Lithuanian linguists will unite their efforts to address the challenges of modern computational linguistics, since some of them are existentially important for our language.
BENDRINĖ KALBA 90 (2017)
www.bendrinekalba.lt
ISSN 2351–7204
Page 15
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 15
LITERATŪRA
Banerjee S., Lavie A. 2005: METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. – Proceedings of Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization at the 43th Annual Meeting of the Association of Computational Linguistics (ACL-2005), Ann Arbor, Michigan, 62–72.
Bar-Hillel Y. 1958: Some linguistic obstacles to machine translation. – Proceedings of the Second International Congress on Cybernetics, Namur, 197–207.
Bojar O. 2007: English-to-Czech factored machine translation. – Proceedings of the Second Workshop on Statistical Machine Translation, Prague, Czech Republic, 232–239.
Brown P. F., Cocke J., Della Pietra S. A., Della Pietra V. J., Jelinek F., Mercer R. L., Roossin P. S. 1988: A statistical approach to language translation. – Proceedings of COLING 1988: The 12th International Conference on Computational Linguistics, Budapest, 71–76.
Brown P. F., Della Pietra S. A., Della Pietra V. J., Mercer R. L. 1993: Mathematics of statistical machine translation: Parameter estimation. – Computational Linguistics 19(2), 263–311.
Cho K., Bart van Merrienboer, Gulcehre C., Bougares F., Schwenk H., Bengio Y. 2014: Learning phrase representations using RNN encoder-decoder for statistical machine translation. – Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics, Doha, Qatar, 1724–1734.
Commission of the European Communities 1994: Final Evaluation Of The Results Of Eurotra: A Specific Programme Concerning The Preparation Of The Development Of An Operational Eurotra System For Machine Translation. DIANE Publishing Company, 1995, 194.
Daubarienė A., Ziezytė G. 2013: Machine translation: translated texts in terms of standards of textuality. – Kalbų studijos 22, 55–61.
Daudaravičius V., 2006: Pradžia į begalybę. Mašininis vertimas ir lietuvių kalba. – Darbai ir dienos: Pažangos šuoliai 45, 7–18.
Daumas M. 1965: Les machines à traduire de Georges Artsrouni. – Revue d'histoire des sciences et de leurs applications, tome 18, n°3, 283–302.
BENDRINĖ KALBA 90 (2017) www.bendrinekalba.lt ISSN 2351–7204
Page 16
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 16
De Beaugrande R. A., Dressler W. U. 1981: Introduction to Text Linguistics, London: Longman, 270.
Doddington G. 2002: Automatic Evaluation of Machine Translation Quality Using Ngram Co-occurrence Statistics. – Proceedings of the Second International Conference on Human Language Technology Research, San Francisco, CA, USA., Morgan Kaufmann Publishers Inc., 138–145.
Forcada M. L, Ginestí-Rosell M., Nordfalk J., O'Regan J., Ortiz-Rojas S., Pérez-Ortiz J. A., Sánchez-Martínez F., Ramírez-Sánchez G., Tyers F. M. 2011: Apertium: a free/open-source platform for rule-based machine translation. – Machine Translation 25 (2), Free/Open-Source Machine Translation (June 2011), 127–144.
Hajič J., Hric J., Kuboň V. 2000: Machine Translation of Very Close Languages. – Proceedings of the 6th Applied Natural Language Processing Conference, Association for Computational Linguistics, 7–12.
Homola P., Rimkutė E. 2004: Artimų kalbų mašininis vertimas. – Kalbų studijos 6, 77–81.
Hutchins J. 2004a: Two precursors of machine translation: Artsrouni and Trojanskij. – International Journal of Translation 16(1), 11–31.
Hutchins J. 2004b: The Georgetown-IBM experiment demonstrated in January 1954. – Proceedings of the 6th Conference of the Association for Machine Translation in the Americas, AMTA, Washington, DC, 102–114.
Johnson M., Schuster M., Le V. Q., Krikun M., Wu Y., Chen Z., Thorat N., Viégas F. B., Wattenberg M., Corrado G., Hughes M., Dean J. 2017: Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation. – Transactions of the Association of Computational Linguistics 5 (1), 339–351.
Kalchbrenner N., Blunsom P. 2013: Recurrent Continuous Translation Models. – Proc. of EMNLP, October, Association for Computational Linguistics, Seattle, Washington, USA, 1700–1709.
Koehn P., Hoang H. 2007a: Factored translation models. – Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, ACL, 868–876.
Koehn P., Hoang H., Birch A., Callison-Burch C., Federico M., Bertoldi N., Cowan B., Shen W., Moran C., Zens R., Dyer C., Bojar O., Constantin A., Herbst E. 2007b: Moses: Open source toolkit for statistical machine
BENDRINĖ KALBA 90 (2017)
www.bendrinekalba.lt
ISSN 2351-7204
Page 17
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 17
translation. – 45th Annual Meeting of the Association for Computational Linguistics (ACL), Companion Volume, 177–180.
Koehn P., Och F. J., Marcu D. 2003: Statistical phrase-based translation. – Proceedings of HLT-NAACL, 127–133.
Kos K., Bojar O. 2009: Evaluation of Machine Translation Metrics for Czech as the Target Language. – Prague Bull. Math. Linguistics 92, 135–148.
McCulloch W., Pitts W. 1943: A logical calculus of the ideas immanent in nervous activity. – Bulletin of Mathematical Biophysics 5, 115–133.
Mikolov T., Sutskever I., Chen K., Corrado G. S., Dean J. 2013: Distributed Representations of Words and Phrases and their Compositionality – Proceedings of the 26th International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada, Vol. 2, 3111–3119.
Mitamura T., Nyberg E., Carbonell J. 1991: An efficient interlingua translation system for multi-lingual document production. – Proceedings of Machine Translation Summit III, Washington, DC. Perspausdinta: – Progress in Machine Translation, ed. S. Nirenburg, IOS Press, (1993), 105–118.
Papineni K., Roukos S. Ward T., Zhu W. J. 2002: BLEU: a method for automatic evaluation of machine translation. – ACL-2002: 40th Annual meeting of the Association for Computational Linguistics, 311–318.
Reifler E. 1960: The solution of MT linguistic problems through lexicography. – Proceedings of the National Symposium on Machine Translation, (February 2–5, 1960), ed. H.P. Edmunson, London: Prentice-Hall, (1961), 312–316.
Rimkutė E., Kovalevskaitė J. 2008: Linguistic Evaluation of the First English-Lithuanian Machine Translation System. – Proceedings of the Third Baltic Conference on Human Language Technologies (2007), Kaunas, 257–264.
Skadinš R., Goba K., Šics V. 2010: Improving SMT for Baltic Languages with Factored Models. – Proceedings of the Fourth International Conference Baltic HLT, Frontiers in Artificial Intelligence and Applications, Vol. 219, IOS Press, 125–132.
Snover M., Dorr B., Schwartz R., Micciulla L., Makhoul J. 2006: A Study of Translation Edit Rate with Targeted Human Annotation. – Proceedings of the 7th Conference of the Association for Machine Translation in the Americas, Morristown, NJ, USA, 223–231.
BENDRINĖ KALBA 90 (2017)
www.bendrinekalba.lt
ISSN 2351–7204
Page 18
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 18
Sutskever I., Vinyals O., V. Le Q. V. 2014: Sequence to sequence learning with neural networks. – Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal, Canada, Vol. 2, MIT Press Cambridge, MA, USA, 3104–3112.
Šveikauskienė D. 2005: Graph Representation of the Syntactic Structure of the Lithuanian Sentence. – Informatica 16 (3), 407–418.
Toma P. 1977: Systran as a multilingual machine translation system. – Proceedings of the Third European Congress on Information Systems and Networks, Overcoming the language barrier, München, 569–581.
Vasiljevs A., Skadiņš R., Tiedemann J. 2012: LetsMT!: a cloud-based platform for do-it-yourself machine translation – Proceedings of the ACL, System Demonstrations, Jeju Island, Korea, 43–48.
Vauquois B. 1976: Automatic translation – A survey of different approaches. – COLING-76, Ottawa. Perspausdinta: – Readings in machine translation, eds. S. Nirenburg, H. L. Somers, Y. Wilks, MIT Press, (2003), 333–337.
Weaver W. 1949: Translation. – Perspausdinta: – Machine Translation of Languages, MIT Press, Cambridge, MA. (1955), 15–23.
Įteikta 2017 11 13
Priimta 2017 12 20
MACHINE TRANSLATION FOR LITHUANIAN LANGUAGE
Summary
The paper presents a historical overview as well as current state of the art of the machine translation. Translation from another language is always a certain intellectual challenge. In 1949 Warren Weaver suggested using computers to translate texts. The term "machine translation" (MT) appears. Machine translation has been rapidly developing during the first decades in order to gain a strategic advantage in the Cold War. Most popular translated languages were Russian and English. Word-to-word translation and large bilingual dictionaries covering more than 170,000 words prevailed.
BENDRINĖ KALBA 90 (2017)
www.bendrinekalba.lt
ISSN 2351–7204
Page 19
DANIELIUS ALGIRDAS RALYS. Masininis vertimas lietuvių kalbai | 19
In the 1950s and 1960s, machine translation systems appeared which could be called rule-based. They are based on the assumption that a language can be described using a set of rules (including grammatical). It was a rather optimistic period – it was expected to create a perfect machine translation in a few years. But a computer hardly "understands" grammar. Highly inflected languages require tens of thousands thoroughly hand-tuned and mutually consistent rules. Nobody has done this properly yet. The most advanced systems were the SYSTRAN MT system, launched in the European Commission and the Russian PROMT translation system. Subsequently, the rule-based MT progress slowed down. The EUROTRA project (1982–1992), by some estimates costing more than 50,000,000 ECU, fails – even hundreds of recruited specialists failed to create a functioning MT system. This marked a serious rule-based MT crisis. Development simply stalled for many forthcoming years.
The question was raised: if we cannot write so many rules, can it be translated without grammar at all? In 1990 a new breakthrough emerges – the research team at the IBM Thomas J. Watson Research Center formulates the basics of statistical machine translation. The translation process has been considered as a transmission of a certain message over a noisy channel. Decoding then has been performed on the basis of the Bayesian theorem. Translation is based on text corpora, especially on large parallel bilingual text corpora. There was a rapid improvement of statistical MT. The EuroMatrix project, supported by the European Commission, has created a universal open source machine translation software package MOSES, based on industry-level MT systems. Good results have been obtained – it turns out that you can translate without any dictionary or grammar! This method has greatly facilitated the translation of highly inflected languages too.
The achievements of machine translation today are effectively applied to the Lithuanian language as well. During 2005–2007 Vytautas Magnus University has carried out an EU-funded project "Internet Information Translator". The result was a public online translation service from English to Lithuanian. (http://vertimas.vdu.lt/twsas/). The rule-based translation engine was provided by the Russian company PROMT, while other linguistic resources were prepared in Lithuania. The overall quality of text translation in BLEU metrics (in percent) is about 10. In practice, this means that only every third sentence can be adequately understood. This translation tool still has considerable potential for improving, for example by expanding phrase dictionary. Since September 25, 2008 Google Translate also supports Lithuanian. According to the results of the tests (2014), the BLEU translation quality was estimated to be around 17.
BENDRINĖ KALBA 90 (2017)
www.bendrinekalba.lt
ISSN 2351-7204
Page 20
DANIELIUS ALGIRDAS RALYS. Mašininis vertimas lietuvių kalbai | 20
2012–2014 Vilnius University implemented the EU-funded project "Creation of English-Lithuanian-English and French-Lithuanian-French machine translation system based on statistical methods". The result was a public online translation service (https://www.versti.eu/). According to the tests carried out in 2014, the BLEU estimates of the translation quality exceeded more than twice the rule-based translation results and were practically equivalent to the Google translation system. When translating the documents of certain domain (such as law) the achieved BLEU score is roughly twice as high as translating general texts and far exceeds Google's results (19-09-2014). However, even the best machine translations often require human intervention and final editing in order to get the perfect translation. So, can machines ultimately translate fine?
The last few years promise new breakthroughs using a neural machine translation. Neural networks themselves construct transformation rules. It is likely that in the near future the neural MT systems will translate better than an average translator. In 2018 Vilnius University is preparing to launch the EU-funded project of a new generation neural machine translation of English, Lithuanian, Polish, French, Russian and German.
Thus, the latest achievements in machine translation apply to the Lithuanian language as well.
