Aspects of creativity in AI-powered Polish-to-English literary translation

SKASE Journal of Translation and Interpretation, 2025; 18(3): 112–126 doi: 10.33542/JTI2025-2-7 112 Aspects of creativity in AI-powered Polish-to-English literary translation Małgorzata Kodura, University of the National Education Commission Abstract This research paper explores the capabilities and limitations of various AI models in translating Polish literary texts into English. The study examines the impact of different prompt settings on translation output, focusing on the creativity, accuracy, and correctness of AI-generated texts. The aspect of creativity in the generated texts is analysed through the application of creative shifts as proposed by Bayer-Hohenwarter. By evaluating translations generated by diverse AI models with varied prompts, this research aims to provide insights into the role of AI in literary translation and its potential to bridge linguistic and cultural gaps. The paper highlights the potential challenges and considerations associated with augmented creativity, reviews the current state of research, and offers recommendations for further studies in this domain. Keywords: translation creativity, artificial intelligence, creative shifts, machine translation, literary translation 1. Introduction The advent of advanced artificial intelligence (AI) models, particularly large language models (LLMs), has led to significant advancements in the field of machine translation. However, the translation of literary texts – rich in stylistic nuances, cultural references, and subtle meanings – presents a unique challenge. This research aims to explore the capabilities and limitations of various AI models in translating Polish literary texts into English, with a particular focus on the impact of different prompt settings on the translations produced, their creative solutions, translation accuracy, and the preservation of source text imagery. Through this investigation, the study seeks to deepen our understanding of AI’s role in literary translation and assess its potential to bridge linguistic and cultural divides. Artificial intelligence is increasingly being utilised across various fields, including translation. While AI translation tools, particularly those based on machine translation (MT), have made considerable progress, concerns remain about their ability to capture the subtleties and intricacies of literary language. Nevertheless, translators are actively exploring and incorporating AI tools into their workflows, seeking innovative ways to leverage these technologies. Recent developments in AI-powered translation systems, notably neural machine translation (NMT), have ignited discussions within translation studies about the concept of creativity in machine-generated translations. Although AI models have shown remarkable proficiency in producing fluent and accurate translations, their ability to generate truly creative output remains a subject of ongoing research and debate. Małgorzata Kodura 113 2. Literature review Early investigations by Guerberof Arenas & Toral (2020) explored the notion of creativity in machine-generated translations of the story Murder in the Mall. The study defined creativity as a blend of novelty and acceptability, comparing the outcomes of machine translation, postediting of machine translations, and human translations. The results consistently showed that human translations attained the highest creativity scores, while raw MT scored the lowest. This suggests that NMT systems, even when trained on literary data, lack the ability to produce creative translations and tend to favour literal solutions. Moreover, the study revealed that using MT as a foundation for post-editing can potentially constrain the creativity of human translators. Post-edited translations, compared to those produced solely by humans, consistently exhibited a lower degree of creativity. This was evident in a reduced tendency to deviate from literal renderings and a diminished ability to handle units of creative potential effectively. While these distinctions may seem minor, they are nonetheless significant, as subtle elements such as voice, rhythm, and style are precisely what elevate a translation from merely functional to truly distinctive. Toral & Way (2018: 263) investigated the quality of NMT on literary text, which they claim is “the greatest challenge for MT”. They compared the performance of NMT with the previous dominant paradigm, phrase-based statistical machine translation (PBSMT), using English-to-Catalan translations of novels. The researchers first trained both NMT and PBSMT systems with over 100 million words of literary texts and evaluated the resulting translation with both automatic evaluation metric (BLEU) and using human evaluation. They concluded that NMT provided a viable solution for literary translation and proposed that future work should focus on integrating NMT into professional literary translation through post-editing. Toral and Way also observed that the difficulty of training MT systems on a vast amount of bilingual parallel literary text was a challenge before the advent of e-books. The wide availability of e-books in digital format created an opportunity to build MT systems specifically for novels, which in turn resulted in better performance. Subsequent research by Guerberof Arenas & Toral (2022) further corroborated the finding that human translation generally exhibits greater creativity than MT or post-editing. This study highlighted the crucial role of human intervention in navigating cultural contexts and making subjective choices, both vital components of creative translation. More recently, Jiménez-Crespo (2024) explored the influence of AI and large language models on the translation landscape. While AI can generate novel content, Jiménez-Crespo (2024: 319) emphasised that human-generated texts are perceived as more creative, since creativity in translation involves processes richer than just creating “never-before-seen content”. This highlights the continued importance of human expertise in creative endeavours such as translation, advocating for a human-centred AI approach that augments human capabilities rather than replacing them, particularly in creative fields. In contrast, a study carried out by Hu & Li (2023) showcased the creative potential of the AI-powered translation system DeepL in translating Shakespearean plays into Chinese. DeepL achieved over 80% accuracy and fluency, demonstrating its capability to handle complex texts across languages. Furthermore, it exhibited creativity through techniques such as adding words for clarity, shifting perspectives, and employing specific Chinese grammatical structures. However, identified accuracy and fluency errors, including overly literal translations, indicated that AI is not yet ready to fully replace human translators. It SKASE Journal of Translation and Interpretation 114 should also be noted that the researchers did not include the analysis of the Shakespearean verse in their study. At the same time, Wang (2023) investigated the increasing application of AI-powered machine translation, highlighting the unique characteristics, complexity, and cultural significance of literary translation – often overlooked amidst the advancements in MT. Employing sociological theories, Wang identifies three key challenges in applying MT to literary texts: the deeply ingrained habits and skills translators develop over time, a complex network of human and non-human actors, and the cultural and ethical implications of MT in literary contexts. Wang argues that current MT systems merely simulate human translation, potentially undermining the processes of meaning generation and cultural knowledge production inherent in literary translation. The study concludes that until MT can be proven both effective and safe for literary texts, human translators remain indispensable for managing the complexities and nuances in those texts. Łoboda & Mastela (2023) observed in their study that although it is believed that in some areas, e.g. translating for EU institutions, neural machine translation engines can reach surprisingly high-quality scores, their use in literary texts is a controversial subject. Łoboda and Mastela suggest introducing post-editing exercises into translator education to enhance students’ cultural sensitivity and to make them aware of the limitations of machine translation. The authors propose that such exercises, particularly with culture-bound texts, can serve as an innovative way to develop students’ technological, linguistic, and cultural competencies. By working with texts rich in cultural elements, students learn to identify when a machine is likely to fail. The study found that students with prior training were more critical of machine translation output and better at detecting and correcting errors, demonstrating the value of this pedagogical approach. Recent research offers diverging perspectives on the role of AI in creative translation. For example, Gao et al. (2024) explored the creative potential of artificial intelligence in translating Chinese poetry into English. They observed that ChatGPT could generate novel combinations of words and phrases, capturing the essence of the original poems while offering fresh interpretations. However, fidelity issues persisted, and ChatGPT struggled to apply its understanding of specific phrases consistently during translations. Conversely, Runco (2023) presents a contrasting viewpoint, contending that AI cannot exhibit true creativity. He argues that while AI outputs may appear original and effective, they lack key characteristics of human creativity, such as self-actualisation and emergent thought. Runco proposes viewing AI output as a form of pseudo-creativity. Bridging these perspectives, Škobo (2023) examined the integration of AI in literary translation, finding that while AI offers benefits such as time-saving and improved consistency, it struggles with the nuances, metaphors, and figurative language intrinsic to literary works. The study highlights that human expertise, creativity, and critical thinking are essential for capturing the artistic value of the original text. The impact of AI on translation, particularly in the literary domain, is a complex and multifaceted issue. While AI demonstrates impressive capabilities in generating fluent and accurate translations, its capacity for true creativity remains a subject of debate. Human expertise, with its nuanced understanding of language and culture, continues to play a vital role in preserving the artistic value and essence of literary works in translation. However, the increasing prevalence of inexpensive and easily accessible machine translations is increasing its share of the market, with publishers offering low rates to translators for post-editing machine-translated texts, raising ethical concerns regarding transparency and fair Małgorzata Kodura 115 compensation. As Li (2023) notes, the increasing use of AI in translation challenges the professional identity of translators, downplaying the human contribution to a supporting role. Li highlights that this trend directly impacts translation quality, as AI systems are prone to biases derived from their training data, which is based on statistics rather than personal understanding. These biases include issues related to gender, race and nationality. While some companies like Nuanxed and China Literature openly embrace AI in translation, using human collaborators to edit machine-translated books, ethical questions persist. Under international intellectual property law, a translation is considered a derivative work, but an ethical question emerges whether an AI translation can have authorship. Li (2023: 537) refers to the US Supreme Court case Naruto v. Slater, which established that a non-human individual cannot claim copyright ownership, thus an AI translator is merely a tool. Another issue mentioned by Li (2023: 538) in the context of ethics is the AI retranslations of existing works, as they integrate the language expressions of previous human translations. Those issues, not properly managed by translators’ associations so far, need to be addressed more systematically, especially in translator education. Fortunately, the use of AI translation is not commonly encouraged, for instance platforms like Amazon KDP advise against the use of AI for literary content, emphasizing its potential to damage an author’s reputation (Albarino 2023). In the realm of machine translation, LLMs like ChatGPT-4 – while not specifically designed for translation – are increasingly being utilised for this purpose due to their extensive training data encompassing various languages. Although specialised machine translation systems still outperform LLMs in terms of translation quality, particularly for specialised texts, LLMs offer a valuable complementary role, providing faster initial translations or suggestions for professional translators. In the context of literary translation, LLMs face challenges in capturing the nuances and complexities inherent in literary language. While AI excels at translating everyday language, the subtle wordplay and deeper meanings often found in literature can be lost in AI-generated translations. Advancements in NMT, which utilises artificial neural networks to make contextual connections between words and phrases, offer promising avenues for improvement, initiating a growing trend of employing AI in the domain of literary translation. These experiments serve not only to demonstrate the evolving capabilities of machines, but also to actively challenge the perceived boundaries of AI’s potential in handling complex literary forms. A notable example of AI’s expanding role in literary translation is a recent initiative undertaken by a publishing house from Kraków, which launched the “Conceptual Line” series with a unique translation of Alfred Jarry’s play Ubu Roi, generated entirely by Google Translate. This attempt, led by Aleksandra Małecka and Piotr Marecki (2018), was not aimed at creating a flawless translation but rather at conducting an experiment in conceptual art. They intentionally selected Ubu Roi, a play renowned for its absurdity and its goal of creating theatre without theatre. By employing machine translation, they sought a “translation without translation”, embracing the awkwardness, nonsensical phrases, and errors produced by the algorithm as an extension of Jarry’s spirit of absurdity. Another project employed Chat GPT-4 to translate the play from French to Polish, with prompts engineered by Jan K. Argasiński (Argasiński & Marecki 2024). The goal was to produce a simplified translation, followed by a summary, a dramatic retelling, and a prosaic adaptation infused with modern youth slang and profanity. The project, showcasing both the potential and limitations of AI in capturing the essence of the play while creatively adapting its language, reached its apex with the publication of UBU GPT (Jarry 2024), a physical book presenting these AI-generated interpretations. This experiment followed the concept of uncreative SKASE Journal of Translation and Interpretation 116 writing, challenging traditional notions of creativity and authorship, where text is treated as data to be manipulated and transformed. It highlighted AI’s potential in literary studies and creative practices by generating new insights and interpretations of classic works, positioning translation within the realm of conceptual art. The idea was not to replace human translators but to broaden the creative possibilities of translation and encourage reflection on technology’s role in artistic expression, with AI’s transformative potential used to generate new forms of literary expression. In a similar vein, the experiment described in this paper aims to examine the possibilities and limitations of the most advanced technology when applied to a specific literary genre, and to verify whether and to what extent currently available AI tools are capable of replicating the nuanced understanding and resourcefulness of contemporary literary texts. Using the framework of creative shifts proposed by Bayer-Hohenwarter (2009, 2010, 2011), the study poses the question whether the output produced by modern large language models can be regarded as creative and comparable to human creative translations. 3. The source material The source material selected for this study is a fragment from Love Song, a book by Karolina Krasny – a young writer, poet, and designer from Kraków – published in 2024 by Ha!Art, the same publishing house that released the ChatGPT-generated UBU GPT. According to the publisher’s description, Love Song is a narrative that explores the transformative power of love on its surroundings. The two protagonists exist in a state of correspondence with nature, their community, and the urban fabric they inhabit. Reviewers have compared Krasny’s poetic text to cubist paintings, with characters as letters and relationships as impossible figures. The fragment selected for the study, titled “Field Wedding”, describes a wedding ceremony where the main characters, Zet and XY (the bride and groom) feel alienated and overwhelmed by the formality and expectations of the event. The text features a distinctive style characterised by a skilful blending of vivid imagery and a sense of detachment, creating a unique, surreal atmosphere. The author frequently uses figurative language, unconventional metaphors and similes, enriching the text and adding layers of meaning and emotional depths. The couple is likened to metal, their vows to steel, and their shrinking bodies to fabric folding in on itself. Sensory details and visual and tactile imagery immerse the reader in the scene, with the hard and tense surroundings, cold or hot metal, and the scent of perfume mingling with the buzzing of flies. Apart from figurative language and sensory descriptions, other challenges awaiting the translator of this text include the blending of formal and informal language, as well as the occasional use of fragmented sentences and ellipses, all of which contribute to the dreamlike atmosphere of the text. The text is characterised by a distinctive contrast between mundane elements and surreal imagery. This juxtaposition is evident in the author’s lexical choices, blending colloquialisms with poetic language and creating tension between the ordinary and the extraordinary. Sudden shifts in perspective, particularly the transition to first-person narration in cudzy oddech był tym samym, co mój (lit. ‘someone else’s breath was the same as mine’), present a challenge in maintaining coherence and narrative flow in the target language. The translator must decide whether to preserve this shift or to normalise it for the sake of consistency in English. Małgorzata Kodura 117 4. AI tools and methods The study, conducted in December 2024, used three selected AI models (ChatGPT 4o created by OpenAI, Claude 3.5 Sonet built by Anthropic, and Gemini Advanced 1.5 Pro offered by Google AI) and three different prompts to generate multiple English translations of this text. The prompts were designed specifically to obtain responses corresponding to three levels of the temperature parameter settings. In AI-generated texts, temperature settings play a pivotal role in controlling the balance between creativity and coherence. The temperature parameter in LLMs is intrinsically linked to their capacity for creative output, particularly in the realm of narrative generation. By controlling the randomness of the LLM’s output, temperature influences the diversity and unpredictability of the generated text (Bhupen 2024). Higher temperature values encourage the model to explore less probable options, leading to more diverse and imaginative outputs. Conversely, lower temperature values constrain the model to favour the most likely tokens, resulting in more focused and predictable text. The choice of temperature setting depends on the specific application and the desired outcome. While higher temperatures stimulate novelty and creative expression, they also increase the risk of generating incoherent or nonsensical responses. Therefore, it is crucial to evaluate the generated text for inaccuracies or AI hallucinations, as strategic adjustment of the temperature parameter allows users to control the behaviour of LLMs and achieve desired levels of creativity. However, as it proved infeasible in this study to directly manipulate the temperature in all three AI tools, an intermediate solution was applied to modify the temperature within the prompt itself (Ramlochan 2024). Each model generated three translations based on three distinct prompts designed to simulate varying levels of the temperature parameter: low, medium, and high. The lowtemperature prompt was: “Translate the following text conservatively into English, focusing on literal meaning rather than idiomatic expressions”. The medium-temperature prompt instructed the tool to: “Translate the following text while maintaining a balanced tone – clear and informative but also engaging and natural. Ensure the translation captures the essence of the original text without being too rigid or too free”. Lastly, the high-temperature prompt asked the AI model to: Provide a vivid, expressive translation of the following text. Feel free to use colourful language, idiomatic expressions, and creative phrasing to capture the spirit and mood of the original in a way that resonates strongly in the target language. While preserving the core meaning, prioritise the sound of the target language and follow the melody of the original, do not focus on rhyme in English, do not create a poem, but try to reconstruct the rhythm from the Polish language. These prompts were finalised after preliminary testing on a short sample, which revealed the need to instruct the models explicitly to avoid generating poetry or rhymes, as they were being produced in response to the prompt asking for prioritising the language. All translations were produced using a new chat each time to avoid any references to previous prompts and contexts. The resulting outputs were labelled as Ch1, Ch2, Ch3 (for ChatGPT), Cl1, Cl2, Cl3 (for Claude), G1, G2, G3 (for Gemini), according to the applied prompt. These labels are utilised throughout the subsequent analysis presented in this paper and a literal translation of the Polish text is provided in brackets with each reference to the source text. SKASE Journal of Translation and Interpretation 118 This study employs a mix of quantitative and qualitative methods to analyse the translation data generated by different AI models and prompt settings. The objective is to assess the impact of these variables on translation quality, identify patterns and trends, and draw conclusions regarding the strengths and weaknesses of AI in literary translation. To ensure a comprehensive evaluation, the output translations were assessed based on a set of predefined criteria. First, their correctness was determined by examining grammatical accuracy, appropriateness of vocabulary, and adherence to English language conventions. Second, accuracy was evaluated by assessing the faithfulness of the translation to the original meaning and intent of the Polish source text. Lastly, to evaluate creativity, the study utilised the approach proposed by Bayer-Hohenwarter (2009, 2010, 2011), focusing on the identification of creative shifts such as abstractions, concretisations, and modifications, and their frequency in the analysed texts as indicators of the creative features of the translated text. Furthermore, the study investigated the processing of figurative language, metaphors, and stylistic choices to effectively capture the aesthetic qualities of the original text. This multi-faceted approach was designed to provide valuable insights into the capabilities and limitations of AI in the domain of literary translation. 5. Results To be creative, translation must first and foremost be fit for purpose, i.e. linguistically correct and accurate. Therefore, the initial phase of the analysis involved a careful examination of the generated translations, focusing on their linguistic correctness. Each translation was evaluated independently for the presence of any linguistic errors. The most common issue identified was the use of punctuation, such as the overuse of dashes instead of semicolons or some commas that could have been used differently to ensure better flow. However, this unconventional use of punctuation in some places corresponded to the rhythm of the original text and did not depend on any specific prompt. Major errors were not common in the AIgenerated translations; the only grammar problems concerned the past tense. For example, in two translations generated by Claude (Cl1 and Cl2), the use of tenses followed the unconventional but correct structure of the Polish language, as shown in (1). (1) ST: Naraz zrzucili każdą z możliwych warstw, chwycili się i stoją. (lit. ‘At once they have shed every possible layer, grasped each other and are standing’) Cl1, Cl2: At once, they shed each possible layer, grasped each other and stand. In another fragment, as in (2), instead of applying the expected sequence of tenses, the translation mirrored the form of the source text. (2) ST: Chciał krzyczeć o udach, na których dziś zaśnie. (lit. ‘He wanted to shout about the thighs on which he would fall asleep today’) Cl1: He wanted to shout about the thighs he’ll sleep on today. Małgorzata Kodura 119 Translation versions 1 and 2 generated by Claude were characterised by calques, with several instances of literary translation of individual words that failed to produce correct collocations in the target language or distorted the meaning. For example, sweat spots (meaning ‘heat rash’) was used as a translation of plamki potu (lit. ‘little spots of sweat’) or give slaps was a literal translation of dawać klapsy in Cl1 and Cl2. In most responses generated for prompts 1 and 2, the target sentences followed the structure of the source language, resulting in awkward and unnatural phrasing. For instance: In the field they chose, a circle was mowed and an approach to this circle two meters wide (as a direct equivalent of Na polu, które wybrali, wykoszono koło i dojście do tego koła o szerokości dwóch metrów). This tendency was also evident in the translation of the phrase młoda para (lit. ‘young couple’, meaning ‘the bride and the groom’) in several translations (Cl1, Cl2, Ge2). In this particular context, the translation contradicted the source text, as the bride was an older woman. However, only ChatGPT consistently used the correct phrase the bride and groom in all three translations, at least in the first part of the text. Later in the text, ChatGPT reverted to employing the calque the young couple for subsequent occurrences of młoda para. Regarding the aspect of accuracy, ChatGPT also showed a tendency to alter the name of the character Zet. In two outputs (Ch1 and Ch3), the form Zeta was used, despite the fact that nothing in the source text suggested this variation, as the name consistently appeared in its uninflected form. A grammar issue related to accuracy was observed in the use of tenses in example (3), which refers to the internal monologue of the main character XY. (3) ST: XY nie działa, pomyślał, no tak, to byłoby chore, gdyby XY działał. (lit. ‘XY doesn’t work, he thought, well yes, it would be sick if XY worked’) Individual translations rendered this fragment using the following different structures: Ch1: XY doesn’t work, he thought. Well, of course, it would be insane if XY worked. Ch2, Ch3: XY is malfunctioning, he thought. Of course, it would be twisted if XY worked properly. Cl1: XY doesn’t work, he thought, well, it would be sick if XY worked. Cl2: XY doesn’t work, he thought, well, it would be sick if XY did work. Cl3: XY’s on the fritz, he mused, yeah, it’d be messed up if XY actually worked right. G1, G3: XY doesn’t work, he thought, well, yeah, it would be sick if XY worked. G2: XY doesn’t work, he thought, yeah, it would be sick if XY worked. The choice between the present continuous and present simple in this case does not result in significant changes to the story but reflects the internal state of the main character, who is described using the imagery of a machine (e.g. sweat-like smear, vows of steel, metal surrounding). Using the continuous form (which was surprisingly rare in the AI-generated output) would better correspond to XY’s present condition, emphasising that he is not functioning as he should at that specific moment. SKASE Journal of Translation and Interpretation 120 The creativity of the AI-generated texts was examined through the analysis of creative shifts and non-standard renderings of poetic imagery. The task of measuring creativity has long been considered challenging due to the perceived difficulty of quantifying the inherently subjective quality of a text. Nonetheless, Bayer-Hohenwarter (2009: 45) has proposed criteria for measuring and promoting translational creativity, focusing on the concept of creative shifts. In this approach, creative shifts are cognitive categories contrasting with traditional shift concepts such as addition, change, or omission, which are primarily form-oriented categories. Bayer-Hohenwarter argues that creative shifts involve semantic changes at the level of abstraction, where a translator employs abstraction, modification, or concretisation to achieve novelty. This demonstrates flexibility, a key indicator of creativity in translation. To produce a translation that effectively fulfils its intended purpose, translators often need to move beyond simple word-for-word translations that require less cognitive effort. Instead, they may need to employ more complex strategies involving understanding the deeper meaning of the source text beyond its literal wording. These strategies require significantly more cognitive effort than reproduction, the term used for non-creative translation. It might be expected that AI-generated texts would tend to adhere closely to the source text, resulting in a machine-like quality often described as translationese. However, unlike machine translation tools that typically provide one single version of the translation – albeit with some modifiable elements, as suggested in changes offered by tools like DeepL – AI tools can follow a series of instructions, adapt to the context, and comply with the imposed style as specified in the prompt. Additionally, the nature of the response generated by the tool can be determined by adjusting the temperature setting. Given the inability to use all three tools via an IT interface that allows parameter adjustments, the prompts were carefully structured to ensure responses corresponding to three temperature levels: low, medium, and high. The analysed texts demonstrated the presence of all three types of creative shifts identified by Bayer-Hohenwarter: abstractions, concretisations, and modifications. These shifts are summarised in Table 1, categorised by specific AI models and prompts. Table 1: Creative shifts identified in AI-generated translations Ch1 Ch2 Ch3 Cl1 Cl2 Cl3 G1 G2 G3 Abstractions 1 1 1 Concretisations 3 13 14 43 5 6 6 Modifications 2 6 4 1 16 1 1 2 Total number of creative shifts 5 20 19 0 1 59 7 7 8 The analysis of nine texts generated by three AI tools demonstrated that, although all three types of creative shifts were present in the translations, abstractions – referring to cases where translators resort to solutions that are more general or abstract than the source text meaning – were clearly the least frequent. Abstractions were identified in only two cases: the couple as a translation of młoda para (‘the bride and groom’, lit. ‘young couple’) in Ch 3 and G1; In the chosen area is used as a translation of Na polu, które wybrali (lit. ‘In the field they selected’).