The Effectiveness of the 'REPLIKA' Application Compared to Visual Media in Reading Learning

INTERNATIONAL JOURNAL OF MULTIDISCIPLINARY RESEARCH AND ANALYSIS ISSN(print): 2643-9840, ISSN(online): 2643-9875 Volume 08 Issue 09 September 2025 DOI: 10.47191/ijmra/v8-i09-30, Impact Factor: 8.266 Page No. 5182-5188 IJMRA, Volume 08 Issue 09 September 2025 www.ijmra.in Page 5182 The Effectiveness of the 'REPLIKA' Application Compared to Visual Media in Reading Learning Natasya Veronika1, Margana2, Yuyun Yulia3 1,2,3English Language Education Study Programs, Faculty of Languages, Arts, and Cultures, Yogyakarta State University, Indonesia ABSTRACT: This study investigates the effectiveness of the REPLIKA application, an AI-powered chatbot, in enhancing second language (L2) reading learning compared to traditional visual media. The integration of artificial intelligence in language learning offers personalized, interactive engagement, yet empirical studies comparing its impact against conventional instructional tools remain limited. This quasi-experimental research employed a nonequivalent control group pre-test and post-test design involving 38 fourth-semester English Education students at Yogyakarta State University. Participants were divided into an experimental group (REPLIKA) and a control group (visual media). Reading learning was assessed using the Critical Reading Skills Test (CRST), validated and tested for reliability through a pilot study with third-year students. Data analysis was conducted using SPSS, including descriptive statistics, normality and homogeneity tests, and independent samples t-tests. The findings revealed a significant difference between the two groups, with the experimental group showing greater improvement. The REPLIKA group’s mean score increased by 21.95 points, compared to 8.52 points in the visual media group. These results suggest that the REPLIKA application is more effective in supporting students' L2 reading skill, likely due to its adaptive features, immediate feedback, and interactive learning experience. The study highlights the potential of AI-based applications in addressing individual learning needs and enhancing reading outcomes in higher education contexts. KEYWORDS: REPLIKA, artificial intelligence, reading skill, visual media, language learning I. INTRODUCTION In today’s increasingly globalized world, second language (L2) proficiency—particularly in reading skill—plays a vital role in both academic success and professional development. Reading is not only central to knowledge acquisition but also foundational for developing vocabulary, critical thinking, and disciplinary literacy (Grabe, 2009). Traditionally, reading instruction in higher education has relied heavily on teacher-directed methods and static learning resources such as printed texts or visual media. While these conventional tools, including projected academic texts, offer benefits in terms of content delivery and classroom interaction, they often lack adaptability and individualized feedback, which are crucial for effective skill development in diverse student populations (Nation, 2022; Ahmadi, 2017). Technological advancements, particularly the integration of artificial intelligence (AI) in education, have opened new avenues for improving language learning. One such development is the emergence of Mobile-Assisted Language Learning (MALL) tools, including AI-based chatbots like REPLIKA, that provide learners with flexible, personalized, and interactive learning experiences. REPLIKA simulates real-world dialogues, provides contextually relevant prompts, and offers feedback in real-time, potentially improving students’ microand macro-skill skills (Stockwell, 2013; Godwin-Jones, 2017). Moreover, based on Vygotsky’s (1978) social constructivist theory, such interactional tools can enhance L2 acquisition by supporting learner autonomy, fostering motivation, and encouraging meaningful engagement with texts. While conventional visual media such as projected scientific articles support collective text analysis in class and help visualize structural components of language (Kress & van Leeuwen, 2021), they are generally limited in promoting individualized learning. Visual media often fail to provide immediate corrective feedback or support self-paced learning, which can result in passive engagement among students, particularly those with lower proficiency levels. In contrast, AI-powered tools like REPLIKA can adapt to learners’ needs by offering tailored prompts and enabling repeated exposure to content, making it easier to process complex structures and enhance retention (Clayton & Saravani, 2014). The Effectiveness of the 'REPLIKA' Application Compared to Visual Media in Reading Learning IJMRA, Volume 08 Issue 09 September 2025 www.ijmra.in Page 5183 Despite the increasing adoption of educational technology, there remains a lack of empirical evidence comparing the effectiveness of AI-based applications and traditional instructional tools in L2 reading learning, especially in the Indonesian tertiary education context. Existing studies often focus on the effectiveness of AI in writing, vocabulary acquisition, or speaking skills (Etaat, 2024; Song & Song, 2023), leaving a gap in understanding how AI can influence reading performance. Furthermore, while some research has explored the benefits of tools like Duolingo and HelloTalk, few have directly evaluated the REPLIKA application or its comparative effectiveness against classroom-based visual media. This gap leads to an incomplete picture of how different digital platforms influence learner outcomes in reading skill. This study seeks to address that gap by evaluating whether REPLIKA is more effective than visual media in supporting university students’ L2 reading learning. The focus is on students in the fourth semester of the English Education program at Yogyakarta State University, who are at a critical stage of academic transition and are expected to engage with increasingly complex English texts. These learners often face difficulties in decoding complex academic language due to limited exposure to interactive learning tools and a lack of motivation to engage with passive materials (Meniado, 2016; Nation, 2022). By introducing AI-based engagement through REPLIKA, this study examines whether personalized interaction and contextual feedback can improve their reading skills more effectively than teacher-guided use of visual media.The research specifically compares students’ performance before and after the intervention using the Critical Reading Skills Test (CRST) as an instrument. The CRST assesses both microand macroskills and has been validated for use in L2 academic reading studies. The comparison focuses not only on overall reading improvement but also on the relative effectiveness of each instructional medium in delivering measurable learning gains. The findings of this research are expected to provide empirical insights into the potential of AI-based applications in academic reading instruction. This is particularly important for educators seeking to diversify their teaching strategies and integrate digital tools that align with learners’ needs and preferences. Moreover, the study contributes to the broader discourse on technology-enhanced language learning by offering practical evidence to guide curriculum design, teacher training, and the selection of appropriate instructional technologies for higher education. Second Language Acquisition and Reading Skill Second Language Acquisition (SLA) involves the process by which individuals acquire a language other than their mother tongue, incorporating the development of skills such as listening, speaking, reading, and writing (Ellis, 1989). In the context of L2 learning, reading is considered fundamental, as it not only enhances vocabulary and grammar awareness but also supports overall academic success (Grabe, 2009). Micro-skill refers to the understanding of vocabulary, grammar, and meaning at the sentence level, while macro-skill refers to recognizing the main ideas, logical relations, and overall purpose of a text (Koda, 2005; Snow, 2002). Technology-Enhanced Language Learning (TELL) Code-switching refers to the practice of alternately using two or more languages or dialects within a single conversation or discourse. In bilingual or multilingual settings, code-switching occurs naturally as individuals switch between languages to express thoughts, manage communication, and achieve conversational goals. Linguists have identified different types of codeswitching based on the structural and contextual features of the language alternation. The three most prominent types of codeswitching are inter-sentential code-switching, intra-sentential code-switching, and tag-switching, each serving unique purposes and involving distinct linguistic processes. Mobile-Assisted Language Learning (MALL) MALL, a subset of TELL, leverages smartphones and portable devices to deliver language content, allowing for learning that is flexible and accessible anytime and anywhere (Kukulska-Hulme & Shield, 2008). The advantages of MALL include the personalization of content, interactive engagement, and the development of learner autonomy. Applications integrated with AI— such as chatbots and adaptive platforms—further enhance these benefits by offering real-time dialogue, contextual support, and feedback mechanisms (Stockwell & Hubbard, 2013; Godwin-Jones, 2017). Artificial Intelligence in Language Learning Artificial Intelligence (AI) has gained increasing attention in language education for its ability to simulate human-like interaction, tailor feedback, and provide personalized learning paths. AI-powered applications can facilitate conversation practice, reading skill, vocabulary development, and even writing feedback (Ayedoun et al., 2019; Lin & Chang, 2020). AI tools, particularly chatbots, are grounded in social constructivist theory, which emphasizes learner engagement through meaningful interaction (Vygotsky, 1978; Wegerif, 2004). The Effectiveness of the 'REPLIKA' Application Compared to Visual Media in Reading Learning IJMRA, Volume 08 Issue 09 September 2025 www.ijmra.in Page 5184 However, research also points to challenges in AI implementation, such as the lack of emotional engagement and concerns about artificial interaction (Annamalai et al., 2023). Additionally, some studies highlight that learners with higher proficiency levels may find chatbots less satisfying compared to low-proficiency learners (Qinghua & Satar, 2020). Therefore, understanding AI’s impact requires contextual evaluation, particularly when used in formal educational settings. REPLIKA as an AI Chatbot for L2 Learning REPLIKA is an AI chatbot originally designed for mental health companionship but has since been adopted in educational contexts due to its interactive conversational features (Wei, 2023). It supports language learners by offering simulated conversations, contextual prompts, and adaptive feedback. In language learning, REPLIKA can promote reading skills through AIgenerated texts and discussion threads that help students infer meaning, analyze sentence structures, and learn vocabulary in context (Godwin-Jones, 2017). Additionally, its availability 24/7 supports autonomous learning outside of formal classroom instruction. In higher education settings, REPLIKA has the potential to complement reading instruction by allowing learners to engage in guided reflection, summary writing, and skill tasks based on chatbot conversations. Its adaptive features enable it to adjust the complexity of input according to the user's language proficiency, thereby aligning with differentiated instruction principles (Stockwell, 2013). Visual Media in Reading Instruction Visual media, such as projected PowerPoint slides, scientific texts, and infographics, remain a staple in academic reading instruction. These tools provide structure, highlight key vocabulary, and allow teachers to model reading strategies in real-time (Mayer, 2020; Kress & van Leeuwen, 2021). Authentic academic texts projected in class expose students to complex syntax and discipline-specific vocabulary, which are essential for developing academic reading skills (Amalia, 2016). The dual-coding theory supports the use of visuals and verbal input to enhance skill (W. Li et al., 2022). Despite their advantages, visual media often lack the interactive and personalized aspects necessary to meet diverse learner needs. Passive reliance on teacher explanations and limited real-time feedback can hinder the development of deeper skill, especially for students who require individualized scaffolding (Nation, 2022). Previous Studies Several prior studies have explored the impact of AI tools on language skills. Etaat (2024) examined the use of Wordtune and InstaText for EFL writing development and reported improved mechanics and time efficiency. Song and Song (2023) found that ChatGPT improved academic writing skills and motivation among EFL learners. Aulia (2020) showed that Duolingo significantly enhanced vocabulary mastery. Zulaikah (2024) demonstrated that HelloTalk improved reading ability in narrative texts, and Nushi & Shirvani (2021) reviewed Speaky as effective for enhancing speaking through peer language exchange. However, none of these studies directly examined REPLIKA's effectiveness for reading skill or compared it to traditional visual media in an academic setting. This research seeks to fill that gap by providing empirical evidence from an Indonesian university context. II. METHOD Research Design This study employed a quasi-experimental design using a nonequivalent control group with pre-test and post-test. This design was selected to determine the comparative effectiveness of the REPLIKA application and visual media in improving students' reading skill in English as a second language (L2). The experimental group received instruction using the REPLIKA AI chatbot, while the control group used conventional visual media—specifically, projected scientific texts delivered via PowerPoint. The independent variable in this research was the learning media (REPLIKA vs. visual media), and the dependent variable was students’ L2 reading skill scores. The pre-test and post-test design allowed the researcher to measure the improvement of reading skills over the course of the treatment period and assess whether significant differences occurred between the two groups. Research Setting and Participants The study was conducted at the English Education Study Program of Yogyakarta State University in the even semester of the 2024/2025 academic year. The participants were fourth-semester students enrolled in a reading-related course. A purposive sampling technique was used to select two intact classes that had relatively similar levels of proficiency and course background. A total of 38 students participated in the study. The experimental group, consisting of 19 students, was exposed to the REPLIKA application as a learning tool. The control group, also consisting of 19 students, used visual media in the form of projected academic texts. These two classes were taught by the same instructor to control for teacher-related variables. The Effectiveness of the 'REPLIKA' Application Compared to Visual Media in Reading Learning IJMRA, Volume 08 Issue 09 September 2025 www.ijmra.in Page 5185 Instrumentation Reading skill was measured using the Critical Reading Skills Test (CRST). The test was developed based on critical reading indicators such as identifying the main idea, making inferences, recognizing the author’s purpose, and distinguishing facts from opinions. It consisted of 40 multiple-choice items and was administered during both the pre-test and post-test sessions. Before its use, the CRST underwent a pilot study involving 30 third-year students. This trial ensured the clarity of instructions, appropriateness of text length, and alignment with the test objectives. The results showed that the test was both valid and reliable: 1. Construct validity was assessed using Pearson’s correlation between each item and the total score, yielding correlation coefficients ranging from 0.395 to 0.607 at the 0.01 significance level. 2. Reliability was established through the test-retest method, resulting in a correlation coefficient of 0.573 at the 0.01 significance level, indicating a satisfactory level of stability over time. Procedure Both groups received instruction for several weeks, following distinct learning procedures: 1. The experimental group interacted with the REPLIKA app based on guided prompts designed to elicit responses related to academic reading topics. Students were trained to use the chatbot and instructed to maintain topic relevance. After each session, they completed reflection activities such as reading summaries or skill logs. 2. The control group learned through teacher-led instruction using visual media, including PowerPoint slides displaying scientific texts. The teacher guided the students through close reading, vocabulary exploration, and critical thinking activities. The learning material was kept parallel across both groups to ensure that the primary difference lay in the medium of delivery rather than content. Data Analysis The data were analyzed using SPSS software in the following steps: 1. Descriptive statistics were used to calculate means and standard deviations for pre-test and post-test scores. 2. Normality tests were conducted using the Kolmogorov-Smirnov and Shapiro-Wilk methods to determine whether the data were normally distributed. 3. Homogeneity tests assessed whether the variance between groups was similar, which is a requirement for valid comparison. 4. Effectiveness tests (paired sample t-tests) were used to measure within-group gains from pre-test to post-test. 5. Independent samples t-tests were employed to compare post-test scores between the experimental and control groups. All statistical tests were conducted at the 0.05 significance level. III. FINDING AND DISCUSSION Descriptive Statistics Data Descriptive Control Pre Test Control Post Test Experimental Pre Test Experimental Post Test Mean 92.11 100.63 91.26 113.21 N 19 19 19 19 Std. Deviation 3.943 3.235 4.175 5.181 Minimum 85 96 85 96 Maximum 98 105 98 120 To examine the impact of the REPLIKA application and visual media on students’ L2 reading skill, descriptive statistics were computed for both groups. The control group, which used visual media, had a pre-test mean score of 92.11 (SD = 3.94), which increased to 100.63 (SD = 3.24) in the post-test, resulting in a mean gain of 8.52 points. In contrast, the experimental group using the REPLIKA app had a pre-test mean of 91.26 (SD = 4.18) and a post-test mean of 113.21 (SD = 5.18), with a mean gain of 21.95 points. These results suggest a greater improvement in the experimental group, indicating that REPLIKA may be more effective than visual media in enhancing reading skill. The Effectiveness of the 'REPLIKA' Application Compared to Visual Media in Reading Learning IJMRA, Volume 08 Issue 09 September 2025 www.ijmra.in Page 5186 Assumption Tests Prior to inferential analysis, normality and homogeneity tests were conducted to validate the assumptions of the t-test. 1. The normality test using the Kolmogorov-Smirnov and Shapiro-Wilk methods showed that data for both groups were normally distributed (p > 0.05). 2. The homogeneity of variance test indicated equal variance across groups (p > 0.05), supporting the use of parametric testing. These results confirmed the suitability of t-tests for further statistical analysis. Effectiveness Test (Paired Samples t-Test) A paired samples t-test was performed to evaluate the within-group improvement from pre-test to post-test. 1. For the control group, the improvement was statistically significant (t(18) = -14.03, p = 0.000), confirming the positive impact of visual media on students’ reading skill. 2. For the experimental group, the improvement was also statistically significant (t(18) = -15.92, p = 0.000), but the magnitude of gain was considerably higher. This confirms that both instructional methods were effective, but REPLIKA yielded stronger results. Comparison Test (Independent Samples t-Test) To compare the post-test scores between the two groups, an independent samples t-test was conducted. 1. The result showed a Sig. (2-tailed) = 0.000, which is below the alpha level of 0.05. 2. This indicates a statistically significant difference between the control and experimental groups in favor of the REPLIKA group. These findings support the hypothesis that the REPLIKA application is more effective than traditional visual media in improving students’ L2 reading skill. DISCUSSION The findings of this study are consistent with the emerging body of literature supporting the role of AI-based applications in second language learning. The REPLIKA application demonstrated higher effectiveness compared to visual media, as shown by the larger gain in reading scores. Several factors may explain this result. Effectiveness test Sig. (2-tailed) Control Pre - Control Post 0.000 Experimental Pre - Experimental Post 0.000 The Effectiveness of the 'REPLIKA' Application Compared to Visual Media in Reading Learning IJMRA, Volume 08 Issue 09 September 2025 www.ijmra.in Page 5187 First, REPLIKA offers personalized, interactive dialogue that allows learners to engage with language in a meaningful and self-paced manner. Unlike teacher-led visual instruction, which relies on group-based delivery and limited individual feedback, REPLIKA responds to each learner’s input in real-time. This aligns with theories of social constructivism (Vygotsky, 1978) and adaptive learning (Godwin-Jones, 2017), which emphasize that learning is more effective when learners are active participants in constructing meaning. Second, the 24/7 accessibility of REPLIKA enabled students to engage in independent learning beyond classroom hours. This may have led to increased reading exposure, practice time, and self-directed learning habits, which are critical for L2 reading development (Nation, 2022). Third, REPLIKA promotes both microand macro-skill. Through dialogue-based interaction, students could decode sentence-level information (micro-skill) and also explore broader discourse structures and meanings (macro-skill). The chatbot’s ability to prompt critical thinking and generate questions may have supported students in applying reading strategies such as inference-making and main idea identification (Grabe, 2009; Koda, 2005). In contrast, while visual media is also beneficial—especially for displaying authentic texts and supporting teacher modeling—it lacks personalization. Visual media are limited in adapting to the proficiency levels of each learner, which may contribute to a plateau in skill gains for more advanced students or confusion among lower-proficiency students (Ahmadi, 2017). Moreover, projected visual media often encourage passive engagement, with students relying on teacher explanations rather than constructing meaning themselves. This limits the opportunity for repeated exposure and reflection, both of which are essential for deeper skill (Mayer, 2020). The statistically significant difference between the two groups aligns with previous studies on AI in education. For example, Song & Song (2023) found that AI-assisted writing improved student motivation and language outcomes, while Etaat (2024) confirmed the benefits of AI feedback in enhancing writing efficiency. However, this study extends those findings to reading skill, an area less frequently explored in the context of chatbot-based learning. It is worth noting that this study also provides important pedagogical implications. Educators may consider integrating AI tools such as REPLIKA as supplementary resources in reading instruction. The combination of traditional classroom methods and AI-driven interaction can create a more balanced and personalized learning experience. Limitations While the results are promising, several limitations must be acknowledged: 1. Limited sample size: With only 38 participants, the findings cannot be generalized across larger populations without further replication. 2. Short treatment duration: The intervention was conducted over a limited time frame, which may not capture the longterm effects of AI-based instruction. 3. Restricted use of REPLIKA: Students were instructed to stay on academic topics, limiting the full exploratory potential of the chatbot's capabilities. 4. Lack of qualitative data: This study relied solely on test scores, and did not explore learners’ experiences, perceptions, or engagement levels through interviews or surveys. Future studies could address these limitations by employing mixed-methods research, expanding participant numbers, and examining long-term language retention. IV. CONCLUSION AND IMPLICATIONS This study explored the effectiveness of the REPLIKA AI chatbot in comparison to traditional visual media in improving L2 reading skill among university students. The findings demonstrated that both instructional methods led to significant improvements in reading performance; however, students who engaged with the REPLIKA application experienced a significantly greater gain. The REPLIKA group achieved a mean improvement of 21.95 points compared to 8.52 points in the control group, as confirmed by statistical analysis (p < 0.05). The results support the growing body of evidence that AI-powered tools can serve as effective learning platforms in language education. REPLIKA's personalized feedback, interactive dialogue, and 24/7 accessibility likely contributed to higher learner engagement and skill outcomes. The tool allowed students to develop both microand macro-skill by simulating meaningful interactions and encouraging reflective reading practices. These findings have important pedagogical implications. Language educators are encouraged to integrate AI-based applications like REPLIKA into their instruction to complement traditional classroom practices. Such tools may help address individual learning needs, foster motivation, and promote autonomous learning—particularly in reading-intensive academic contexts. The Effectiveness of the 'REPLIKA' Application Compared to Visual Media in Reading Learning IJMRA, Volume 08 Issue 09 September 2025 www.ijmra.in Page 5188 Nevertheless, given the limited scope of this study, future research is needed to explore long-term effects, learner perceptions, and the integration of AI across different language skills. Expanding the sample size, length of intervention, and inclusion of qualitative data may further enrich our understanding of AI’s role in L2 acquisition. REFERENCES 1) Ahmadi, M. R. (2017). The impact of motivation on reading comprehension. International Journal of Research in English Education, 2(1), 1–7. https://doi.org/10.29252/ijree.2.1.1 2) Amalia, R. (2016). Authentic materials in EFL classrooms: Their effects on students’ academic reading comprehension. Journal of English Language Teaching and Linguistics, 1(2), 1–11. 3) Annamalai, N., Zuraidah, M. D., & Zulkifli, M. (2023). Push and pull factors in chatbot use for English learning: The mediating role of user habit. Computer Assisted Language Learning, 36(1-2), 172–196. https://doi.org/10.1080/09588221.2022.2122757 4) Ayedoun, E., Hayashi, Y., & Seta, K. (2019). A chatbot for practicing affective and listening skills in foreign language learning. International Journal of Artificial Intelligence in Education, 29(4), 527–547. 5) Bartsch, R. A., & Cobern, K. M. (2003). Effectiveness of PowerPoint presentations in lectures. Computers & Education, 41(1), 77–86. 6) Bitchener, J., & Ferris, D. (2012). Written corrective feedback in second language acquisition and writing. Routledge. 7) Cain, K., & Oakhill, J. (2007). Children’s comprehension problems in oral and written language: A cognitive perspective. Guilford Press. 8) Chau, J., & Lee, A. (2014). Technology-enhanced language learning: A study of student experiences. The Journal of Asia TEFL, 11(4), 85–113. 9) Clayton, J., & Saravani, S. (2014). Digital learners: The role of mobile devices in language learning. In Learning and Collaboration Technologies, 372–381. Springer. 10) Dörnyei, Z. (2005). The psychology of the language learner: Individual differences in second language acquisition. Routledge. There is an Open Access article, distributed under the term of the Creative Commons Attribution – Non Commercial 4.0 International (CC BY-NC 4.0) (https://creativecommons.org/licenses/by-nc/4.0/), which permits remixing, adapting and building upon the work for non-commercial use, provided the original work is properly cited.