AI based multilingual chatbot: A review on multilingual AI chatbot using transformer

 Corresponding author: Payoshni Sanjay Gade Copyright © 2025 Author(s) retain the copyright of this article. This article is published under the terms of the Creative Commons Attribution License 4.0. AI based multilingual chatbot: A review on multilingual AI chatbot using transformer Ms. Payoshni Sanjay Gade 1, * and Dr. Sheetal S. Dhande 2 1 Research Scholar, MTech Computer Science and Engineering SIPNA College of Engineering, Amravati, India. 2 Professor, Computer Science and Engineering, SIPNA College of Engineering and Technology Amravati, India. World Journal of Advanced Research and Reviews, 2025, 28(02), 1567–1579 Publication history: Received on 03 October 2025; revised on 14 November 2025; accepted on 17 November 2025 Article DOI: https://doi.org/10.30574/wjarr.2025.28.2.3836 Abstract The rapid advancement of Artificial Intelligence (AI) and Natural Language Processing (NLP) has significantly influenced the development of conversational agents. Multilingual chatbots, enabled by transformer-based architectures, address the critical challenge of cross-linguistic communication in domains such as education, healthcare, and customer service. Unlike traditional rule-based or LSTM-based models, transformer models leverage self-attention mechanisms to provide contextual understanding, scalability, and superior performance in multilingual settings. This paper presents a consolidated review of existing research on multilingual AI chatbots, focusing on their architectures, applications, and challenges. Prior studies have shown effective use of machine translation systems, integration with large language models, and reinforcement learning strategies to enhance dialogue quality. However, persistent gaps remain in cultural adaptability, low-resource language support, and bias mitigation. The paper highlights the need for advanced research to develop robust, culturally aware, and resource-efficient multilingual chatbots. The insights presented serve as a roadmap for future research, demonstrating the transformative role of transformer-driven chatbots in bridging global communication barriers. Keywords: Artificial Intelligence (AI); Natural Language Processing (NLP); LSTM; transformer models. 1. Introduction Conversational AI has evolved from rule-based chatbots to intelligent multilingual systems driven by transformerbased architectures. A multilingual AI chatbot is designed to understand, translate, and generate human-like responses across multiple languages, breaking communication barriers in global applications. With increasing reliance on AIpowered interfaces in education, healthcare, and business, transformers have become the backbone of advanced Natural Language Processing (NLP) pipelines. Unlike statistical and recurrent models, transformers utilize self-attention mechanisms for parallel processing, contextual representation, and scalability across languages. The importance of multilingual capabilities is further amplified in cross-cultural communication and inclusive services. This section introduces the technological basis of multilingual chatbots and outlines their role in modern AI ecosystems. The evolution of conversational agents from rule-based systems to advanced neural architectures has marked a paradigm shift in natural language processing (NLP). Multilingual AI chatbots represent one of the most impactful applications of this transition, enabling real-time, cross-lingual communication and interaction between humans and machines. Unlike monolingual systems that are constrained to a single linguistic domain, multilingual chatbots must process heterogeneous linguistic structures, idiomatic variations, and cultural nuances while ensuring semantic consistency and pragmatic accuracy across multiple languages. In the multilingual setting, pre-trained large language models (LLMs) such as m BERT, XLM-R, and GPT-based architectures leverage massive cross-lingual corpora to build shared embedding spaces. This allows for zero-shot World Journal of Advanced Research and Reviews, 2025, 28(02), 1567–1579 1568 and fewshot transfer learning, where knowledge from high-resource languages can generalize to low-resource ones. Such scalability is crucial for deploying multilingual chatbots in realworld applications where training data for certain languages may be limited. Moreover, transformer-driven multilingual chatbots integrate neural machine translation (NMT) pipelines, crosslingual embeddings, and reinforcement learning for dialogue management, resulting in improved robustness in handling diverse conversational contexts. When deployed in domains such as healthcare, education, and customer support, these systems demonstrate capabilities including real-time multilingual consultation, medical query interpretation, and domain-specific recommendation generation. Despite these advances, significant technical challenges persist. Transformers exhibit computationally intensive training requirements, necessitating high-performance hardware and optimized architectures for real-time deployment. Additionally, semantic drift in lowresource languages, bias amplification from training corpora, and insufficient cultural adaptation remain open research problems. Addressing these issues requires innovations in multilingual dataset construction, lightweight transformer variants (e.g., Distil BERT, ALBERT), and fairness-aware NLP techniques. Thus, multilingual AI chatbots powered by transformers represent a convergence of linguistic intelligence and computational scalability, positioning themselves as pivotal tools for inclusive, globalized digital communication. Their ability to transcend linguistic barriers not only enhances user experience but also drives adoption in critical sectors, thereby underscoring their transformative potential in the field of artificial intelligence. 2. Background and Motivation 2.1. Growth of multilingual chatbots in global communication Multilingual chatbots serve as AI-driven conversational agents that transcend linguistic barriers by enabling seamless dialogue across multiple languages. According to Vanjani et al., these systems leverage machine translation technologies such as Google Translate to integrate conversational platforms like Tutor Mike with support for over 103 languages. By coupling NLP and AI techniques, multilingual chatbots provide a universal communication interface, where responses remain coherent in structurally diverse languages such as German, Spanish, and Korean. They address the limitations of monolingual bots by providing cross-lingual message translation, semantic alignment, and contextual consistency, thus fostering global communication, international business interactions, and digital service delivery. However, translation quality varies depending on language families—European languages achieve higher coherence compared to Asian languages due to linguistic structural differences, underscoring the need for adaptive translation strategies. 2.2. Importance in education, healthcare, and customer service Education: Galadima et al. highlight the role of multilingual chatbots in academic consultation systems such as ACE-DS at the University of Rwanda. These bots facilitate student–faculty interactions in multiple languages, improving access to educational support. By leveraging transformer-based architectures, they offer context-aware dialogue management and real-time language adaptation, thereby bridging linguistic divides in multicultural educational environments. Healthcare: Munjal et al. Multilingual healthcare propose a multilingual virtual healthcare assistant built on transformer models. Unlike LSTMs, transformers leverage self-attention to handle long-range dependencies and parallel processing, yielding superior performance in symptom analysis, disease prediction, and cross-lingual medical query resolution. With BLEU scores varying across language pairs (e.g., English–French 0.7 vs. English–Telugu 0.39), the study demonstrates both the potential and the limitations of multilingual healthcare chatbots. Such systems reduce communication barriers between patients and providers, enhance treatment adherence, and extend healthcare services to linguistically diverse populations. Customer Service: Both studies Multilingual healthcare emphasize that multilingual chatbots are critical in global customer service platforms, where they automate interactions, reduce operational costs, and improve user engagement by supporting multiple languages in real time. The technical advantage lies in their ability to integrate semantic analysis, sentiment detection, and context tracking to deliver domainspecific, natural responses. World Journal of Advanced Research and Reviews, 2025, 28(02), 1567–1579 1569 3. Literature Review Table 1 Literature Review Author(s )/ Year Title Objective/ Focus Methodoloy / Approach Key Findings Limitations/ Gaps Relevance to Multilingual AI Chatbot using Transformer Alan Turing (1950) Computing Machinery and Intelligence Pose the question of machine intelligence; introduce the Imitation Game (Turing Test) Philosophical analysis and thought experiment Framed machine intelligence evaluation; inspired conversationa l AI evaluation frameworks Not empirical; conceptual rather than technical; predates modern ML Provides evaluation perspective and motivation for conversational agents Weizenba um (1966) ELIZA — A Computer Program for the Study of Natural Language Communica tion Between Man and Machine Demonstrat e simple patternmatching conversatio nal program Rule-based decompositio n and reassembly scripts (pattern matching) Showed how simple rules can simulate conversation and how users anthropomorp hize systems Outdated rule-based limits; brittle, no real understandin g; languagespecific scripts Earliest chatbot example; shows conversational interface design and limits for modern systems Winograd (1972) Understandi ng Natural Language (SHRDLU) Investigate language understandi ng via restricted domain (blocks world) Symbolic, logic-based semantic representatio n and procedural reasoning Demonstrated deep understanding in constrained domains using symbolic representation s Not scalable to open domains; heavy knowledgeengineering Historic foundation for dialog systems and semantic reasoning Brown et al. (1993) The Mathematic s of Statistical Machine Translation Introduce statistical models (IBM models) for machine translation Probabilistic alignment models and EM estimation using parallel corpora Established core SMT models; showed statistical methods can learn translation lexicons and alignments Relied on word-level alignment; poor fluency vs later neural methods Key precursor to statistical approaches later replaced by neural and transformer-based MT Hochreite r & Schmid Huber (1997) Long ShortTerm Memory (LSTM) Address vanishing gradients in RNNs to model long-range Propose gated recurrent architecture (LSTM) with memory cells Enabled effective learning of long-term dependencies; widely used in Computation cost; sequential (non-parallel) training compared to Transformers Important for early sequence modelling and dialog before transformers World Journal of Advanced Research and Reviews, 2025, 28(02), 1567–1579 1570 dependenci es sequential tasks later Bengio et al. (2003) A Neural Probabilistic Language Model Learn distributed word representat i ons to improve Neural network language model that embeds words and Introduced neural embeddings and parametersharing to Early models small-scale compared to later contextual models Foundation for word embeddings and neural LMs used in dialog systems language modelling predicts next word alleviate sparsity Sutskever et al. (2014) Sequence to Sequence Learning with Neural Networks End-to-end sequence transductio n (e.g., translation) using neural nets Encoderdecoder LSTM architecture trained endto-end on parallel corpora Demonstrated strong MT performance without specialized features; enabled many downstream tasks Performance sensitive to long-range info; later improved with attention Key step toward neural conversational systems and seq generation Bahdana u et al. (2015) Neural Machine Translation by Jointly Learning to Align and Translate Introduce attention mechanism to overcome fixedlength bottleneck Encoderdecoder with soft attention alignment Improved translation quality; attention provides interpretabilit y for alignments Still RNNbased; attention cost grows with sequence length Introduced attention — core ingredient later generalized in Transformers Vaswani et al. (2017) Attention Is All You Need Propose the Transforme r architecture relying solely on attention Self-attention encoderdecoder Transformer; positional encodings State-of-theart MT results; highly parallelizable and scalable Large compute/dat a requirements ; issues with very long sequences Architectural foundation for modern multilingual transformer chatbots Johnson et al. (2017) Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation Show a single NMT model translating between many languages and zeroshot transfer Multilingual NMT with language token prefixes and shared parameters Enabled translation between unseen language pairs (zeroshot) Performance varies by languageresource balance; interference across languages Early demonstration of multilingual transfer useful for multilingual chatbots Jacob Devlin et al. (2018) BERT: Pretraining of Deep Bidirectiona l Transformer s for Language Understandi Introduce masked language modeling and deep bidirectiona l pretraining Transformer encoder pretraining (masked LM + next sentence prediction) Strong improvements across many NLP tasks; generalpurpose contextual embeddings Primarily encoder-only; limited for generation; multilingual variant mBERT has biases BERT architecture influenced multilingual representation techniques for chatbots World Journal of Advanced Research and Reviews, 2025, 28(02), 1567–1579 1571 ng JunczysDowmunt et al. (2018) Marian: Fast Neural Machine Translation in C++ Practical, efficient NMT toolkit optimized for production use Efficient Transformer and RNN implementati ons; decoder optimizations Enabled rapid experimentati on and deployment of NMT models Primarily engineering; not a new modeling paradigm Tooling that accelerated multilingual MT and practical chatbot pipelines Mahesh Vanjani, Milam Aiken, Mina Park (2019) Chatbots for Multilingual Conversatio ns Build/evalu ate a multilingua l chatbot linking Tutor Mike with Google Translate to support 103 languages. Implemented a Visual Studio wrapper linking Tutor Mike and Google Translate; user study (29 students) with 12 prompts Enabled 103language conversations; German/Spani sh translations largely natural; Korean less accurate. Small sample, few prompts, English raters instead of native speakers; relies on external MT (Google Translate). Early example of integrating translation with chatbot; highlights translation disparities—valuable baseline prior to transformer-based multilingual models. across EN/DE/ES/K R; quantitative ratings. Lample & Conneau (2019) Crosslingual Language Model Pretraining (XLM) Extend generative pretraining to crosslingual settings Unsupervise d and supervised cross-lingual language modeling objectives Strong crosslingual transfer and improved MT and classification High compute; depends on monolingual corpora quality Showed how crosslingual pretraining benefits multilingual tasks for chatbots Conneau et al. (2020) XLM-R: Unsupervise d Crosslingual Representat i on Learning at Scale Scale multilingua l masked LM to 100+ languages (XLM-R) Transformer masked LM trained on massive CommonCra wl multilingual data Outperforms mBERT on many crosslingual tasks; good lowresource gains Large compute/dat a ; capacity dilution trade-offs across languages State-of-the-art multilingual encoder used for understanding in multilingual bots Liu et al. (2020) mBART: Multilingual Denoising Pre-training for NMT Sequenceto-sequence multilingua l denoising pretraining (mBART) Seq2seq denoising autoencoder pretraining across many languages Improves many-tomany and zero-shot MT; strong for low-resource pairs Pretraining heavy; occasional degradation for some high-resource pairs Useful for multilingual response generation and translation components of chatbots Raffel et al. (2020) T5: Exploring the Limits of Transfer Learning Unify NLP tasks into text-to-text and scale T5 Encoderdecoder Transformer pretraining (span Strong taskagnostic transfer; simple unified framework Very large models; compute/res o urce heavy Text-to-text paradigm underpins many chatbot pipelines (generation+underst a nding) World Journal of Advanced Research and Reviews, 2025, 28(02), 1567–1579 1572 with a Unified Text-to-Text Transformer corruption) and largescale finetuning Xue et al. (2021) mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer Multilingua l variant of T5 covering 101 languages Pretrain T5 on multilingual CommonCra wl (mC4) and adapt to multilingual tasks State-of-theart on many multilingual benchmarks; supports generation tasks Large scale and compute; imbalance across languages Directly applicable as multilingual generative backbone for chatbots Feng et al. (2020) LaBSE: Languageagnostic BERT Sentence Embedding Produce languageagnostic sentence embedding s for retrieval and similarity Dual-encoder training on translation pairs + margin loss (BERTbased) High-quality cross-lingual sentence embeddings useful for retrieval Focus on embeddings; less about generation Important for multilingual retrieval and intent/response ranking in chatbots Radford et al. (2018) Improving Language Understandi ng by Generative Pre-Training (GPT-1) Show pretraining of a Transforme r decoder LM improves downstrea m tasks Transformer decoder pretraining (unsupervise d) then finetuning Demonstrated generative pretraining benefits; sparked autoregressive LM trend Smaller scale vs later GPTs; initial exploration Autoregressive generation approach used for conversational agents Radford et al. (2019) Language Models are Unsupervise d Multitask Show large LM yields strong Large-scale Transformerdecoder LM Strong generation/ze r o-shot performance; Controllabilit y and safety; large compute Advanced conversational generation Learners (GPT-2) zero-shot capabilities trained on WebText raised safety concerns foundation for chatbots Brown et al. (2020) Language Models are Few-Shot Learners (GPT-3) Scale-up autoregress ive LM to 175B parameters and analyze few-shot learning Massive autoregressiv e Transformer trained at scale; incontext learning experiments Strong fewshot/incontext learning capabilities; broad capabilities across tasks Extremely computeheavy; hallucination and bias issues Large LMs enabled general-purpose chat interfaces and fewshot dialogue tuning Zhang et al. (2019/20 2 0) DialoGPT: Large-Scale Generative Pre-training for Conversatio nal Response Generation Adapt GPTstyle pretraining to conversatio nal data (Reddit) Transformerdecoder pretraining on 147M Reddit conversation s Improved response relevance and conversationa l fluency over baselines Single-turn focus; Reddit biases; safety/conten t issues Directly targets conversational generation using transformer pretraining World Journal of Advanced Research and Reviews, 2025, 28(02), 1567–1579 1573 Ouyang et al. (2022) Training Language Models to Follow Instructions with Human Feedback (InstructGP T / RLHF) Align LMs to human preferences using RL from human feedback Collect demonstratio ns and ranking data; supervised fine-tuning + PPO-based RLHF Improved helpfulness and reduced toxicity; models better follow instructions Resource intensive; depends on human label quality; not perfect alignment Core technique (RLHF) behind instructive/chatoptimized multilingual chatbots 4. Related Work 4.1. Review of multilingual chatbot research Recent literature on multilingual conversational agents emphasizes architectures and engineering practices that enable cross-lingual generalization, cultural-context modelling, and robustness to low-resource languages. Several applied studies adopt pipeline designs that combine language identification, neural translation (or shared multilingual encoders), and downstream task modules (intent classification, slot filling, response generation) to support many language pairs; empirical evaluations show that performance is tightly coupled to the quantity and quality of perlanguage data and to tokenization/token-alignment choices for morphologically rich languages. Practical systems typically rely on pretrained multilingual encoders (shared sub-word vocabularies or aligned cross-lingual embeddings such as MUSE) and on transfer learning + fine-tuning to bootstrap low-resource languages from high-resource ones. Engineering-level evaluations report high task accuracy in curated datasets but reveal BLEU/translation and dialog quality degradation for underrepresented languages, motivating language-specific preprocessing and dataaugmentation strategies. 4.2. Comparative studies: LSTM vs Transformers Comparative experimental work across the uploaded studies consistently attributes superior multilingual and longcontext performance to transformer architectures relative to sequential recurrent models (LSTM/RNN), primarily for three architectural reasons: (1) self-attention provides direct modelling of long-range dependencies and high-order interactions across tokens, (2) parallelizable computation reduces training instability and enables scaling to large pretraining corpora, and (3) sub-word / sharedvocabulary pretraining yields stronger cross-lingual latent spaces. Empirical results from domain experiments report large margins in task accuracy and validation stability: one encoderonly transformer implementation attained approximately 85 percent accuracy (validation approximately 90 percent) versus approximately 65 percent for an LSTM baseline, with transformer training exhibiting steadier validation loss and improved generalization, while LSTM often converged faster but with higher variance and overfitting tendencies. These findings are replicated in symptom-prediction and translation evaluations where transformers also produced higher BLEU for majority languages and more stable classification metrics. Methodological notes from the corpus: when models are compared, it is critical to control for (a) pretraining (offtheshelf frozen embeddings vs fully pre-trained encoder), (b) vocabulary/tokenizer design (BPE/unigram shared vs language-specific), and (c) data sampling strategy (balanced vs natural long-tailed distributions). Several papers stress that benefits of transformers are most pronounced when large multilingual pretraining or cross-lingual transfer is available; in extremely low-data regimes, careful augmentation and unsupervised alignment (e.g., MUSE) remain necessary to close gaps. 4.3. Bibliometric trends in LLMs Bibliometric analyses of LLM research show an exponential publication trajectory and rapid topical diversification since the introduction of transformer architectures. A recent WoS-based bibliometric study quantifies extreme growth (annual scientific production growth cited at 220.74 percent in the analyzed interval), a shift from model-engineering papers to application-driven and domainspecific LLM investigations (medicine, education, finance, environmental sciences), and the emergence of “LLMs + domain” subcommunities. The study uses standard bibliometric facets (annual production, prolific authors/institutions, source clustering, keyword n-grams, and LDA topic models) World Journal of Advanced Research and Reviews, 2025, 28(02), 1567–1579 1574 to demonstrate that Transformers → GPT/ChatGPT represent both a conceptual pivot and an accelerant for crossdisciplinary adoption. Key bibliometric observations that inform system design and evaluation choices for multilingual chatbots: (1) concentration of high-impact work in well-resourced languages and institutions implies abundant pretrained weights and data for certain language pairs but relative scarcity elsewhere, (2) evaluation benchmarks are proliferating but remain uneven across domains and languages, motivating combined metric suites (generation quality + task accuracy + safety/ethics checks), and (3) reproducibility and open-data efforts (open LLMs, released datasets and replication packages) are expanding -enabling more reliable cross-study comparisons and multilingual model audits. 4.4. Synthesis and design implications for multilingual transformer chatbots The corpus indicates the following design principles for a transformer-based multilingual chatbot: (i) adopt an encoder or encoder-decoder transformer backbone with large multilingual pretraining or checkpoint initialization; (ii) use shared sub-word vocabularies and cross-lingual alignment techniques to bootstrap low-resource languages; (iii) evaluate with combined metrics (task F1/accuracy, BLEU/translation fidelity, per-language calibration); and (iv) publish datasets, hyperparameters and ablation studies (including LSTM baselines) to support reproducibility and domain-specific auditability. These principles are motivated by empirical performance gaps across languages, the architectural advantages of transformers, and the bibliometric evidence of rapid, application-driven LLM growth. 5. Methodology 5.1. Data Acquisition and Preprocessing Corpus collection: Studies collected multilingual datasets from domain-specific contexts (education, healthcare, business) along with open parallel corpora (Wikipedia dumps, CommonCrawl, translation datasets). Preprocessing techniques: Applied tokenization, stemming/lemmatization, and normalization; most adopted subword-level tokenization (BPE, SentencePiece) to enable shared vocabulary across languages. Low-resource strategies: Implemented backtranslation, synthetic data generation, and embedding alignment (e.g., MUSE) to mitigate scarcity in underrepresented languages. Cultural context modeling: Some works enriched datasets with cultural idioms and references for crosscultural communication. 5.2. Model Architectures. Transformer-based models: Encoder-only models (e.g., mBERT, XLM-R) for intent recognition, slot filling, and classification tasks. Encoder-decoder models (e.g., mT5, GPT-style architectures) for dialogue generation and translation. Comparative baselines: Several studies benchmarked LSTM/RNNs against transformers, consistently showing that transformers outperform LSTMs in multilingual understanding, long-range dependency modeling, and accuracy. Hybrid architectures: Pipelines combining speech-to-text → translation → transformer-based NLU → transformer decoder for response generation were adopted in healthcare and education chatbots. 5.3. Training Strategies Transfer learning: Fine-tuning pre-trained multilingual checkpoints (mBERT, XLM-R, BLOOM) on taskspecific dialog corpora. Cross-lingual transfer: Leveraging high-resource languages (English, Spanish, French) to improve performance in low-resource ones through shared embeddings and transfer learning. Multitask learning: Training models jointly on intent classification, slot filling, and response generation to improve generalization. World Journal of Advanced Research and Reviews, 2025, 28(02), 1567–1579 1575 Balanced sampling: Avoiding dataset bias by controlling for disproportionate representation of highresource languages. 5.4. Evaluation Methodologies Task metrics: Classification: Accuracy, precision, recall, F1. Generative: BLEU, ROUGE, perplexity. Comparative evaluation: Reported transformer accuracy approximately 85 percent vs LSTM approximately 65 percent in healthcare tasks; BLEU scores varied significantly across languages (English→French: 0.7; English→Telugu: 0.39). Qualitative evaluation User studies (e.g., student and patient feedback) validated multilingual coherence and cultural appropriateness. 5.5. System Integration End-to-end pipeline: 1.Input (speech/text) 2.Language detection 3.Transformer-based NLU (intent, slot filling) 4.Response generation (encoder-decoder transformers) 5.Translation/localization for output. Cloud deployment: Some implementations integrated REST APIs and cloud-based services for scalability in realworld multilingual interactions. Domain adaptation: Specialized vocabularies and ontologies (e.g., medical terms in healthcare assistants) were embedded into training to improve domain relevance. 5.6. Transformer Architecture for Multilingual Chatbots The transformer architecture forms the foundation of multilingual conversational AI, offering a paradigm shift from sequential models like RNNs and LSTMs to parallelized, self-attention–driven processing. As highlighted in Cotfas et al., transformers use an encoder–decoder framework, where the encoder generates context-rich embeddings of input sequences, and the decoder produces target outputs such as translated or contextually aligned responses. 5.7. Encoder–Decoder Mechanism in Transformers The encoder–decoder mechanism is the core of transformer-based architectures, providing the foundation for multilingual chatbots. Encoder: Processes the input sequence and transforms it into contextual embeddings. Each encoder layer applies selfattention to capture dependencies among all tokens, followed by feed-forward networks for nonlinear transformation. The model first converts discrete multilingual user inputs (words/tokens) into dense vector embeddings. These embeddings form the continuous representation space required for the transformer to process textual data. Decoder: Generates the target sequence (e.g., translated or chatbot response). It employs both self-attention (on previously generated tokens) and cross-attention (to align with encoder outputs), ensuring semantic and syntactic alignment across languages. Self-Attention Mechanism – captures global dependencies among tokens, regardless of position. Multi-Head Attention – enables the model to attend to different semantic subspaces simultaneously. The transformer in the multilingual healthcare assistant replaces sequential LSTM processing with a selfattention mechanism. Each input token embedding is projected into queries (Q), keys (K), and values (V), and attention weights are computed across the entire input sequence. This enables the model to learn relationships among all tokens in parallel, regardless of distance — crucial for handling long-range dependencies in multilingual inputs. Mathematically, the operation follows the scaled dot-product attention formulation: Attention(Q,K,V) = SoftMax(Q,K,V) = SoftMax(QKT/√dk)V. The multi-head design allows the model to capture contextual information across different subspaces simultaneously, improving understanding of diverse linguistic structures. Positional Encoding – preserves sequence order, compensating for the absence of recurrence. Unlike recurrent models, transformers do not have inherent sequence order awareness. To handle sequential information in multilingual healthcare queries, the transformer employs self-attention enhanced with positional encodings. These encodings enable the model to capture relationships among tokens in parallel across the entire sequence, ensuring effective contextual understanding across multiple languages.