A two-stage Q-learning routing approach for quantum entanglement networks

Annals of Telecommunications (2025) 80:807–822 https://doi.org/10.1007/s12243-025-01090-4 A two-stage Q-learning routing approach for quantum entanglement networks Diego Abreu1·Arthur Pimentel1·David Moura2·Christian Rothenberg2·Antônio Abelém1 Received: 31 August 2024 / Accepted: 9 April 2025 / Published online: 22 April 2025 © Institut Mines-Télécom and Springer Nature Switzerland AG 2025 Abstract The emerging field of quantum internet offers multiple applications, enabling quantum communication across diverse networks. However, the current entanglement networks exhibit complex processes, characterized by variable entanglement generation rates, limited quantum memory capacity, and susceptibility to decoherence rates. Addressing these issues, we propose a two-stage routing system that harnesses the power of reinforcement learning (RL). The first stage focuses on identifying the most efficient routes for quantum data transmission. The second stage concentrates on establishing these routes and improving how and when to apply entanglement swapping and purification. Our extensive evaluations across various network sizes and configurations reveal that our method not only sustains superior end-to-end route fidelity but also achieves significantly higher request success rates compared to traditional methods. These findings highlight the efficacy of our approach in managing the complex dynamics of quantum networks, ensuring robust and scalable quantum communication. Our method’s adaptability to changing network conditions and its proactive management of quantum resources make an important contribution to quantum network efficiency. Keywords Quantum network ·Routing ·Reinforcement learning ·Quantum internet 1 Introduction The quantum internet represents a significant leap forward in information transmission and processing, aiming to enhance security and efficiency [5,23]. It facilitates quantum communication across multiple nodes, enabling transformative Diego Abreu, Arthur Pimentel, David Moura, Christian Rothenberg, and Antônio Abelém contributed equally to this work. BDiego Abreu [email protected] Arthur Pimentel arthur[email protected] David Moura [email protected] Christian Rothenberg [email protected] Antônio Abelém [email protected] 1Federal University of Pará (UFPA), Belém, Brazil 2Universidade Estadual de Campinas (UNICAMP), Campinas, Brazil applications such as quantum key distribution (QKD) [42], distributed quantum computing [10], and the creation of entanglement networks [32]. These applications are crucial for the future of classical and quantum communication and computing technologies [2]. However, several technological limitations affect the development of the quantum internet. Current quantum technologies struggle with low rates of entanglement generation, limited quantum memory at each node, and quantum decoherence that leads to low-quality entangled states and qubits. These issues significantly affect the performance of quantum networks, especially in routing information [19]. In addition to this, routing quantum information through entanglement networks is particularly challenging [18]. Quantum information, unlike in classical networking, cannot be copied and forwarded due to the quantum no-cloning theorem. Instead, it requires specific quantum operations for transmission and must maintain high fidelity while using minimal resources. This inherently dynamic nature of quantum networks necessitates a carefully designed routing strategy. As a result, the development of efficient and dependable routing strategy is crucial for ensuring the effective functioning of any quantum communication network. 123 808 Annals of Telecommunications (2025) 80:807–822 To face these challenges, this paper introduces a twostage quantum routing approach that applies Q-learning [24], a reinforcement learning technique [37], to enhance routing decisions in quantum entanglement networks, both on the path selection and route establishment stages. Our comprehensive experiments demonstrate that our method surpasses traditional routing methods in maintaining highfidelity routes and achieving superior request success rates under various network conditions. This work contributes the following to the field: – Development of a two-stage reinforcement learning approach to improve routing within quantum entanglement networks. – Detailed analysis of our method’s efficacy in enhancing route fidelity and handling increasing routing requests across diverse network configurations. – Comparative performance assessment of our approach against four established routing strategies, highlighting its robustness and effectiveness in simulated scenarios. The remainder of this paper is organized as follows: Sect.2 outlines the theoretical background on routing in quantum entanglement networks. Section3reviews related work, providing context for our approach. Section 4details our approach. Section5describes the evaluation methodology used in our experiments. Section6discusses the results obtained. Finally, Sect.7concludes the paper and suggests directions for future research. 2 Quantum entanglement networks Quantum entanglement networks generate and distribute entangled pairs across multiple nodes within a network. This capability enables various applications, including the transmission of quantum data through quantum teleportation [23], applications in quantum sensing [12], distributed quantum computing, and blind quantum computation [6] across these networks. EPR pairs, also called Bell pairs, are a type of entangled quantum state representing two entangled qubits. Specifically, an EPR is a maximally entangled state of two qubits. There are four states, which can be described as follows: |Φ+= 1 √2(|00+|11), |Φ−= 1 √2(|00−|11), |Ψ+= 1 √2(|01+|10), |Ψ−= 1 √2(|01−|10). (1) In these notations, |0and |1represent the basis states of a qubit. The EPRs are essential for various quantum communication protocols because the measurement outcomes of the two qubits are strongly linked regardless of the distance separating them. The state |Φ+= 1 √2(|00+|11)represents a perfect correlation where both qubits are either in the state |0or |1. The state |Φ−= 1 √2(|00−|11)is similar but with a relative phase difference, affecting quantum interference. The state |Ψ+= 1 √2(|01+|10)shows anti-correlation, where one qubit is in |0and the other in |1. The state |Ψ−= 1 √2(|01−|10)also represents anticorrelation but with a phase difference, influencing quantum interactions. These entangled states enable secure communication, as any eavesdropping disturbs the entanglement, making them indispensable for quantum internet applications. In quantum networks, EPR pairs are generated predominantly through protocols that employ heralded entanglement generation [11]. To initiate the formation of an EPR pair, the source node entangles a memory qubit with a transmission qubit. This entangled qubit is then dispatched via an established quantum channel to a target node. Upon receipt, the transmission qubit is entangled with a memory qubit at the target node, culminating in the creation of an EPR pair between the two nodes. The successful establishment of the EPR pair is subsequently verified through a classical communication channel, such as the Internet, which informs both nodes of the entanglement’s successful realization. Nonetheless, this procedure is susceptible to failures, introducing significant reliability challenges to quantum communication systems. In addition to this, as the distance increases, the quality of the entanglement—measured by the EPR fidelity— decreases. Therefore, to maintain high-fidelity entanglement over long distances, quantum repeaters are necessary. These repeaters work by dividing the total distance into shorter segments, creating high-fidelity EPR pairs within each segment and then connecting these segments through entanglement swapping (ES) and creating an end-to-end entanglement (E2E). The entanglement swapping process uses link-level EPR pairs to produce a new EPR pair that connects the origin and destination. In this process, the link-level EPR pairs are consumed, necessitating the creation of newer EPR pairs for each link afterward. Additionally, the ES process is probabilistic and susceptible to noise and quantum decoherence, leading to the degradation of the final EPR pair’s state fidelity. The fidelity resulting from the entanglement swapping process, denoted Fswap(f1,f2), is given by Eq.2. Here, f1and f2 are the EPRs fidelities, from two distinct links, before the 123 Annals of Telecommunications (2025) 80:807–822 809 swapping process: Fswap(f1,f2)=f1·f2+(1−f1)·(1−f2)(2) There are multiple entanglement swapping strategies, each with its benefits and drawbacks [1,28,29]. Figure1 presents three ES strategies: sequential, nested, and opportunistic [15,22]. The straightforward method to perform entanglement swapping is sequentially (or cascaded), as presented in Fig.1a. In this approach, swaps are executed one at a time between the origin (Alice) and intermediary repeaters until the destination (Bob) is reached, creating end-to-end entanglement. The capacity for this method is given by EPRrate×ESp(N−1). Here, Nis the number of repeaters, ESpis the success probability of the swap, and EPRrateis the rate that the EPR can be created on the link. In the nested ES strategy, as presented in Fig.1b, the path is divided to allow multiple swaps to occur simultaneously, converging in the middle to form an end-to-end entanglement. This method typically requires fewer steps than the sequential approach, which is crucial for reducing the impact of EPR decoherence over time. The advantage of this strategy is its polynomial scaling with the number of repeaters, with an end-to-end capacity of EPRrate×ESplog N. However, if one end of the process succeeds while the other fails, the successful pair will partially lose fidelity due to decoherence, necessitating repeated attempts until both ends succeed. The opportunistic strategy (also known as ad-hoc or as swap-as-soon-as possible), as presented in Fig.1c, starts to swap as soon as enough resources are available for each segment, unlike previous methods, which wait for all link-level EPR pairs to be ready. This approach can lead to fidelity decay with the number of repeaters, as EPR pairs may wait until sufficient resources are available. However, the opportunistic strategy generally results in the lowest latency, making it a viable option for time-sensitive applications. Fig. 1 Different approaches to entanglement swapping and purification 123 810 Annals of Telecommunications (2025) 80:807–822 In our approach, we model sequential, nested, and opportunistic strategy. Other existing approaches, such as parallel [28,29], might also be included in future works. Another fundamental process in quantum communication is quantum purification or distillation [40,41], where multiple low-fidelity EPR pairs can be used to produce a higher fidelity EPR pair, as presented in Fig.1d. Considering a bit-flip channel model and the DEJMPS [13] purification protocol, the resulting fidelity Fpur after one purification round, where f1and f2are the fidelities of the original EPR pairs, is given by Eq.3[8,33]. Fpur(1,2)=f1·f2 f1·f2+(1−f1)·(1−f2)(3) As illustrated in Fig.1d, the purification process can be performed at a local level, using EPR pairs from the same channel, or at higher levels, by combining two or more EPR pairs, after swapping, to purify them. It can also be done at the end-to-end level, purifying two end-to-end EPR pairs between the same origin and destination. However, similar to the entanglement swapping process, purification consumes the original EPR pairs and is also a probabilistic process that is prone to failure. Therefore, to establish a route between the origin and destination of a quantum application, an end-to-end pair must be created through the process of entanglement swapping. Additionally, quantum purification is necessary to maintain the minimum fidelity requirements of each application. The final fidelity after the necessary entanglement swaps and purification is referred to as the end-to-end (E2E) fidelity. Although the link end-to-end capacity may serve as an initial indicator of the feasibility of a given route, it does not fully capture the quality of the resulting entanglement. The final fidelity, referred to as the end-to-end fidelity, is not only determined by the selected path, but also by the sequence of actions applied during the route establishment phase, such as the choice and placement of swapping and purification operations, and the network state. These aspects are jointly improved in our two-stage routing design, which integrates both path selection and route refinement to improve the overall quality of the entanglement distribution. Figure2shows an example of the establishment of a quantum end-to-end entanglement between a source (Alice) and a destination node (Bob). At time t0, Alice already possesses two EPR pairs with Repeater 1 (R1), which can undergo purification to enhance the fidelity of the EPR pair. However, further along the communication route, additional EPR pairs are still needed to be generated. At time t1, R1 and Repeater 2 (R2) successfully create an EPR pair. However, attempts to establish EPR pairs between R2 and Repeater 3 (R3), as well as between R3 and Bob, fail. As a result, the process of generating EPR pairs must be repeated. Using opportunistic entanglement swapping, Alice, R1, and R2 perform entanglement swapping between their EPR pairs, creating a new (virtual or temporary) EPR link between Alice and R2. At Fig. 2 Quantum entanglement distribution and swapping process in a multi-hop network 123 Annals of Telecommunications (2025) 80:807–822 811 times t3and t4, the virtual link is swapped with the repeaters’ EPR pairs, ultimately establishing an end-to-end entanglement between Alice and Bob. 2.1 Routing in quantum entanglement networks The routing process in quantum entanglement networks consists of two main stages: path selection and route establishment. The path selection part consists of the discovery, analysis, and selection of the most appropriate path for a specific request involving the creation of an end-to-end EPR pair to be used in a quantum network application. In the route establishment stage, once the path is selected, the goal is to create the end-to-end EPR pair itself, using the ES and purification process. The end-to-end EPR created must have at least the minimum fidelity required by the routing request; otherwise, the request will not be successfully attended. According to [27], existing research on quantum routing focuses primarily on two models with respect to the operational workflow of the entanglement distribution: the advanced generation model and the on-demand generation model. The advanced generation model involves generating and distributing entanglement prior to selecting the communication path, effectively segregating the functions of the link layer and the network layer. In contrast, the on-demand generation model allows for the distribution of entangled pairs as needed along predetermined paths. This model integrates the operations of the link and network layers, necessitating consideration of path selection, entanglement generation, and potential failures within the routing design. In this paper, we explore both models. The network continuously attempts to create EPR pairs and qubits in advance, allowing them to be readily integrated into selected paths for use by the network layer in establishing end-to-end pairs. Additionally, EPR pairs can be generated on-demand to accommodate paths where pre-existing pairs are unavailable or to facilitate entanglement swapping and purification protocols. In quantum networking, the routing of requests is aimed at identifying either a single best path or multiple viable paths, depending on the context. The definition of optimal in this scenario typically encompasses criteria such as the maximum number of end-to-end entangled states within a given time frame, the highest possible fidelity, and the minimal use of entangled resources. However, achieving this ideal becomes complex in dynamic and diverse network environments, where factors such as variable link capacities, different physical lengths, diverse qualities, and the changing availability of quantum memory and EPR pairs play significant roles. Thus, given the complexity and dynamic nature of quantum networks, routing strategies must be inherently flexible and continuously adaptive, evolving in accordance with the shifting dynamics of the network’s state. In various applications, the use of quantum networks may require routing requests with differing minimum fidelity requirements. As a result, a suitable routing strategy for one set of requests might not be sufficient for another. Therefore, it is crucial to adopt adaptable strategies to address the varying demands. In response to these challenges, this paper introduces a novel routing framework designed to dynamically adjust to the evolving landscape of quantum networks. Our framework seeks to improve both path selection and routing establishment using the network information and the specific demands of quantum communication tasks, thereby enhancing the efficiency and performance of quantum data transmission. 3 Related work Various research studies explore the development and current trends in quantum networks from multiple perspectives [7, 20,21]. This section reviews related work that focuses on routing in quantum entanglement networks. This topic has been the subject of recent surveys [1,14], highlighting the importance and challenges of routing in quantum networks. One of the pioneering studies on quantum routing by Van Meter et al. (2013) [39] introduces qDijkstra, an adaptation of the classic Dijkstra algorithm tailored for quantum networks. The method selects routes based on the number of hops between nodes, taking into account the available qubits at each node and the EPR pairs in each channel. In our research, we compare qDijkstra with our proposed method which uses additional metrics to determine the best route. In this context, various metrics have been employed to guide routing decisions. Channel fidelity is a key metric in several studies focused on routing strategies [17,26,36]. The availability of EPR pairs is another important measure of route quality [16,35]. The availability of memory qubits at each node has also been examined from different perspectives. For instance, Patil et al. (2021) [34] assume infinite memory qubit availability in their routing protocol. Other research underscores the critical importance of memory [31]. Miguel-Ramiro et al. (2023) [30] propose a routing strategy aimed at minimizing memory qubit consumption while considering decoherence effects. Our approach, in contrast, evaluates fidelity, EPR pair availability, and memory availability to determine the best routing decision. Machine learning methods have been increasingly applied to quantum entanglement networks. Le et al. (2022) [25] introduce a reinforcement learning-based routing protocol. However, their approach prioritizes the simultaneous allocation of multiple requests. In contrast, our proposal focuses on identifying the most efficient route for each individual entanglement request and what action to choose in the route establishment stage. Le et al. (2022) use the Dijkstra algo123 812 Annals of Telecommunications (2025) 80:807–822 rithm for route selection, emphasizing the number of hops between nodes. Their model considers the qubit capacity of each node and the availability of EPR pairs. Roik et al. (2024) [38] propose a reinforcement learning-based routing protocol utilizing the proximal policy optimization algorithm, specifically focusing on the path selection stage. The study compares this approach with Monte Carlo and Dijkstra algorithms. It analyzes networks affected by different types of noise (white noise, amplitude damping, phase noise), evaluating how the optimal path for a given network can be determined in the presence of such noise. In contrast, our work focuses on both path selection and route establishment. In our previous works [3,4], we explored the application of a Q-learning method to the quantum routing problem. The current paper is an extension of [4] in which we modeled key factors such as fidelity EPR pairs and qubits, demonstrating how adjusting network parameters could influence model performance in terms of the end-to-end fidelity. Then, we further model the local purification and entanglement swapping process and evaluate the success request rate under different conditions. In this current work, we enhance the method by modeling a two-stage routing system, addressing both the path selection and route establishment stages of quantum routing. Additionally, we refine our reward function to better align with the objectives of our method. These improvements allow us to experiment with different entanglement swapping and purification configurations, leading to a more realistic and comprehensive study. 4 Reinforcement learning routing for quantum entanglement networks The proposed method leverages an adapted version of the Qlearning algorithm [9], a reinforcement learning technique, to improve routing decisions in quantum networks. Q-learning is a model-free reinforcement learning algorithm that enables an agent to learn an action selection policy by interacting with the environment. Updates a Q-table based on rewards received from previous actions, gradually converging to an optimal strategy. We selected Q-learning for its ability to handle dynamic and uncertain environments without requiring a predefined model of the quantum network. By learning from accumulated experience rather than relying on a pre-trained model, our proposal dynamically adapts to the conditions of quantum networks, making it an effective and flexible solution for routing in entanglement networks. 4.1 Q-learning agent The Q-learning agent in our system is the central decisionmaker, responsible for selecting actions based on the data it receives from the environment. This environment represents the quantum network, which encompasses all nodes, communication channels, available EPR pairs, and the fidelity of those channels. As the agent interacts with the environment, it receives updates on the state of the network and rewards for the actions it takes, guiding its decision-making process. Figure3illustrates the reinforcement learning approach. The state of the network (S) is defined by factors such as channel fidelity, the number of available EPR pairs, and the availability of qubit at each node. Based on this state, the agent can choose from a variety of actions (A), including selecting different routes and selecting the ES and purification strategies. The reward (R) is provided as feedback, indicating how the chosen action contributes to network performance, such as by improving route fidelity, efficiently using resources (qubits and EPR pairs) and if the request was successfully attended. The agent follows a policy (π), a strategy that determines the next action based on the current state. This policy is crucial for guiding the agent to make decisions that enhance the overall performance of the quantum network, aiming for the most efficient and effective routing possible. Fig. 3 Reinforcement learning approach 123 Annals of Telecommunications (2025) 80:807–822 813 4.2 Reward function The reward function in our Q-learning framework is designed to balance multiple objectives that the agent must consider during its operation. These objectives include request success, improved fidelity compared to average fidelity, and efficient use of EPR pairs and qubits. The agent receives rewards based on the success or failure of the routing request: Rewardrequest =+θif success −θif failure (4) Fidelity is compared to the average fidelity Favg: Rewardfidelity =⎧ ⎪ ⎨ ⎪ ⎩ +κif F>Favg 0ifF=Favg −κif F<Favg (5) EPR usage is evaluated against the average usage Eavg: RewardEPR =⎧ ⎪ ⎨ ⎪ ⎩ +λif EPRused <EPR avg 0ifEPRused =EPR avg −λif EPRused >EPR avg (6) The reward structure for qubit usage is similar to that of EPR pairs: RewardQubit =⎧ ⎪ ⎨ ⎪ ⎩ +δif Q<Qavg 0ifQ=Qavg −δif Q>Qavg (7) The reward function relies on several key variables that influence the decision-making process. Frepresents the fidelity of the established entanglement link, indicating how closely the generated quantum state resembles the ideal Bell state. Favg is the average fidelity across the network, serving as a reference value to evaluate improvements or degradations in link quality. This value is periodically updated based on network statistics. The variable EPR used denotes the number of EPR used. Similarly, EPR avg is the average EPR pair consumption across different routing requests, providing a reference for assessing whether a given routing decision is resource-efficient. The parameter Qrepresents the number of qubits allocated to the routing process. Since qubit resources are limited in a quantum network, minimizing qubit usage while maintaining performance is crucial. Lastly, Qavg refers to the average number of qubits used in the network, which helps determine whether the agent’s routing decision is more or less efficient than the average. In our Q-learning framework, these rewards are combined to form the overall reward function, which guides the agent in making decisions that improve network performance across multiple criteria. By considering request success, fidelity, and efficient use of both EPR pairs and qubits, the agent is better equipped to handle the complexities of a quantum network. The final reward function R(s,a,s)is computed as a weighted sum of the individual rewards and penalties from each category. R(s,a,s)=w1·Rewardrequest +w2·Rewardfidelity +w3·RewardEPR +w4·RewardQubit (8) In this formulation, srepresents the current state of the quantum network, adenotes the action taken by the agent, and sis the resulting state after executing action a.This structure allows the reward to reflect not only the immediate effects of an action but also its impact on the future state of the network. For our experiments, the parameters θ,κ,λ, and δwere set to 1. Furthermore, the weights w1,w 2,w 3, and w4were also set at 0.25, ensuring equal importance in all categories. This configuration allows for a balanced assessment, with the maximum possible reward being +1 and the minimum -1, enabling us to evaluate the influence of each component on the overall performance in a controlled manner. 4.3 The two stages routing algorithm The goal of the routing algorithm is to train the agent to select communication routes and subsequent actions for end-toend entanglement distribution, considering metrics such as fidelity, EPR pair availability, and qubit availability at network nodes. Algorithm 1 presents the two-stage approach. The agent is designed to improve routing decision across a quantum network by selecting the most effective communication routes and actions, taking into account key metrics such as fidelity, EPR pair availability, and qubit availability at various network nodes. Initially, the agent is set up with key parameters, including the learning rate (α), discount factor (γ), and the probabilities of selecting random actions (ε). The agent also receives information about the network topology, represented by nodes and channels. The learning rate controls how quickly the agent updates its knowledge based on new information. A higher αallows faster adaptation to new experiences, but may cause instability if set too high; a lower αpromotes stability but may slow learning. The discount factor determines how much the agent values future rewards compared to immediate rewards. A value closer to 1 means the agent considers long-term benefits, while a lower value makes it focus more on short-term gains. The exploration rate balances the trade-off between 123 814 Annals of Telecommunications (2025) 80:807–822 Algorithm 1 Reinforcement learning algorithm 1: Input: Quantum network topology, including channels and nodes. 2: Output: Actions on the route. 3: Agent initialization: 4: Set learning rate (α), discount factor (γ) and probabilities of selecting random actions (ε). 5: Initialize Q-table with arbitrary values. 6: State definition: 7: Capture the current network configuration, including channel fidelity, number of available EPR pairs, and qubits at the nodes. 8: Stage 1: path selection 9: Choose path based on ε-greedy policy. 10: Execute the path and observe the new state and reward. 11: Stage 2: action selection on path 12: For the chosen path, select action using ε-greedy policy. 13: Possible actions: entanglement swapping or purification. 14: If ES is chosen, select one of three methods: sequential, nested, opportunistic. 15: If purification is chosen, select one of three methods: local, in the middle, end-to-end. 16: Execute the action and observe the new state and reward. 17: Q-value update: 18: Calculate combined reward 19: Update Q-table using the formula: 20: Q(s,a)=Q(s,a)+αR(s,a)+γmaxaQ(s,a)−Q(s,a) 21: return Route and actions. exploration and exploitation. With probability ε, the agent selects a random action to explore new paths; with probability 1 −ε, it chooses the action with the highest estimated reward based on current knowledge. As Algorithm 1 shows, in stage 1, path selection, the agent selects the path using an ε-greedy policy. With probability ε, a random action is chosen, and with a probability of 1 −ε, the action that maximizes the Q value is chosen. The chosen path is evaluated based on fidelity, EPR pair availability, and qubit availability. In stage 2, action selection on path, for the selected path, the agent chooses the best action to perform, either entanglement swapping or purification, with specific methods for each. ES can be executed using sequential, nested, or opportunistic methods, while purification can be done locally, in the middle, or end-to-end. The actions are also chosen based on an ε-greedy policy. After executing both stages, the combined reward is calculated. The agent maintains a Q-table, where each entry Q(s,a)represents the expected cumulative reward for taking action ain state s. These Q-values guide the agent in selecting optimal actions based on past learning. The Q-values are updated to reflect this reward, guiding the agent’s learning process. The Q-values are updated using the formula: Q(s,a)=Q(s,a)+ αR(s,a,s)+γmax aQ(s,a)−Q(s,a)(9) where –Q(s,a)is the estimated Q value for the state-action pair. –αis the learning rate. –R(s,a,s)is the received reward. –γis the discount factor. The reward is calculated based on the overall success of the request, considering path end-to-end fidelity, resources used to create the path, and whether the request was successfully attended. This structured approach allows the agent to learn and adapt, improving the performance of the quantum network over time. 5 System model and experiments To assess the effectiveness of our proposed method, we modeled the quantum network using a discrete simulation based on graph representation. The topology chosen for the case study was a lattice structure, as depicted in Fig.4.In this model, the network is formally represented as G= (V,E), where Vdenotes the set of quantum nodes. Each of these nodes is equipped to perform essential quantum communication operations, such as quantum teleportation and entanglement swapping, which facilitate the transmission and reception of quantum information. The set Ecorresponds to the edges, representing the quantum communication links between the nodes. The capacity of each node is defined by the number of qubits it can store in its quantum memory at any given time, which can be utilized for tasks like teleportation, entanglement generation, and swapping. This simulation setup allows us to effectively evaluate how our method performs in a quantum network. The network configuration parameters used in our experiments are as follows: The initial number of qubits available at each node at the beginning of each simulation, denoted Qi,is set between [0.5n,0.75n]of the total node capacity. The initial amount of EPR pairs available at each link, EPR i, ranges between [4,8]pairs. The initial fidelity of the created EPR pairs, represented by Fi, is between [0.99,0.95]. The rate of attempts to create EPR pairs at each time slot at each link, denoted as EPR rate, is set to 2 attempts per time slot. Similarly, the rate of attempts to create new qubits at each time slot at each node, referred to as Qrate, is fixed at 4 attempts per slot. The entanglement swap success probability, ESp, and the purification success probability, Purp, both range between [0.95,0.9]. We consider bipartite entangled Bell states as the foundation of our system. For the experiments, we assumed a symmetric entanglement purification strategy, where two Bell pairs are used to produce a higher-fidelity pair [8,33]. Lastly, the minimum acceptable fidelity for the network operations, labeled as min fidelity, is set between [0.70,0.55]. It is important to note that time in our simula123 Annals of Telecommunications (2025) 80:807–822 815 Fig. 4 Quantum network in the lattice topology tion is segmented into discrete slots, allowing for a granular examination of network dynamics over time. To incorporate the effect of quantum memory limitations, we consider that stored EPR pairs do not retain their initial fidelity over time. Instead, their quality degrades progressively due to memory decoherence, following an exponential decay with each timeslot they remain stored. In our model, EPR pairs can persist for multiple timeslots, but their fidelity decreases accordingly, directly affecting their usability for routing decisions. This modeling reflects a more realistic behavior of quantum memories and emphasizes the importance of timely resource utilization in entanglement-based communication networks. During simulation, the fidelity of all stored EPR pairs is updated at each timeslot. If the fidelity of a stored EPR pair falls below the minimum required threshold min fidelity, the pair is considered unusable and discarded. This modeling approach introduces a realistic constraint on entanglement availability, directly affecting routing performance and decision-making. In our experiment, an episode refers to a complete cycle of network operations, starting with an initial allocation of resources and ending after a fixed number of routing requests have been processed. Each episode generates a fixed number of requests with randomly assigned origin and destination nodes. Each request comes with a minimum fidelity requirement, meaning that if the generated E2E EPR pair does not meet this threshold, the request will not be successfully attended. Network resources such as EPR pairs and qubits can be prepared in advance and on-demand. At the start of each episode, each channel is pre-equipped with EPR pairs (EPRi) each at an initial fidelity Fi, and each node has Qi Qubits, also at fidelity Fi. Throughout each timeslot, the network attempts to create additional EPR pairs and qubits at EPRrate and Qrate. If ongoing ES operations or purification processes exhaust the available resources, additional EPR pairs and qubits are generated on-demand to meet these needs. We evaluate our proposal in various network configurations by varying parameters such as EPR and qubit rates (EPRrate and Qrate), entanglement swapping and purification success probabilities (ESp and Purp), initial available qubits (Qi), initial available EPR pairs (Epri), and the minimum fidelity requirements for requests. The impact on the average end-to-end route fidelity and request success rate is assessed under these different conditions. The request success rate represents the proportion of successfully completed requests relative to the total number of requests generated within an episode. A request is considered successful if the established end-to-end EPR pair meets or exceeds the minimum fidelity requirement. This metric provides insight into the network’s ability to fulfill communication demands under different conditions. The average end-to-end route fidelity quantifies the quality of the 123 822 Annals of Telecommunications (2025) 80:807–822 32. Nötzel J, DiAdamo S (2020) Entanglement-enhanced communication networks. In: 2020 IEEE International conference on quantum computing and engineering (QCE) 33. Panigrahy NK, Vasantam T, Towsley D, Tassiulas L (2023) On the capacity region of a quantum switch with entanglement purification. In: IEEE INFOCOM 2023-IEEE Conference on computer communications, pp 1–10. IEEE 34. Patil A, Jacobson JI, Van Milligen E, Towsley D, Guha S (2021) Distance-independent entanglement generation in a quantum network using space-time multiplexed Greenberger–Horne–Zeilinger (GHZ) measurements. In: 2021 IEEE International conference on quantum computing and engineering (QCE), pp 334–345. IEEE 35. Patil A, Pant M, Englund D, Towsley D, Guha S (2022) Entanglement generation in a quantum network at distance-independent rate. npj Quantum Inf 36. Pirker A, Dür W (2019) A quantum network stack and protocols for reliable entanglement-based networks. New J Phys 21(3):033003 37. Qiang W, Zhongli Z (2011) Reinforcement learning model, algorithms and its application. In: 2011 International conference on mechatronic science, electric engineering and computer (MEC), pp 1143–1146. IEEE0 38. Roik J, Bartkiewicz K, ˇ Cernoch A, Lemr K (2024) Routing in quantum communication networks using reinforcement machine learning. Quantum Inf Process 23(3):89 39. Van Meter R, Satoh T, Ladd TD, Munro WJ, Nemoto K (2013) Path selection for quantum repeater networks. Netw Sci 3(1):82–95 40. Victora M, Tserkis S, Krastanov S, de la Cerda AS, Willis S, Narang P (2023) Entanglement purification on quantum networks. Phys Rev Res 5(3):033171 41. Yan PS, Zhou L, Zhong W, Sheng YB (2023) Advances in quantum entanglement purification. Sci China Phys Mech Astron 66(5):250301 42. Zapatero V, van Leent T, Arnon-Friedman R, Liu WZ, Zhang Q, Weinfurter H, Curty M (2023) Advances in device-independent quantum key distribution. npj Quantum Inf 9(1):10 Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. 123