Two-Stage Deep Q Learning Routing in Entanglement Networks

Two-Stage Deep Q Learning Routing in Entanglement Networks Diego Abreu∗, Arthur Pimentel∗, Christian Esteve Rothenberg†, Antˆ onio Abel´ em∗ ∗Federal University of Par´ a (UFPA), Bel´ em, Brazil. †Universidade Estadual de Campinas (UNICAMP), Campinas, Brazil. Abstract—Efficient routing in entanglement distribution networks is a challenging task that requires the coordinated allocation of quantum memories, repeaters, and quantum operations such as entanglement swapping and distillation across dynamic network conditions. In this paper, we present a Deep Q-Learning (DQN) Two-Stage Routing strategy designed to improve both path and quantum operations selection. Our approach enables the agent to make decisions over a diverse action space, including request scheduling, multipath routing, and the strategic application of quantum operations to maximize entanglement success and resource efficiency. We further extend DQN to integrate Double DQN, Dueling DQN, and Prioritized Experience Replay (PER) architectures. Simulation results indicate that our approach significantly outperforms baseline methods regarding request success rate, especially in large-scale networks under strict fidelity requirements. Index Terms—Quantum Network, Routing, Reinforcement Learning I. INTRODUCTION Quantum entanglement networks form the foundation of the future Quantum Internet, enabling applications such as Quantum Key Distribution (QKD), Distributed Quantum Computing, and Quantum Sensing [1]. In these networks, entanglement pairs (Bell pairs) are distributed through entanglement generation, distillation, and swapping operations. Efficient routing in entanglement distribution networks fundamentally differs from classical network routing due to the unique properties of quantum information [2]. In classical networks, data packets are transmitted through predefined paths, and intermediate nodes can store, copy, and forward information without altering its integrity. In contrast, quantum networks rely on the distribution of Bell pairs as the fundamental resource for quantum communication, which, unlike classical packets, cannot be copied due to the no-cloning theorem [3]. Bell pairs are generated over physical links connecting two adjacent nodes in a quantum network. However, due to environmental noise and imperfections in quantum hardware, the fidelity of these pairs—measuring their quality and correlation strength—degrades over time. To ensure reliable end-to-end quantum communication, entanglement distillation can be applied, where multiple lower-fidelity Bell pairs are combined to produce a higher-fidelity pair. Additionally, entanglement swapping is used to extend entanglement beyond direct neighbors: an intermediate node performs a quantum operation on two separate Bell pairs, creating an entangled connection between distant nodes. Because quantum states gradually lose fidelity due to decoherence, Bell pairs must be used promptly to avoid information loss, adding challenges to the routing process in quantum networks. These challenges require effective routing strategies that balance resource allocation, fidelity management, and network dynamics to maximize communication success [4]. In this paper, we propose the Deep Q-Learning (DQN) TwoStage Routing strategy, specifically designed for the unique demands of quantum networks. The agent learns to select routing paths and quantum operations based on the network’s current state. The proposed framework consists of two stages: (1) path selection, where the agent determines a route for entanglement distribution between source and destination nodes, and (2) action selection, where the agent chooses quantum operations (such as entanglement swapping and distillation) to apply along the selected path to improve entanglement fidelity and resource use. Our main contributions include the introduction of a twostage Deep Q-Learning framework that enables the agent to select both paths and quantum operations in quantum networks. We expand the action space to incorporate request scheduling, multipath strategies, and quantum operations such as entanglement swapping and distillation, while also comparing different learning architectures. Finally, through simulations, we show that the proposed approach enhances entanglement success rates in quantum network routing. The rest of this paper is organized as follows: Section II reviews related work on entanglement routing in quantum networks. In Section III, we describe the proposed two-stage Deep QLearning routing algorithm and discuss the DQN extensions and their implementation in the routing framework. Simulation results are provided in Section IV, followed by concluding remarks in Section V. II. RELATED WORKS Entanglement routing in quantum networks is a significant challenge, prompting research into strategies for efficient path selection to sustain high-fidelity quantum links [2]. A notable approach in this field is Q-CAST [5], which improves the success rates of end-to-end entanglement by employing multi-hop paths and redundancy strategies. Using Dijkstra’s algorithm for pathfinding, Q-CAST identifies weak points in the route and introduces alternative paths to prevent failures caused by the probabilistic nature of quantum operations. Building upon Q-CAST, the REPS [6] (Redundant Entanglement Provisioning and Selection) method introduces redundancy provisioning to further enhance network reliability. REPS strategically allocates backup entanglement resources based on their success probabilities, enabling higher communication success rates under resource constraints. Reinforcement learning has become a powerful tool in quantum network decision-making, with Islam et al. [7] introducing a proactive entanglement swapping strategy (Proactive DQN) that enables agents to anticipate future network demands, optimizing entanglement swaps on key segments to reduce resource overhead and improve success rates. Another significant approach, the Deep Quantum Routing Agent (DQRA) [8], utilizes deep reinforcement learning with a self-attention mechanism, embedding node, and channel features to enable more contextually aware routing decisions based on network topology and resource availability. Furthermore, Chaudhary et al. [9] proposed a learning-based route selection method for noisy quantum networks, using a multi-armed bandit framework to identify paths with minimal noise without prior knowledge of network conditions, significantly enhancing fidelity compared to distance-based methods and emphasizing the value of realtime adaptability in probabilistic environments. More recently, Abreu et al. [10] proposed qRL, a reinforcement learningbased routing protocol for quantum entanglement networks, demonstrating improvements in fidelity and request success rates across various network configurations. While qRL primarily focuses on path selection, our approach introduces a two-stage framework that jointly improves routing, entanglement swapping, and distillation strategies. Despite these advances, existing reinforcement learningbased strategies for entanglement routing face critical challenges, including inefficient resource allocation, poor adaptability to dynamic network conditions, and limited scalability in large quantum networks. Many approaches, such as qRL and Proactive DQN, focus primarily on path selection without dynamically optimizing quantum operations like entanglement swapping and distillation, leading to suboptimal routing decisions. Additionally, these methods often suffer from Q-value overestimation, which results in ineffective action selection under uncertain conditions. To address these issues, we propose a two-stage Deep Q-Learning routing strategy that jointly optimizes path selection, request scheduling, and quantum operations to maximize entanglement success and resource efficiency. By incorporating Double DQN, Dueling DQN, and Prioritized Experience Replay, our approach mitigates overestimation, improves learning stability, and prioritizes critical network states, resulting in higher success rates and better scalability in complex quantum networks. III. DEEP Q TWO-STAGE ROUTING In this section, we introduce our Deep Q Two-Stage Routing strategy, which leverages reinforcement learning to improve entanglement distribution in quantum networks. We first present the routing agent model, describing how the problem is formulated and how the agent interacts with the network environment. Next, we detail the two-stage decisionmaking process: in Stage 1, the agent selects both the routing path and request scheduling strategy, considering network conditions and resource availability. In Stage 2, the agent chooses quantum operations such as entanglement swapping and distillation to enhance fidelity and ensure successful entanglement distribution. We also discuss the reward function, which guides the learning process by balancing entanglement success, fidelity, and resource efficiency. A. Network System Model Efficient entanglement distribution in quantum networks requires a well-structured system that manages both physical and control operations. To implement the proposed method, the network is organized into two distinct planes: a physical plane and a digital control plane. The physical plane consists of quantum nodes, repeaters, and quantum channels responsible for entanglement generation, storage, and transmission. The digital control plane oversees network operations, monitors entanglement resources, and executes routing decisions based on real-time network conditions. This separation enables efficient coordination of quantum resources while adapting to network fluctuations. The network follows a fixed topology, where nodes generate requests for entanglement based on application demands. These requests specify a source-destination pair and require an end-to-end Bell pair with at least a given minimum fidelity. The primary challenge in this setting arises from the dynamic nature of quantum resources: while the network structure remains fixed, the availability of Bell pairs on each link and their fidelity fluctuate over time. This variability stems from ongoing entanglement usage, decoherence effects, and probabilistic quantum operations such as entanglement swapping and distillation. As the network handles multiple requests, entanglement links degrade and must be replenished or improved through active management strategies. Consequently, routing algorithms must dynamically adapt to the real-time state of the network to maximize the success rate of end-toend entanglement while efficiently allocating limited quantum resources. B. Overview of the Routing Agent The routing problem is defined as a decision-making process in which the agent interacts with the quantum network environment and selects actions based on the observed state. The network state includes information such as channel fidelity, the availability of Bell pairs, qubit capacity at nodes, and the list of pending entanglement requests. The agent continuously monitors these variables and selects appropriate actions to manage entanglement distribution effectively. One key aspect of the agent’s decision-making process is request scheduling, which determines the order in which entanglement requests are processed. Several scheduling strategies can be applied, each with different trade-offs. Shortest Job First (SJF) prioritizes smaller requests that require fewer entangled pairs. First-In-First-Out (FIFO) follows a strict arrival-time order. Earliest Deadline First (EDF) addresses time-sensitive demands by prioritizing requests with the closest deadlines. Highest Expected Throughput First (HEFT), adapted from Q-CAST, selects paths based on their expected throughput, which is computed using the fidelity values of individual links [11]. Beyond scheduling, the agent also decides when and how to perform entanglement swapping, which is essential for establishing long-distance entanglement links. Different swapping strategies impact both the success rate and the efficiency of entanglement distribution. Sequential Swapping, or Cascaded Swapping, processes swaps in a step-by-step manner from the source to the destination, but fidelity decreases exponentially with each additional swap. The effective capacity (C) in this approach is given by C=BellPairrate ×ES(N−1) p, where Nis the number of repeaters, BellPairrate is the Bell pair creation rate, and ESpis the swap success probability. On the other hand, Nested Swapping divides the path into multiple segments, performing swaps concurrently in smaller sub-sections before merging the entanglement path, reducing the overall effect of decoherence but requiring additional coordination. The scaling of Nested Swapping follows C= BellPairrate×ESlog N p, which offers a polynomial improvement over the sequential approach. Opportunistic Swapping, or swap ASAP (as soon as possible), initiates swaps as soon as resources become available, prioritizing faster entanglement establishment but risking lower fidelity due to prolonged waiting times. In addition to entanglement swapping, the agent determines the necessity and placement of entanglement distillation, which is performed to improve the fidelity of Bell pairs. Distillation can occur at different points along the entanglement path, such as locally at individual nodes, midway along the route, or end-to-end at the destination. The number of distillation rounds required depends on the initial fidelity of the Bell pairs and the target fidelity needed to satisfy the request. By carefully selecting where and how distillation is performed, the agent enhances the quality of entanglement while balancing resource consumption. Through this combination of request scheduling, entanglement swapping, and distillation, the routing agent makes informed decisions that help sustain high-fidelity entanglement links. C. Stage 1: Path and Scheduling Selection In the first stage, the agent selects both a path and a scheduling policy to allocate pending entanglement requests. The path selection process takes into account the current network state, including resource availability and channel conditions. The information about the network state can be obtained in two ways: periodically received from nodes, or through quantum network tomography techniques[12]. Based on the collected data, the agent determines the most suitable routing decisions. The agent first selects a scheduling policy among SJF, FIFO, EDF, or HEFT, considering network load, resource availability, and request urgency. The agent also determines whether to create alternative paths for entanglement distribution. If multipath routing is selected, the agent can choose between two approaches: REPS, which introduces redundancy to mitigate failures, or Q-CAST, which establishes concurrent multipath entanglement routes to enhance success probability. By selecting an appropriate scheduling policy and routing strategy, the agent improves the efficiency of entanglement request processing while enhancing the network’s capacity to sustain high-fidelity quantum links. D. Stage 2: Action Selection on Path Once the path (or paths) and scheduling strategy are determined, the second stage involves selecting quantum operations to enhance the fidelity and success rate of entanglement distribution. Since entanglement links degrade over time due to decoherence, the agent must decide which quantum operations to apply along the selected paths to sustain end-to-end fidelity. Entanglement swapping plays a crucial role in extending entanglement beyond direct neighbors. The agent selects a swapping strategy based on network conditions, choosing between sequential, nested, or opportunistic approaches, as discussed in Section III-B. In addition to swapping, distillation is applied to improve the fidelity of Bell pairs before they are used in quantum communication. The agent determines the placement of distillation, which can be performed locally at individual nodes, at intermediate points along the path, or in an end-to-end manner at the destination. The number of distillation rounds is also chosen based on the initial fidelity of the Bell pairs and the minimum fidelity required for successful communication. By dynamically selecting the appropriate quantum operations, the agent adapts to real-time network conditions, ensuring that entanglement transmission maintains the necessary fidelity while managing limited quantum resources efficiently. E. Reward Function The reward function is designed to guide the agent’s decisions by balancing entanglement success, fidelity improvements, and efficient use of network resources. The agent receives a reward for each request based on its success or failure, with a positive reward +θfor successful entanglement distribution and a penalty −θif the request fails. Additionally, fidelity improvement is rewarded when it exceeds the average fidelity Favg, granting +κif fidelity is higher than average, a neutral score if it matches, and a penalty −κif it falls below. Resource efficiency is another critical component. The agent is rewarded for minimizing the use of Bell pairs relative to the average usage BPavg; specifically, it earns +λif Bell pair usage is below the average, no reward if it matches, and −λ if usage exceeds BPavg. A similar approach applies to qubit usage, where the agent receives +δfor using fewer qubits than the average Qavg, no reward for equal usage, and −δif it surpasses Qavg. The overall reward function is computed as a weighted sum of these components, providing a balanced metric that enables the agent to account for entanglement success, fidelity, and resource efficiency in its routing decisions. This structure supports systematic improvements in entanglement distribution while adapting to the network state. F. DQN Extensions To improve the performance and stability of the baseline Deep Q-Learning algorithm for quantum network routing, we incorporate three key extensions: Double DQN and Dueling DQN [13], and Prioritized Experience Replay [14]. These enhancements address common limitations in standard DQN, including Q-value overestimation, inefficient sampling from the replay buffer, and the difficulty of distinguishing between state values and action advantages. Double DQN reduces Q-value overestimation by separating the action selection and evaluation processes. The policy Qnetwork selects the action with the highest Q-value, while the target Q-network evaluates its expected return. This separation prevents the agent from consistently overestimating future rewards, leading to more accurate learning. This refinement is particularly relevant in quantum networks, where unpredictable state transitions—such as fluctuations in channel fidelity and qubit availability—require careful assessment to avoid incorrect routing decisions. Dueling DQN is designed to handle scenarios where the specific action taken has little influence on long-term outcomes. It introduces two separate streams: the state-value stream, which estimates the overall benefit of being in a given state, and the advantage stream, which quantifies the impact of individual actions. This architecture allows the agent to better differentiate between inherently valuable states and those that require specific actions to improve network conditions. This distinction is particularly useful in quantum networks, where high-fidelity channels may naturally reduce the need for intervention, while resource-limited states demand precise routing choices. Prioritized Experience Replay improves the learning process by assigning higher priority to transitions that contain valuable learning signals. Instead of sampling experiences uniformly, the agent focuses more frequently on transitions with greater temporal difference errors, ensuring that critical experiences contribute more significantly to learning. This targeted approach accelerates convergence and enhances decision-making, particularly in environments with large stateaction spaces, such as quantum networks. By integrating these extensions, the proposed routing approach achieves more consistent learning, better adapts to the constraints of quantum entanglement distribution, and enables more reliable routing strategies. These enhancements conclude the proposed routing framework, laying the foundation for an agent capable of handling the challenges of quantum networks with greater accuracy and efficiency. IV. PERFORMANCE EVALUATION A. Simulation Settings To assess the effectiveness of our proposed method, we modeled the quantum network as a discrete-time simulation using a graph-based representation. The network topology for the experiment was a grid structure, where the nodes represent quantum devices capable of performing key quantum operations such as teleportation and entanglement swapping. Each node is equipped with quantum memory for storing qubits, while the edges represent the quantum communication links responsible for distributing entanglement between nodes. This setup allowed us to analyze the performance of the proposed two-stage Deep Q-Learning approach and its extensions. The quantum channels in the simulation follow a probabilistic model, where each link between two nodes is characterized by an initial fidelity value and a probability of successful Bell pair generation. The fidelity of a Bell pair degrades over time due to decoherence and successive entanglement operations, requiring distillation processes to restore fidelity levels. The entanglement swap success probability and distillation success probability are both set between 0.90 and 0.95. The experiments are configured with default parameters unless stated otherwise. Each node starts with a predefined number of qubits, ranging between 50% and 75% of the total node capacity. The quantum links begin with a set number of Bell pairs, ranging from 4 to 8 pairs per link, with initial fidelities between 0.95 and 0.99. The simulation proceeds in discrete time slots, during which additional qubits and Bell pairs can be generated on-demand as needed. The minimum fidelity required for a request to be considered successful, denoted as Fmin, ranges between 0.55 and 0.70. Each simulation episode generates a fixed number of requests, with source and destination nodes selected randomly across the grid network. Each request requires an end-to-end Bell pair with fidelity above Fmin for successful transmission. If the fidelity falls below this threshold, the request is considered unsuccessful. To account for resource availability, Bell pairs and qubits can either be pre-generated at the start of each episode or created during execution at fixed rates. The rates for Bell pair generation and qubit generation per time slot are set at 2 and 4, respectively, ensuring a continuous supply of quantum resources for teleportation, entanglement swapping, and distillation processes. B. Simulation Results The evaluation of the proposed Deep Q Two-Stage Routing method, along with its extensions (Double DQN, Dueling DQN, and Prioritized Experience Replay), was conducted in comparison with the baseline DQRA and Proactive DQN methods. The analysis focuses on the Request Success Rate, measured under different conditions, including request load, minimum fidelity requirements, entanglement swap probability, and network scale (number of nodes). The results are shown in Figure 1. Figure 1(a) presents the effect of increasing the number of entanglement requests per time slot. As expected, success rates 10 15 20 25 30 35 40 45 50 Requests per Timeslot 60.00 65.00 70.00 75.00 80.00 85.00 90.00 Success Rate (%) Success Rate vs Requests per Timeslot with Standard Deviation DQN D2QN DXDQN PER DQN ProactiveDQN DQRA (a) Requests per Time-Slot. 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 Fmin 65.00 70.00 75.00 80.00 85.00 90.00 95.00 Success Rate (%) Success Rate vs Fmin with Standard Deviation DQN PER DQN D2QN ProactiveDQN DXDQN DQRA (b) Fmin. 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 Swap Probability 65.00 70.00 75.00 80.00 85.00 Success Rate (%) Success Rate vs Swap Probability with Standard Deviation D2QN DQN ProactiveDQN PER DQN DXDQN DQRA (c) Swap/Purification Probability. 20 40 60 80 100 Network Scale 55.00 60.00 65.00 70.00 75.00 80.00 85.00 90.00 Success Rate (%) Success Rate vs Network Scale with Standard Deviation D2QN DQN ProactiveDQN PER DQN DXDQN DQRA (d) Network Scale (number of nodes). Fig. 1: Request Success Rate comparison under varying parameters. decline as network congestion increases due to resource limitations and the probabilistic nature of entanglement operations. However, Double DQN and DQRA maintain higher success rates compared to other methods, demonstrating their ability to allocate resources more effectively under high request loads. The significant drop in success rate observed for Proactive DQN and standard DQN suggests that these methods struggle to adapt to resource contention, leading to frequent request failures. The performance gap highlights the advantage of reinforcement learning models that incorporate action-value refinements, preventing overestimation of entanglement feasibility in high-load scenarios. Beyond handling congestion, the ability to sustain high request success rates is also influenced by the minimum fidelity requirement for entangled links. Figure 1(b) illustrates how increasing Fmin leads to a decline in success rates across all methods, as stricter fidelity thresholds reduce the number of usable entanglement links. Among all tested methods, Double DQN and PER DQN show the highest resilience to these constraints. Their performance advantage can be attributed to more effective path selection and scheduling of distillation operations, ensuring that high-fidelity Bell pairs are preserved for critical requests. In contrast, Proactive DQN and DQN exhibit the steepest decline, reinforcing their difficulty in managing environments where entanglement quality fluctuates significantly. The results suggest that methods with improved learning mechanisms, such as prioritization and value-function decomposition, are better suited for networks where fidelity constraints play a major role in routing. Another key factor influencing entanglement distribution success is the probability of entanglement swapping. Figure 1(c) shows that as the swap success probability increases, all methods benefit from improved request success rates. However, Double DQN and Prioritized Experience Replay continue to outperform other approaches, particularly in scenarios with lower swap success probabilities. Their ability to select more reliable entanglement paths and anticipate failure risks contributes to their advantage. In contrast, Dueling DQN and Proactive DQN show significant performance degradation at lower swap success probabilities, indicating that they struggle to compensate for weak entanglement links. These findings emphasize the importance of reinforcement learning models that account for operational uncertainty when routing entanglement, ensuring that the selected paths can sustain endto-end entanglement even in challenging conditions. The scalability of routing strategies is crucial as quantum networks expand. Figure 1(d) shows that certain methods—particularly Double DQN (D2QN) and DQRA—achieve higher request success rates as the number of nodes increases. This improvement can be attributed to the greater diversity of available paths in larger networks, which increases the chances of finding routes that meet fidelity requirements, even if the end-to-end distance is longer. Advanced reinforcement learning models are able to exploit this expanded set of routing options, selecting viable paths that balance fidelity and resource availability. Likewise, PER DQN and DXDQN maintain strong performance as the network scales, demonstrating robustness in more complex topologies. On the other hand, standard DQN and Proactive DQN show limited improvements, suggesting that their decision-making processes are less effective in leveraging the increased flexibility provided by larger networks. These results reinforce the advantage of models equipped with value decomposition, prioritization, and other enhancements that enable efficient routing in scalable quantum infrastructures. Overall, the findings confirm that Double DQN and PER DQN consistently outperform other methods across different network conditions. Their ability to handle congestion, stricter fidelity constraints, fluctuating swap probabilities, and increasing network scales demonstrates the effectiveness of reinforcement learning techniques that refine action selection and prioritize key transitions during training. The results emphasize that more advanced learning mechanisms are essential for ensuring reliable entanglement distribution, particularly in resource-limited quantum networks. The scalability and robustness of these methods position them as strong candidates for real-world quantum networking applications. V. CONCLUSION In this paper, we introduced a Deep Q-Learning Two-Stage Routing algorithm for optimizing entanglement distribution in quantum networks. The proposed approach jointly addresses path selection and the scheduling of quantum operations such as entanglement swapping and distillation, allowing for adaptive decision-making in the presence of probabilistic entanglement generation and resource constraints. To enhance learning stability and decision-making accuracy, we integrated Double DQN, Dueling DQN, and Prioritized Experience Replay, which led to improved handling of network uncertainties. Simulation results demonstrated that the proposed framework outperforms baseline methods in terms of request success rate, particularly under high request loads, stricter fidelity requirements, and varying swap success probabilities. Future work may explore additional enhancements, including more advanced scheduling policies that dynamically adapt to network conditions and further integration of multi-agent learning techniques to coordinate routing decisions across distributed quantum nodes. These extensions could further improve the robustness and scalability of quantum network routing, paving the way for practical implementations in realworld quantum communication systems. ACKNOWLEDGEMENTS This work was partially funded by the National Council for Scientific and Technological Development (CNPq) under Grant 405940/2022-0, 444978/2024-0, the Coordination for the Improvement of Higher Education Personnel (CAPES), and the S˜ ao Paulo Research Foundation (FAPESP) under projects 2023/00811-0, 2023/00673-7, 2021/00199-8 (CPE SMARTNESS), 2020/04031-1, and 2018/23097-3. REFERENCES [1] K. Azuma, S. E. Economou, D. Elkouss, P. Hilaire, L. Jiang, H.-K. Lo, and I. Tzitrin, “Quantum repeaters: From quantum networks to the quantum internet,” Reviews of Modern Physics, vol. 95, no. 4, p. 045006, 2023. [2] A. Abane, M. Cubeddu, V. S. Mai, and A. Battou, “Entanglement routing in quantum networks: A comprehensive survey,” IEEE Transactions on Quantum Engineering, pp. 1–36, 2025. [3] S. Wehner, D. Elkouss, and R. Hanson, “Quantum internet: A vision for the road ahead,” Science, vol. 362, no. 6412, p. eaam9288, 2018. [4] A. S. Cacciapuoti, M. Caleffi, F. Tafuri, F. S. Cataliotti, S. Gherardini, and G. Bianchi, “Quantum internet: Networking challenges in distributed quantum computing,” IEEE Network, vol. 34, no. 1, pp. 137–143, 2019. [5] S. Shi and C. Qian, “Concurrent entanglement routing for quantum networks: Model and designs,” in Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication, 2020, pp. 62–75. [6] Y. Zhao and C. Qiao, “Redundant entanglement provisioning and selection for throughput maximization in quantum networks,” in IEEE INFOCOM 2021-IEEE Conference on Computer Communications. IEEE, 2021, pp. 1–10. [7] T. Islam, M. Arifuzzaman, and E. Arslan, “Reinforcement learning based proactive entanglement swapping for quantum networks,” in 2024 International Conference on Quantum Communications, Networking, and Computing (QCNC). IEEE, 2024, pp. 135–142. [8] L. Le and T. N. Nguyen, “Dqra: Deep quantum routing agent for entanglement routing in quantum networks,” IEEE Transactions on Quantum Engineering, vol. 3, pp. 1–12, 2022. [9] V. Chaudhary, K. Li, and K. Chowdhury, “Learning-based route selection in noisy quantum communication networks,” in ICC 2023-IEEE International Conference on Communications. IEEE, 2023, pp. 4188– 4193. [10] D. Abreu and A. Abel´ em, “qrl: Reinforcement learning routing for quantum entanglement networks,” in 2024 IEEE Symposium on Computers and Communications (ISCC), 2024, pp. 1–6. [11] S. Shi, X. Zhang, and C. Qian, “Concurrent entanglement routing for quantum networks: Model and designs,” IEEE/ACM Transactions on Networking, 2024. [12] M. G. De Andrade, J. Navas, S. Guha, I. Monta˜ no, M. Raymer, B. Smith, and D. Towsley, “Quantum network tomography,” IEEE Network, 2024. [13] M. Sewak and M. Sewak, “Deep q network (dqn), double dqn, and dueling dqn: A step towards general artificial intelligence,” Deep reinforcement learning: frontiers of artificial intelligence, pp. 95–108, 2019. [14] T. Schaul, J. Quan, I. Antonoglou, and D. Silver, “Prioritized experience replay,” 2016. [Online]. Available: https://arxiv.org/abs/1511.05952