A Deterministic Scheduler for Hybrid Wired and Wireless Network Resources over Cloud-Edge Continuum Computing Platform

A Deterministic Scheduler for Hybrid Wired and Wireless Network Resources over Cloud-Edge Continuum Computing Platform Kundjanasith Thonglek†∗, Chonho Lee†, Hirotake Abe†, Arata Endo†, Kohei Taniguchi†, Takahiro Hirofuchi‡, Ryousei Takano‡, and Susumu Date† †Osaka University, Japan ‡National Institute of Advanced Industrial Science and Technology, Japan ∗Email: [email protected] In the current era of the fifth-generation (5G) network technology, characterized by ultra-low latency networks, there is an anticipation that 5G will provide robust support for emerging digital twin applications requiring time-sensitive traffic management [1]. These applications, traversing interdisciplinary domains such as factory automation, autonomous vehicles, remote medicine, and immersive environments, highlight the diverse and extensive impact of cutting-edge 5G networks. The handling of time-sensitive traffic in these scenarios involves strict observance of quality-of-service (QoS) requirements, specifically addressing objectives such as delay, reliability, and jitter [2]. It is noteworthy that these QoS requirements diverge from the goals of previous network generations, which were primarily focused on enhancing data rates. Achieving ultra-low latency in 5G networks involves implementing sophisticated techniques, including seamless integration of cloud and edge resources. By deploying computing resources closer to the network edge and seamlessly integrating cloud services, the distance data needs to travel is minimized, resulting in significantly reduced latency [3]. For instance, in autonomous vehicles, edge computing allows realtime decision-making at the vehicle level, improving response times for critical tasks like collision avoidance. Rather than transmitting data directly to the cloud, we leverage an edge server to offload certain tasks, resulting in faster response times delivered directly to the edge device and minimizing latency. This technique indicates how installing computing resources contributes to achieving ultra-low latency, enhancing the overall performance of 5G networks. However, deploying additional computing resources proves insufficient due to the allocation and scheduling of limited network resources. Therefore, an efficient scheduling approach for network resources becomes imperative to ensure the allocation of adequate network resources for each computing resource, both on the edge and cloud sides, all within the designated time constraints. Through advanced scheduling algorithms, network can prioritize and allocate resources efficiently, ensuring that time-sensitive tasks, like those in autonomous systems, receive immediate attention. The network scheduling is essential for mitigating transmission delays, crucially addressing the diverse QoS requirements. Existing network resource schedulers, such as proportional fair, round-robin, earliest-deadline-first, and maximum throughput, lack a specific focus on addressing the needs of time-sensitive traffic. This insufficiency serves as our motivation to design a novel network scheduling approach tailored to meet QoS requirements for time-sensitive traffic in 5G networks, encompassing both wired and wireless connections. While there have been advanced in developing schedulers for time-sensitive traffic in wired networks through the timesensitive networking (TSN) standardization [4], there remains a critical space in addressing similar requirements for wireless connections. Specht et al. [5] advanced this field with an urgency-based scheduler that allocates hard real-time data flows to queues, impacting latencies significantly. Since ultra-reliability and low latency (URLLC) in 5G networks facilitates performance levels related to wireless networks, thereby unlocking new possibilities for diverse applications. Khoshnevisan et al. designed of end-to-end 5G wireless networks specifically tailored for industrial factory automation [6]. Moreover, Gintor et al. analyzed the technical challenges for implement time-sensitive traffic over wireless connection in 5G [7]. Their work proposed a simulator to investigate the impact and defining QoS for the system. Despite these advancements, there is currently a lack of an optimal scheduler for hybrid wired and wireless network resources. Therefore, we propose the development of a deterministic scheduler tailored for hybrid wired and wireless network resources on a cloud-edge continuum computing platform as shown in Fig. 1. This scheduler leverages 5G network technology, where ax(t)denotes the action on resource xat time t,sx(t)represents the state of network resource xat time t, and rx(t)is the reward from resource xat time tfor action ax(t)in state sx(t). The set τx(t)includes transitions, pairing sx(t)and ax(t)up to time t. Parameters µand Qcorrespond to the actor and critic models. The scheduling problem in hybrid wired and wireless networks can be framed as an optimal control problem within a Markov Decision Process (MDP), making it suitable for addressing with reinforcement learning [8]. Three challenges can be identified across phases including problem formulation, training algorithm, and online implementation. Wireless Network Wired Network Edge Device ( ) Scheduler for Wired Network on Cloud ( ) Simulate wired network communication Cloud models Short-Term memory ( ) Short-term memory ( ) Scheduler for Wireless Network on Edge Server ( ) Simulate wireless network communication Edge models Short-term memory ( ) Short-term memory ( ) Actor Critic The configuration of wired network for each edge server The configuration of wireless network for each edge device Edge Device ( ) Actor Critic Replay memory Long-Term memory ( ) Reward shaper Replay memory Evaluation of long-term reward Action of the given states Policy Wired Network Wireless Network Cloud Scheduler for Wireless Network on Edge Server ( ) Fig. 1. The architecture of proposed deterministic scheduler for hybrid wired and wireless network resources over cloud-edge continuum computing platform We will thoroughly investigate and address these issues in our proposed scheduler. Recently, actor-critic deep reinforcement learning algorithms have emerged to tackle these challenges. These algorithms involve two neural networks approximating the policy and the long-term reward, respectively [9]. In cases where the optimal policy is deterministic, as is often the case in optimal control problems, these algorithms transform into deep deterministic policy gradient (DDPG) methods [10]. Leveraging the DDPG concept, we aim to enhance and apply it to efficiently schedule network resources. As part of our future work, we aim to create a network scheduler for both wired and wireless networks. For the wireless part, we will focus on considering what each user or device needs in QoS. Similarly, when scheduling wired connections from the cloud, we will factor in QoS from network providers or edge servers. The ultimate goal is to blend these wired and wireless networks into a single scheduler that ensures applications receive the service quality they need. This combined scheduler will play a key role in smoothly transmitting time-sensitive data from edge devices to the cloud. ACKNOWLEDGMENT This paper is based on results obtained from “Research and Development Project of the Enhanced Infrastructures for Post5G Information and Communication Systems” (JPNP20017), commissioned by the New Energy and Industrial Technology Development Organization (NEDO). REFERENCES [1] S. Ono, T. Yamazaki, T. Miyoshi, A. Taya, Y. Nishiyama, and K. Sezaki, “AMoND: Area-controlled mobile ad-hoc networking with digital twin,” IEEE Access, vol. 11, pp. 85 224–85 236, 2023. [2] A. Garbugli, A. Sabbioni, A. Corradi, and P. Bellavista, “TEMPOS: Qos management middleware for edge cloud computing faas in the internet of things,” IEEE Access, vol. 10, pp. 49 114–49 127, 2022. [3] D. Milojicic, “The edge-to-cloud continuum,” Computer, vol. 53, no. 11, pp. 16–25, 2020. [4] A. Nasrallah, A. S. Thyagaturu, Z. Alharbi, C. Wang, X. Shao, M. Reisslein, and H. ElBakoury, “Ultra-low latency (ull) networks: The ieee tsn and ietf detnet standards and related 5g ull research,” IEEE Communications Surveys & Tutorials, vol. 21, no. 1, pp. 88–145, 2019. [5] J. Specht and S. Samii, “Synthesis of queue and priority assignment for asynchronous traffic shaping in switched ethernet,” in 2017 IEEE Real-Time Systems Symposium (RTSS), 2017, pp. 178–187. [6] M. Khoshnevisan, V. Joseph, P. Gupta, F. Meshkati, R. Prakash, and P. Tinnakornsrisuphap, “5g industrial networks with comp for urllc and time sensitive network architecture,” IEEE Journal on Selected Areas in Communications, vol. 37, no. 4, pp. 947–959, 2019. [7] D. Ginth¨ or, J. von Hoyningen-Huene, R. Guillaume, and H. Schotten, “Analysis of multi-user scheduling in a tsn-enabled 5g system for industrial applications,” in 2019 IEEE International Conference on Industrial Internet (ICII), 2019, pp. 190–199. [8] C. She, R. Dong, Z. Gu, Z. Hou, Y. Li, W. Hardjawana, C. Yang, L. Song, and B. Vucetic, “Deep learning for ultra-reliable and lowlatency communications in 6g networks,” IEEE Network, vol. 34, no. 5, pp. 219–225, 2020. [9] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed. The MIT Press, 2018. [Online]. Available: http://incompleteideas.net/book/the-book-2nd.html [10] H. Tan, “Reinforcement learning with deep deterministic policy gradient,” in 2021 International Conference on Artificial Intelligence, Big Data and Algorithms (CAIBDA), 2021, pp. 82–85.