Cloud-Native Scheduling and Resource Orchestration: A Deep Dive into AI-Driven Approaches

Cloud-Native Scheduling and Resource Orchestration: A Deep Dive into AI-Driven Approaches Tomás Dias1, Luís Ferreira1, Diogo Fevereiro1, Luis Rosa1, Luis Cordeiro1, and João Fernandes1 OneSource, Coimbra, Portugal {tomas.dias,luis.ferreira,duarte.fevereiro,luis.rosa,luis.cordeiro,joao.fernandes} @onesource.pt Abstract. Cloud-native computing has transformed modern application development, deployment, and management by enabling scalability and flexibility. However, the increasing complexity of workloads and dynamic resource demands challenge traditional scheduling and resource provisioning techniques, often leading to inefficiencies. This paper explores AI-driven approaches to optimizing cloud-native scheduling and resource provisioning. By leveraging machine learning, deep reinforcement learning, and predictive analytics, AI enhances decision-making, automates scaling, and improves workload distribution. We present a comprehensive review of recent AI techniques applied to container orchestration, and Kubernetes-based scheduling, analyzing their impact on cost reduction, performance optimization, and resource efficiency. Additionally, we discuss key challenges such as model interpretability, real-time adaptability, and integration with existing cloud and edge infrastructures. Ultimately, this paper provides insights into the future of intelligent cloud and edge resource management, emphasizing the necessity of AI-augmented strategies to meet the growing demands of nextgeneration applications. Keywords: Cloud-native computing ·AI-driven scheduling ·Resource provisioning ·Machine learning ·Kubernetes ·Serverless computing 1 Introduction The cloud-native approach has become one of the most widely used strategies for building and hosting microservice-based applications. Such applications are more scalable, portable, and cost-effective than those based on traditional monolithic architectures. However, their lifecycle requires careful monitoring and management. Orchestration frameworks such as Kubernetes [3], Docker Swarm [4], and Cloudify [5] include tools to manage cloud and edge workloads and provide features such as scheduling, load balancing, and auto-scaling. Such features rely on standard algorithms, which are based on heuristics (scheduling) and thresholds (scaling, migration). Currently, AI techniques are being explored [32] and 2 Dias, T. et al. integrated for monitoring and management of 5G cloud-native and edge environments. By leveraging AI-driven intelligence, critical decisions can be automated, such as provisioning the necessary infrastructure and optimising component placement for performance, cost, and efficiency. Artificial Intelligence (AI) plays a crucial role in enhancing the capabilities of orchestration frameworks. The ETSI ZSM standard [1] aims to automate network management, enabling the creation of highly autonomous networks. These networks can self-configure, self-monitor, self-heal, and self-optimise without needing further human intervention. In this regard, this paper reviews AI-driven techniques for optimising resource orchestration in edge and cloud environments, with a particular focus on Kubernetes-based scheduling. Unlike existing studies, which often analyze AI applications in isolation, this work provides a comprehensive, comparative assessment of multiple AI methodologies, highlighting their impact on cost reduction, performance optimization, and resource efficiency. This research offers valuable insights for academia and industry, paving the way for more intelligent, autonomous cloud management solutions. The remainder of the paper is structured as follows: Section 2 revisits the role of AI in resource orchestration. Section 3 reviews the literature on AIbased resource provisioning. Section 4 presents the scheduling as a multi-criteria optimization problem. Section 5 draws the conclusions. 2 AI-driven Cloud-Native Resource Orchestration Cloud-native applications commonly utilise a microservices architecture, where their deployment requires managing a complex, distributed system. Despite the advantages of microservice-based architectures, these benefits come with challenges, including their orchestration. Service orchestration is crucial for efficiently coordinating and managing interactions among microservices. It facilitates effective scaling, deployment, and maintenance of complex systems [33]. This includes challenges related to the deployment, scalability, scheduling, management, and networking of microservices. Moreover, ETSI ZSM discusses the necessity of using closed-loop concepts to build autonomous systems that can be self-managed, removing human error. One subjacent idea is the leverage of AI/ML into such workflows. The concept of self-management through closed loops such as OODA loops, which stands for “observe, orient, decide, act”, was first created by the military strategist Colonel John Boyd. The relevance of the loops was then discovered by other fields and can be applied in the context of cloud-native orchestration. AI-driven resource orchestration sub-fields can be summarized as follows [34]. Scheduling consists of deciding the initial placement of a containerised task unit. Scheduling methods may be aware of the application architecture and task structure. Scaling comprises the ability to scale containerised applications or compute nodes in response to fluctuations in workloads, ensuring that applications have the resources to fulfil their resource requirements (e.g. memory, CPU, storage). Migration involves moving one or more tasks from one node to another. It consists of two key concepts: rescheduling and offloading. Rescheduling AI-Driven Cloud-Native Orchestration and Resource Provisioning 3 refers to relocating tasks between nodes for load balancing or offloading computeintensive tasks in hybrid clouds. This is often necessary due to poor scheduling decisions, which can result in either resource overload or underload. 3 Literature Review 3.1 AI in Edge and Cloud Native AI/ML techniques are increasingly applied in cloud-native environments to automate tasks, predict resource needs, and enhance security. ML’s ability to handle large data volumes and uncover insights through iterative learning makes it a key tool for decision-making. Such techniques are divided into Generative AI, which creates new content and Predictive AI, which forecasts outcomes [20–22]. Other sub-fields of AI have also been investigated for their potential to address a range of optimisation challenges. For instance, evolutionary computation is a subfield where algorithms are inspired by biological evolution. One branch within this family is Swarm Intelligence (SI), which has been the subject of research due to its effectiveness in addressing a diverse range of optimisation problems, including those related to scheduling. These algorithms are based on the collective behaviour observed in organised groups of animals, such as birds, ants or bees, which strive to survive collectively [23]. RL have been attracting researchers’ interest in the scaling field. Khaleq et al. [29] proposed an approach to improve Kubernetes auto-scaling that uses RL agents to learn and identify the auto-scaling threshold values based on the resource demand and Quality of Service (QoS). This approach showcased that auto-scaling based on the pod resource could lower the response time by a factor of 20% compared to default K8s CPU-based auto-scaling (HPA). To achieve that factor, a simulation environment was created in MATLAB [8] to train various RL algorithms, including Q-learning, Deep Q-Network, SARSA, among others. The goal was to evaluate their performance in that environment in two different scenarios: CPU-Intensive and Memory-Intensive. Although Reinforcement Learning (RL) is often referenced in the literature, other ML algorithms can have a crucial role in cloud-native orchestration. A proof of concept by [30] demonstrates that LSTM-based proactive auto-scaling can improve end-to-end latency for cloudnative applications. This approach uses the Digital Ocean public cloud platform [9], which provides management options for Kubernetes master nodes [30]. Dang-Quang and Yoo [31] combined the strengths of the two families, Deep RL, which integrates RL with Recurrent Neural Network (RNN) capabilities of time-analysis. The authors proposed a Proactive Custom Auto-scaler (PCA) framework focused on the analysis and planning phases. For the former was selected a Bi-LSTM which extends the LSTM models applying two LSTMs to the input data - the first trained on the input sequence in its original direction (forward) and the second on the reverse form of the input sequence (backward). The experimental phase was conducted using two different datasets for training and evaluation of models. Following this, Dang-Quang and Yoo [31] developed a simulated environment using Python [10] and ran on the Google Colab environment 4 Dias, T. et al. [11] to assess resource provision accuracy and elastic speed-up. In this evaluation, Bi-LSTM demonstrated the highest overall accuracy and predicted error. Finally, Dang-Quang and Yoo proposed an architecture based on K8s and Proactive Custom Auto-scaler which was compared with the Kubernetes auto-scaler (HPA). The results showcased that PCA outperformed the Kubernetes auto-scaler in accuracy, speed and also efficiency, particularly when dealing with a burst of workload. Lin et al. [24] and Chen and Xiao [25] explored two SI algorithms, ACO and PSO, to optimise microservices container-based scheduling and over-perform the Spread algorithm of Docker Swarm [4]. To validate the proposed algorithm, both used real data from the Alibaba Cluster TraceV2018 cluster dataset in the experiments [6]. Lin et al. [24] proposed Ant Colony Optimization algorithm for Multi-objective of Container-based Microservice Scheduling (ACO_MCMS), a ACO multi-objective optimisation algorithm, to enhance container-based microservice scheduling. This algorithm combines multi-objective heuristic information with a feedback mechanism to avoid falling into the local optimal solution. ACO_MCMS tried to achieve three objective improvements: reducing the network transmission overhead among microservices, balancing the cluster load, and improving cluster services’ reliability. The authors compared the algorithm with different multi-objective algorithms such as GA-MOCA [26], the Docker Swarm spread algorithm and Multiopt [27]. The ACO_MCMS successfully met all the proposed objectives and demonstrated the best overall performance in all categories. Chen and Xiao [25] analysed two container-based microservice scheduling multi-objective optimization algorithms - (GA-MOCA) [26] and ACO_MCMS [24]) - highlighting their drawbacks on large network transmission costs, unbalanced clusters and individual loads, and long optimization times. To address these challenges, the authors proposed MOPPSO-CMS, a multi-objective optimization algorithm based on PSO. Further evaluation showcased that MOPPSO-CMS outperformed the previous approaches in terms of network transmission overhead, local and global load balancing, standard deviation of cluster resources, service reliability, and algorithm running speed. Santos et al. [28] proposed an RL-based Global Topology Manager (GTM) for the deployment of applications in multi-cluster infrastructure. The RL algorithm uses a Deep Set (DS) approach. DS is a family of neural networks tailored for processing data-structured assets. Once trained, DS networks can perform inference to sets of arbitrary size without the need for retraining [2]. In order to facilitate the training of the reinforcement learning algorithm, an OpenAI Gym-based framework [7] was developed to enhance scalability and cost-effectiveness during the training process [28]. The developed framework demonstrates potential advantages for training RL algorithms in this environment. 3.2 Challenges-behind AI Swarm Intelligence Over the past year, SI has also emerged as a significant case study in optimization problems [23]. These algorithms are classified as a series of heuristic, nature-inspired methods used to solve optimization problems where mathematical or traditional approaches are inadequate. They rely AI-Driven Cloud-Native Orchestration and Resource Provisioning 5 on stochastic search methods that iteratively exchange heuristic information to enhance subsequent iterations [23]. Brezočnik et al. [35] highlighted the benefits of SI and provided an overview of its framework. One key advantage is autonomy, where each agent in the swarm independently controls its behaviour without the need for centralized management. Another benefit is self-organisation, where each agent represents a potential solution to a problem. While the collective approach of algorithms leads to solutions that emerge from the swarm, it does not solely focus on individuals. Additionally, scalability represents a benefit as each agent monitors its behaviour and adjusts the solution based on the swarm, ensuring there is no single point of failure. These factors allow the swarm to consist of a few to up to thousands of agents without changing the control architecture. Despite recognizing that SI algorithms offer an alternative and unconventional approach to designing complex systems without centralized control or extensive pre-programming, Ahmed and Glasgow [36] points out several limitations of this approach. One significant limitation is that SI algorithms are unsuitable for time-critical applications requiring real-time system control, prompt decision-making, and adequate solutions within strict time constraints. This is because the solutions produced by SI systems are emergent and not pre-programmed. Additionally, parameter tuning is a significant drawback of SI approaches. The parameters needed for these algorithms are often problemdependent and must be either empirically pre-selected based on trial and error or adaptively adjusted during execution. Another limitation is the potential for stagnation or premature convergence to local optima, which can occur due to the lack of central control. Brezočnik et al. [35] define the standard SI workflow as a sequence of tasks: Initialisation population; Definition of stop condition; Evaluation of fitness function; Update and move agents; Return the global best solution. The first task involves defining the initial candidate solutions, referred to as the Swarm. The next step is to evaluate these solutions after each iteration. In addition to defining the initial candidate solutions, it is essential to establish a stopping condition. If the stopping condition is not met, a new iteration will occur, requiring an update of the Swarm and its agents. Once the condition is met, the algorithm should return the global best solution. Yang et al. [37] explored the key challenges in the development of SI algorithms. The first challenge outlined is a limitation mentioned by Ahmed and Glasgow [36], specifically concerning parameter tuning and control. As previously mentioned, the algorithms have interrelated parameters, and appropriately setting these parameters can significantly impact their performance. Tuning these parameters is essential for maximizing the effectiveness of the algorithm. Typically, tuning approaches involve parametric studies, while parameter control uses stochastic adaptivity, allowing specific parameters to vary randomly within a predefined range. In conclusion, Ahmed and Glasgow [36] argue that it is crucial to have an effective method for parameter tuning that is both automatic and adaptive. The second challenge relates to finding the right balance between optimal exploration and exploitation. Excessive exploration combined with insufficient exploitation can slow down the search process, while too much exploitation with insufficient 6 Dias, T. et al. exploration can lead to premature convergence. Identifying the optimal balance can be difficult, and empirical observations suggest this balance may be problemspecific [37]. Another challenge is scalability. Current literature on optimization problems often deals with a variable count ranging from just a few to a few hundred, which is relatively low compared to real-world applications. Furthermore, there is no unified framework that provides a comprehensive view of an algorithm concerning convergence, rate of convergence, stability, ergodicity, repeatability, and scalability [37]. Lastly, controlling the rate of convergence is crucial because we want to find the best solution to a problem quickly and with minimal computational costs. The authors argue that it can be challenging to maximize search efficiency in this regard [37]. Machine Learning On the other hand, Paleyes et al. [38] examined the challenges associated with ML deployment, utilizing the workflow proposed by Ashmore et al. [39]. Ashmore et al. [39] categorize the deployment process of an ML-based solution in the industry into four stages: Data Management, Model Learning, Model Verification, and Model Deployment. Data management focuses on data preparation and consists of four sub-stages: Data collection, Data preprocessing, Data augmentation and Data analysis. Paleyes et al. [38] identified the main challenges in the data management phase as discovering and understanding what data is available, as well as how to organize convenient storage for it in the data collection sub-stage. In the data analyses sub-stage, the challenge is behind the visualisation of data profiling because there are still too few tools that enable the efficient execution of these data mining tasks [39]. In the Data preprocessing sub-stage, Emmanuel et al. [40] analysed the literature techniques for handling missing values and made a comparison between two imputation methodsk-Nearest-Neighbor (KNN) and an iterative imputation method based on the random forest neighbour. The imputation process involves filling in missing values with predicted values. Techniques for imputation can be categorised into various types, such as simple methods, regression methods, and those inspired by machine learning. Among these, regression is one of the preferred techniques [40], as it replaces missing values with predictions generated from a regression model. KNN is a technique based on ML that classifies the nearest neighbours of missing values and uses those neighbours for imputation, applying a distance measure between instances. According to Emmanuel et al. [40], the data type analysed influences the precision and accuracy of machine learning imputation algorithms. There is no clear evidence favouring one method over another because the results may vary depending on the evaluation metrics used, such as the Mean Absolute Error (MAE), Mean Squared Error (MSE) and Root Mean Square Error (RMSE). According to the literature, RMSE is commonly used as an evaluation metric [40]. However, Emmanuel et al. [40] point out that a limitation in the literature is the failure to use metrics in conjunction. The three most common metrics use the number of samples in the dataset m, the actual value of the ith sample yiand the predicted value of the ith sample, ˆyi. AI-Driven Cloud-Native Orchestration and Resource Provisioning 7 Although there is a difference between them in how they calculate the values, the formulas are described below Mean Absolute Error (MAE): measures the average difference between imputed values yiand true values ˆyi, defined as: MAE =1 m m X i=1 |yi−ˆyi|(1) Mean Squared Error (MSE): It is equal to the sum of variance and the squared predicted missing values, defined by the following equation: MSE =1 m m X i=1 (yi−ˆyi)2(2) Root Mean Square Error (RMSE): computes the difference between imputed value applying a root on the MSE. RMSE =v u u t 1 m m X i=1 (yi−ˆyi)2(3) Zebari et al. [41] explore dimensionality reduction techniques for feature selection and feature extraction. The authors analysed the advantages and constraints of dimension reduction techniques like feature extraction and selection. Feature extraction methods produce a dataset that is a linear combination of features, while feature selection methods yield a sub-list of the most relevant features [41]. Both methods offer distinct advantages. Feature selection helps maintain the original dataset while reducing its size, whereas feature extraction preserves the relative distances between features and retains the inherent structure of the data. However, each method has its limitations. Feature extraction may not be effective if there are too many irrelevant features, while feature selection can struggle if the underlying structure of the data is not adequately represented. Sarker et al. [42] analysed the data augmentation sub-stage in terms of dimensionality reduction. The authors argue that dimensionality reduction is essential because it leads to better human interpretation, lower computational cost and the avoidance of over-fitting and redundancy. Based on machine learning and data science literature, the authors provided an overview of the most commonly used techniques for feature selection, such as Pearson Correlation, Analysis of variance (ANOVA), and feature extraction techniques, such as Principal component analysis (PCA). Pearson Correlation provides insight into the relationship between the two features. The resulting correlation coefficient can range from -1 to 1. A negative value indicates a negative correlation, while a positive value signifies a positive correlation. A value of 0 indicates that the two variables do not have a linear correlation. For two features, X and Y, the correlation coefficient between them is defined as [48]: r(X, Y ) = Pn i=1(Xi−¯ X)(Yi−¯ Y) qPn i=1(Xi−¯ X)2qPn i=1(Yi−¯ Y)2 (4) 8 Dias, T. et al. The symbol r(X, Y )represents Pearson’s correlation coefficient between the variables Xand Y. The symbols Xiand Yidenote individual data points of the variables Xand Y, while ¯ Xand ¯ Yrepresent their respective means (averages). The total number of data points is given by n. Additionally, the sum of the product of deviations from the mean is expressed as Pn i=1(Xi−¯ X)(Yi− ¯ Y). The denominator of the correlation coefficient formula includes the terms qPn i=1(Xi−¯ X)2and qPn i=1(Yi−¯ Y)2, which correspond to the standard deviation components of Xand Y, respectively. ANOVA is a statistical tool used to determine whether there are significant differences between the mean values of two or more groups. This method assumes a linear relationship between the variables and the target, as well as a normal distribution of the variables. ANOVA tests can be classified as either one-way or two-way, depending on the number of variable groups being analysed [47]. The one-way formula is presented below [47] F∗= k P j=1 nj(Mj−M)2 k P j=1 1−nj Ns2 j (5) The symbol krepresents the number of groups. The symbol njdenotes the number of observations in group j. The mean of the group jis represented by Mj, while Mdenotes the overall mean. The total number of observations is given by N. Finally, s2 jrepresents the variance of group j. PCA is a feature extraction method that transforms a set of correlated variables into a set of uncorrelated variables. It is an unsupervised learning technique commonly used in the MLfield [48]. PCA identifies the components with the highest eigenvalues from a covariance matrix and uses these components to project the data into a new subspace, which can have an equal number of dimensions or fewer. The resulting dimensions can vary from one-dimensional (1D) to three-dimensional (3D). In the Model learning stage, the focus is on model selection and training. This stage includes three sub-stages: Model Selection, Training and Hyper-parameter selection. In the model training, Sharir et al. [43] observe that the training dataset size, number of model parameters, and number of operations used by the training procedure all contribute towards the overall cost. The mentioned factors are a challenge that all model implementations must be aware of to avoid expensive development. During the model learning stage, Yu et al. [44] reviewed existing hyper-parameters and highlighted their importance in algorithms. Yu et al. analysed hyper-parameters such as learning rate and optimizer, as well as those related to model design. Additionally, Yang et al. [45] argue that hyperparameter tuning approaches require users to define the entire search space. Setting hyper-parameter optimization remains a challenge, as it is practically impossible to have complete knowledge of the problem. The subsequent phase, known as model verification, ensures the model follows the functional and performance requirements. The challenge behind this stage lies in the test-based verification, AI-Driven Cloud-Native Orchestration and Resource Provisioning 9 which should be done in real-life settings. However, due to challenges related to safety, security, and scalability, testing is frequently replaced with simulation environments [38]. Polyzotis et al. [46] proposed a data validation system for ML, arguing that dataset validation is essential in the machine learning pipeline to prevent common data issues that arise when data generation is separate from the pipeline. The final stage, also known as model deployment, consists of the model integration in the system or infrastructure where it should run. Russell and Norvig [19] classify ML algorithms based on the learning approach, the input and output data type, and the problem solved: in Supervised, Unsupervised, and RL and a few hybrid approaches. Supervised learning algorithms require a training set of labelled examples provided by an external supervisor. The data consists of input-output pairs, where the label represents the correct action the system should take. The goal of this learning type is to generalize responses effectively, ensuring appropriate actions in situations beyond the training set. Unsupervised learning algorithms, in contrast, do not require labelled training data. Instead, they model underlying patterns in the data to uncover its characteristics. The most common unsupervised learning approach is clustering, which groups data points based on similarity. This method is often applied in tasks such as customer segmentation and anomaly detection. Reinforcement learning (RL), also known as Reinforcement Learning from Human Feedback (RLHF), is a dynamic programming technique that trains algorithms through a reward and punishment system. During the learning process, an artificial agent receives rewards or penalties based on its actions, aiming to maximize the total accumulated reward. This approach is particularly useful in decision-making processes and dynamic environments. Russell et Norvig [19] argue that despite supervised learning excels at predicting specific outputs based on labelled data, unsupervised learning can also contribute by analysing historical patterns in resource utilization data. Unsupervised models can identify clusters or outliers that might represent potential bottlenecks. Furthermore, both supervised and unsupervised learning excels at leveraging historical data to inform decision-making. However, the real world necessitates dynamic adjustments based on constantly evolving workloads and system conditions. RL offers a compelling approach for this dynamic environment. 4 Scheduling – a multi-criteria optimization problem Improper scheduling can lead to resource waste and degradation of in-service performance [12]. The goal of scheduling is to find the best edge/cloud resources for upcoming end-user applications to improve QoS parameters such as latency, throughput, response time, and resource utilisation. Improving all these QoS parameters involves significant complexity. In the literature, the scheduling problem is usually an NP-complete problem or an NP-hard problem without a polynomial-time algorithm [13]. The scheduling depends on the nature of resources that should be scheduled. The literature focuses on the difference between