Machine learning regression to boost scheduling performance in hyper-scale cloud-computing data centres

Machine learning regression to boost scheduling performance in hyper-scale cloud-computing data centres Damián Fernández-Cerero a, ⇑ , José A. Troyano a , Agnieszka Jakóbik b , Alejandro Fernández-Montes a a Department of Computer Languages and Systems, University of Seville, Avda. Reina Mercedes s/n., 41012 Seville, Spain b Department of Computer Science, Cracow University of Technology, Cracow, Poland article info Article history: Received 2 November 2021 Revised 13 April 2022 Accepted 14 April 2022 Available online 22 April 2022 Keywords: Data centre Cloud computing Scheduling optimisation Machine learning Gradient boosting abstract Data centres increase their size and complexity due to the increasing amount of heterogeneous workloads and patterns to be served. Such a mix of various purpose workloads makes the optimisation of resource management systems according to temporal or application-level patterns difficult. Datacentre operators have developed multiple resource-management models to improve scheduling performance in controlled scenarios. However, the constant evolution of the workloads makes the utilisation of only one resource-management model sub-optimal in some scenarios. In this work, we propose: (a) a machine learning regression model based on gradient boosting to predict the time a resource manager needs to schedule incoming jobs for a given period; and (b) a resource management model, Boost, that takes advantage of this regression model to predict the scheduling time of a catalogue of resource managers so that the most performant can be used for a time span. The benefits of the proposed resource-management model are analysed by comparing its scheduling performance KPIs to those provided by the two most popular resource-management models: twolevel, used by Apache Mesos, and shared-state, employed by Google Borg. Such gains are empirically evaluated by simulating a hyper-scale data centre that executes a realistic synthetically generated workload that follows real-world trace patterns. Ó2022 The Authors. Published by Elsevier B.V. on behalf of King Saud University. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). 1. Introduction Data centres constitute the core infrastructure for current Internet services, from world-wide web and mobile applications to Bigdata and High-Performance-Computing workloads. Therefore, the performance of such infrastructures is crucial to enable services that require computational and storage resources to be available, scalable, and ready to serve the incoming workloads, which reduces operational costs and offers the most competitive services to the final users. In the past, it was more common for medium and large companies to manage their own clusters, which served a fully controlled workload. In such scenarios, the computing and storage resources can be fine-tuned by the operation teams. Currently, hyper-scale data centres serve a wide range of users and workload requirements. The mix of continental or even world-wide workloads causes difficulties in optimising the computational resources according to temporal or usage patterns, since the job-arrival patterns fade away, and unknown events may cause unexpected demand peaks that may degrade the performance of the data centre notably. In summary, the workloads related to Cloud Computing have mutated from controlled workloads to workloads that present arrival and resource-demand patterns that evolve rapidly and really hard to predict. Data-centre resource managers are considered as the datacentre operating systems, as they are responsible for the management and monitoring of the scheduling process and the monitoring of the computational resources and workload at the highest level. Resource managers usually coordinate one or several scheduling agents, which, in turn, are in charge of the application of scheduling policies that determine the placement of each particular task on the data-centre resources. Fine-tuned monolithic resource managers were typically used when companies used to operate their own small-sized data centres. However, such monolithic resource managers, which usually employ scheduling algorithms designed for the specific workload under consideration, cannot keep up to the massive-scale workloads related to hyper-scale data centres. Even though data-centre operators used to leave some datacentre resource capacity idle to absorb moderate demand peaks without seriously degrading the data-centre performance, the https://doi.org/10.1016/j.jksuci.2022.04.008 1319-1578/Ó2022 The Authors. Published by Elsevier B.V. on behalf of King Saud University. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). ⇑ Corresponding author. E-mail addresses: [email protected] (D. Fernández-Cerero), [email protected] (J.A. Troyano), [email protected] (A. Jakóbik), [email protected] (A. Fernández-Montes). Journal of King Saud University – Computer and Information Sciences 34 (2022) 3191–3203 Contents lists available at ScienceDirect Journal of King Saud University – Computer and Information Sciences journal homepage: www.sciencedirect.com resource managers could not keep up with the high number of scheduling operations present in new Cloud-Computing scenarios (Gog et al., 2016). Various resource managers were developed to overcome the aforementioned limitations: centralised frameworks such as Omega (Schwarzkopf et al., 2013) and Mesos (Hindman et al., 2011), distributed solutions such as Pigeon (Wang et al., 2019), and even fine-tuned hybrid approaches, such as Mercury (Karanasos et al., 2015). The two-level (Mesos) and shared-state (Omega) centralised strategies are the most used currently. However, such resource managers focus on serving a common workload pattern present in hyper-scale data centres (Schwarzkopf et al., 2013), where approximately 90% of batch jobs consume few resources for a short time, whilst the 10% of jobs consume a large amount of resources for a longer time. However, such workload patterns change over time, making those models suboptimal in comparison to other resource-managing models for some use cases, or even only for a period of the same use case (D. Fernández-Cerero et al., 2018). This work focuses on improving the scheduling performance of the data centre by proposing a novel resource manager selector that determines the most performant resource management model for each period. This proposal leverages the industry data-centre resource management architecture, which equips only a single resource management model, by enabling the dynamic selection of a resource management model from a catalogue of existing resource managers. A gradient boosting regression model is responsible for the estimation of the scheduling performance of each resource management model in the catalogue for a given data-centre operating situation and period. According to the estimation of scheduling performance, the most beneficial resource management model is applied during that particular period. The benefit of the proposed resource management model is then illustrated by comparing three alternatives: (a) the utilisation of the two-level resource management model used by Mesos or YARN; (b) the utilisation of the shared-state resource management model used by Google Borg and Google Omega; and (c) the utilisation of the proposed resource management model, Boost, which estimates the scheduling performance of the two resource management models above to apply them dynamically. The centralised two-level and shared-state resource management models have been selected as they constitute the main resource managers used in industry. The performance and energy efficiency results of the three alternatives above were evaluated through an extensive simulation employing a trustworthy simulation tool as well as a realistic hyper-scale data-centre composed of 1,000 machines and synthetic traces that follow the patterns present in Google and Alibaba data-centre traces. In this work, we do not improve data-centre performance by proposing any scheduling algorithm, but a new data-centre resource management model. Any scheduling algorithm can be applied by scheduling agents coordinated by resource managers. The scheduling algorithm employed in the experimental analysis tries to maximise resource utilisation while preventing resource contention by following Google’s latest improvements in this area (Lo et al., 2016). The following contributions are presented in this work: Gradient boosting regression estimator for the prediction of the scheduling performance of the most popular industry-level resource managers. Transformation rules that enable the successful estimation of the scheduling performance of resource managers. Resource management model that leverages the estimator developed to select the most performant resource management model for a period and predicted incoming workload. The benefits of the aforementioned contributions are backed by: Analysis of the impact of each feature of the regression models on the quality of the predictions for the catalogue of resource management models employed to train the regression models used in this work: two level (Mesos) and shared state (Omega). Analysis of the behaviour of our proposed resource management model, Boost, vs. the most utilised resource managers in extreme situations related to hyper-scale data centres, where the workload arrival pattern follows an extreme-value distribution. All the experimentation is based on realistic workload traces, which enables Boost to be applied in production environments. Analysis of the full scheduling time Key Performance Indicator (KPI), in addition to the traditional scheduling queue times evaluated in the literature (Schwarzkopf et al., 2013; Tirmazi et al., 2020). Evaluation of the performance impact of estimating and selecting the resource manager synchronously, that is, every time a job is submitted, vs. carrying it out asynchronously in the background every given period. Three time periods are analysed for the background process. Evaluation of the energy consumption of the proposed resource management model and the comparison with current resource managers. This paper is organised as follows: Section 2examines the related work, while Section 3presents the theoretical basis that supports this study. The machine learning regression model is illustrated in Section 4. The results of the empirical analysis performed are evaluated in Section 5. Finally, Section 6presents the conclusions of this work and future work. 2. Related work Most research on data centres has focused on the increase of performance, reduction of costs, and improvement of computational and storage services. Over the last ten years, the focus has shifted to the application of various Artificial Intelligence techniques to various goals. Workload classification and workload prediction have attracted considerable attention from the research community, as they are essential for the improvement of data-centre performance, the reduction of energy consumption, compliance with the required quality of service (QoS) levels, and the improvement of the scalability of cloud service providers. Concerning workload classification, the authors of Dewangan et al. (YYYY) proposed a modified KNN algorithm for the classification of jobs into four categories, so the scheduling process takes into account both the user priority and the operating costs. The aim of Wu et al. (2018) is to classify jobs according to the required resources, such as CPU, memory, network, and storage, to develop a scheduling algorithm that includes this classification among the heuristic rules used to determine the optimal resources. The authors of Elrotub and Gherbi (2018) propose a deterministic method for the classification of tasks according to their size. Three groups are considered in the proposed scheduling algorithm to select optimal resources for each task: heavy, medium, and light tasks. Iqbal et al. (2018) shows an analysis of a web application Damián Fernández-Cerero, José A. Troyano, A. Jakóbik et al. Journal of King Saud University – Computer and Information Sciences 34 (2022) 3191–3203 3192 workload based on URI requests to auto-scale the needed resources. The authors employ artificial intelligence to generate the so-called probabilistic workload pattern for the prediction of the incoming workload. Such prediction is then used to provision the estimated resource demand. Other works focus only on the analysis of various workloads to identify, classify, and cluster the jobs in differentiated groups. In these works, the authors do not directly employ such classification and clustering models to improve the data-centre KPIs. Genkin et al. (2019) focuses on the classification of Spark and Hadoop workloads by taking into account the container performance. A recent analysis of the Google traces presented in Patel and Kushwaha (2020) reveals that the Gaussian Mixture Model results in a better workload clustering in comparison with other methods. Regarding to workload prediction, the majority of proposals are based on machine learning and artificial intelligence techniques, from Neural Networks to regression-based models. A sample of some representative proposals is presented below. The authors of Singh et al. (2019) propose a Support Vector Machine model for the estimation of the characteristics of the incoming jobs in a given time window so that the resource provisioning process can be optimised. Li et al. (2021) proposes a hybrid model based on Autoregressive Integrated Moving Average model (ARIMA) and Back Propagation (BP) neural networks to predict SLA violations and VM migration times. The authors of Zhang et al. (2018) propose a new Deep Learning model based on canonical polyadic decomposition to efficiently predict the incoming workload. A trace included in the CloudSim simulator is used to validate the efficiency and accuracy of the proposed model. Wamba et al. (2017) compares constraint programming and neural network models to predict and generate traces, and reached the conclusion that neural network models offer better prediction results while constraint programming is more suitable for trace generation. The authors of Kumar et al. (2018) present an interesting combination of three-layered neural networks and a self-adaptive differential evolution algorithm, and claim very good results in terms of workload prediction accuracy to improve resource provisioning. The training data set is based on the requests done to a very specific service (a Canadian university web server and a NASA Kennedy Space Center web server as well). The results reported by Gao et al. (2020) suggest that it is important to perform the prediction some time before it is needed, so that the scheduling algorithm has enough time to adapt. These authors compare various techniques and conclude that clustering methods for prediction can achieve up to 90% of prediction accuracy for CPU and memory. The authors of Tang et al. (2018) performed an analysis to determine suitable techniques for workload prediction to develop energy-aware scheduling algorithms. They combine linear regression and wavelet neural networks to this end and conclude that these techniques can be useful for low-utilisation cloud data centres. In Amiri et al. (2018), the authors propose a prediction model that outperforms the performance of current predictors. Their model is based on episode mining and is able to adapt to the workload changes rapidly, which is then used to improve the resource provisioning process. In summary, as shown in Table 1, various research studies employ artificial intelligence models to analyse, classify, and even apply this knowledge to improve existing scheduling algorithms. However, to the best of our knowledge, no progress has been made at the resource-management level. This work presents a prediction model based on gradient-boosting regression that predicts the scheduling performance of a catalogue of resource management models for a given data-centre operating environment and period, which enables the selection of the resource management strategy that minimises the scheduling time for that period and incoming workload from such catalogue. The proposed resource manage- Table 1 Classification of the related work considered. Boost works at the data-centre resource-managing level. Dewangan et al. (YYYY) Wu et al. (2018) Elrotub and Gherbi (2018) Iqbal et al. (2018) Genkin et al. (2019) Patel and Kushwaha (2020) Singh et al. (2019) Li et al. (2021) Zhang et al. (2018) Wamba et al. (2017) Kumar et al. (2018) Gao et al. (2020) Tang et al. (2018) Amiri et al. (2018) Boost Target layer Scheduling algorithms XX XX Resource provisioning XX XXXX X Workload analysis XX X Resource manager X Environment Simulation XX XXXX Benchmark XXX X Real world XXX XXX X Supervised learning category Classification XXX XXX X X X X Regression XXXX X XX X Damián Fernández-Cerero, José A. Troyano, A. Jakóbik et al. Journal of King Saud University – Computer and Information Sciences 34 (2022) 3191–3203 3193 ment model does not conflict with the state of the art, since resource managers could coordinate the scheduling agents that equip the scheduling algorithms considered, as well as the provisioning and migration techniques. 3. Theoretical framework Let us denote a single workload as W, whilst Wrepresents all workloads to be served. In this work, the values of the following parameters characterise each workload: Inter-arrival time D t j;sub Wei 0:5;k W ðÞrepresents the time between two consecutive job submissions (t j;sub and t j1;sub for the job J j ) of a particular workload for j¼1;2;.... We employ an extreme-value Weibull distribution with a shape value of a = 0.5 to generate the inter-arrival times with a scale value of 1=k W to represent the hardly predictable workload interarrival patterns present in hyper-scale data-centres. Number of tasks n j Exp k j;n  represents the number of tasks that a job is composed of, following an Exponential distribution with a given mean value of 1=k j;n . Job duration l j Exp k j;l  represents the period of time a given job J j consumes resources in the data centre, generated by means of the Exponential distribution with the given expected value 1=k j;l . Resource usage u j is the amount of CPU (denoted by K CPU ) and RAM (denoted by K RAM ) that all the tasks of each particular job J j in a workload consumes. Each workload is composed of a set of jobs W¼J j  n j¼1 for n2N, and each jth job is composed of tasks T j ¼t jn  n j n¼1 for n j 2N. Every time a job J j is submitted to the data centre at a particular operation time, considered as the job submission time t j;sub of the job J j , the scheduling process graphically represented in Fig. 1 starts. Every time a job arrives, one scheduling agent performs a job scheduling action if it is available or puts the job in a queue until it becomes available. A scheduling action can be defined as the process that employs a scheduling algorithm to determine the set of resources where the tasks of a job are to be executed. When a scheduling action is unable to schedule all the tasks in a job, new scheduling actions will be performed until all the tasks of the job are assigned to computing resources or a time-out is reached. Once this process is over, the job is considered fully scheduled at t j;sched . The time required by the scheduling agents to fully schedule the job J j is denoted as S j , which represents the sum of times S ji needed for each scheduling action performed on the job J j , as follows: S j ¼P n jA i¼1 S ji , where n jA denotes the number of scheduling actions needed to fully schedule the job J j . Hence, t j;sched ¼t j;sub þS j . The completion time t j;com of job J j may be denoted, then, as follows: t j;com ¼t j;sched þl j , i.e: t j;com ¼t j;sub þ S j þl j . Let us denote by Athe set of all A W actions, that is: A¼A W ;W2Wfg¼A j;i ;j¼1;2; ::; n;i¼1;2; ::; n jA  ð1Þ Each scheduling action A ji takes some time to compute the algorithms needed to make the deployment decisions, as follows: S A i;jðÞ¼K J jðÞþ X n j;uns n¼1 K T n;jðÞ ð2Þ S A i;jðÞdenotes the time required to perform the ith scheduling action for the jth job, K J jðÞrepresents the time the scheduling algorithm executed at the job level, K T n;jðÞdesignates the time of the scheduling algorithm applied to nth task in the jth job, while n j;uns represents the remaining number of tasks to be scheduled. The equation above assumes the scheduling agent is free when the job arrives. However, this may not happen when previous jobs in the queue are not fully scheduled. Then, the time that a job waits in the queue must be added to the equation as follows: S A i;jðÞ¼Q ji þK J jðÞþ X n j;uns n¼1 K T n;jðÞ ð3Þ Q ji represents the time the jth needed to wait in queue when the ith scheduling action started. Each scheduling action A ji may provide two outcomes: (a) The scheduling action A ji succeeds in the deployment of all tasks T j of the job J j . In such scenario, t j;sched ¼t j;sub þS j , where S j ¼S A 0;jðÞ. In this equation, t j;sched denotes the operation time where the job is fully scheduled and S A 0;jðÞthe time needed to perform the first scheduling action. Fig. 1. General incremental job scheduling and execution process. Damián Fernández-Cerero, José A. Troyano, A. Jakóbik et al. Journal of King Saud University – Computer and Information Sciences 34 (2022) 3191–3203 3194 (b) The scheduling action A ji can’t fully deploy all T j tasks of the job J j , due to two main reasons: the data centre has not enough free resources to execute them, or the transaction that commits the scheduling action raises a conflict in optimisticlocking strategies. In such scenarios, the scheduling action A ji needs to be retried until the number of scheduling actions n jA reaches the limit of scheduling actions n max jA , increasing the number of timed-out jobs J to ; or until the job J j is fully deployed in the successive scheduling actions, when the number of required scheduling actions n jA <n max jA . In such scenarios, S j ¼P n jA i¼1 S ji . Boost employs a Machine Learning algorithm to estimate the performance of the resource manager for a certain data-centre operation period D D to apply the optimum resource-management model for that particular period. In this work, we consider two versions of the dynamic resource-management agent: 1. Background agent which performs the resource-management- model selection decision Devery fixed time period D D , independently from the scheduling actions Ato avoid scheduling overhead in each job scheduling action A ji and the related scalability issues. 2. Online agent which performs the resource-management model selection decision Dfor every job scheduling action A ji , thus, D D ¼fA ji  ;A ji 2Atherefore increases the time needed for each job scheduling action. In this online approach, the estimation algorithm time K D must be taken into account to compute each scheduling action, which may cause scalabilty issues in the scheduling process: S A i;jðÞ¼Q ji þK J jðÞþK D jðÞþ X n j;uns n¼1 K T n;jðÞ ð4Þ The completion time (makespan) C j is the final performance indicator that represents the end of the execution process for the job J j , and may be denoted as follows: C j ¼S j þl j . The following energy measurement model is employed in this work: ETðÞ¼ X t2 D X r2R Pr;tðÞdð5Þ Let t2 D ¼d;2d;...;Tfg, where Tdenotes the total operation time. The power state for each computing resource in every machine Pr;tðÞis measured every dtime period. Two different power levels are considered in this work: P2Idle;Executing fg . The Idle power level denotes the power consumption of the computing resource when no tasks are being executed, while the Executing power level may be defined as follows: P Executing ¼P Max P Idle , where P Max represents the maximum power consumption when the computing resource is used at a maximum level. Hence, the power consumption PrðÞ¼P Idle þ r exec r idle  P Executing , where r exec r idle represents the resource utilisation rate. The proposed resource management model cannot impact negatively in terms of data-centre availability and robustness, since its main task, the prediction and dynamic application of the optimal resource-managing model for a given period, would not interfere in the workload execution. Even in a crash of the service executing the proposed resource management model, the resource manager that was running would keep serving the incoming workload as usual. 4. Gradient boosting regression model 4.1. Generation of the training data set In this work, we will employ a regression model, which will be responsible for the determination of a numeric output from an input vector, to estimate the scheduling performance of two resource-managing strategies every certain period. A set of instances, each one composed of a pair of one input vector and one numeric output, shapes the data set our regression model is trained with. A 15-day data-centre execution trace, provided by the simulation tool SCORE Fernández-Cerero et al. (2018) is taken as the raw training data. Synthetic data-centre execution traces for each resource manager considered are employed to train the estimation model. Such traces follow the workload patterns present in Google cluster 2011 and 2018 real data-centre traces (C. Reiss et al., 2012; Tirmazi et al., 2020). We employed realistic synthetic workload traces for several reasons: Normalisation and obfuscation of the workload. The aforementioned data-centre workload traces from Google 2011 and 2018 hide the real low-level characteristics of both the computational resources, such as the number of CPUs and GBs of RAM of each machine, as well as the actual workload resource consumption. Only a normalised value of such resource consumption can be obtained, whereas other attributes are directly obscured. Due to this, the actual reproduction of the physical data center with the real workload is unattainable - mainly due to the need of big industry players, such as Google and Microsoft, to keep the shape of their infrastructure confidential as a part of the industrial secrets-. Thus, only similar synthetic workloads can be generated, which can reproduce the behaviour of real data centres to some extent. Rapid evolution of cloud computing workload. The study of the traces above (C. Reiss et al., 2012; Tirmazi et al., 2020; Lu et al., 2017) shows a clear pattern where the workload has changed its shape over time: from the inter-arrival time to resource consumption, as well as the heterogeneity and extreme events of the workload. This evolution can make any resource management model developed only for a particular scenario useful only for a short time. Broader applicability of the proposal. Given the evolution of the Cloud-computing workloads, as well as the heterogeneity of the big Internet Data Centers present in the industry, we consider that testing the validity of a resource management model only for one short period of a particular environment could fall short to show its behaviour. Thus, the generation of realistic workload traces is relevant since it enables the evaluation of broader scenarios where industry players could check the benefits and drawbacks of the proposed resource management model and how it fits to their current situation. Validity of the synthetic workload. The fidelity of the synthetic workload to reality was first validated by the Google research team that developed both Google Omega and the first version of the employed simulator Schwarzkopf et al. (2013), and then such workloads have been shown to be highly useful in many previous articles published in prestigious journals, such as (Fernández-Cerero et al., 2018; Fernández-Cerero et al., 2018; Fernández-Cerero et al., 2018; Fernández-Cerero et al., 2021). Some of the features in the raw data-centre execution trace may require some transformations to improve the usefulness of the raw data. The raw trace and the transformed training data set can be Damián Fernández-Cerero, José A. Troyano, A. Jakóbik et al. Journal of King Saud University – Computer and Information Sciences 34 (2022) 3191–3203 3195 found as supplementary material. In this section, the following main data transformations applied to build our training data set are explained: 1. Logarithmic transformation of the output feature. 2. Creation of a delta submission time feature D t j;sub . 3. Framing by time windows. 4. Uncertainty management for unfinished jobs. The performance of the regression model is strongly related to the wideness of the range of values of the output feature. Fig. 2 shows the range of values for the output feature to be predicted, that is, the scheduling time S j . It must be noticed in Fig. 2a that the values are in a fairly wide range of [0–1200] seconds, also presenting a long-tailed distribution. Both the wide range and the distribution skewness of S j are challenging for the successful training of the regression model, since the estimation results must be precise enough in the lower extreme of the distribution while wide enough to consider the opposite extreme of the distribution. A logarithmic transformation is applied to the values of the output parameter S j .Fig. 2 shows the resulting Kernel density estimation (KDE) distribution of S j values after the application of a natural logarithm. As a result, the range is reduced to [0–7], and the distribution is much closer to a normal distribution, as can be shown in Fig. 2b. Even though the job submission time t j;sub contains useful information, the raw values of this feature are not enough for a predictor to make precise decisions, since the constant increase of its value makes extremely hard the establishment of patterns. D t j;sub , explained above, is taken to overcome this limitation and extract valuable patterns. The estimation problem must be modelled as a temporal series that requires the past knowledge to be included in the regression model through the training data set, since a set of jobs submitted in the recent past (window frame) needs to be considered to improve the training process. Fig. 3 shows an example of the possible status of the scheduling process of the previous jobs when J j is considered. Let us use only the scheduling status, that is, the t j;sched parameter, for the coming explanations for the sake of clarity; however, the same explanation applies to the rest of features. At any given submission time t j;sub for the job J j , any previous job J p ;p¼1;2;...;j1 may be in the following scheduling state: (a) The previous job was fully scheduled. In this case, the scheduling time of the previous job, t p;sched occurred before t j;sub , and therefore is known. This is the status of the jobs J j4 ;J j3 , and J j1 in Fig. 3. (b) The previous job was not fully scheduled. In this case, the scheduling time of the previous job, t p;sched will eventually occur in an unknown future. In Fig. 3,J j2 represents this state. Fig. 2. Original vs. logarithmic Kernel density estimation (KDE) distribution for the class feature scheduling time S j . Fig. 3. Scheduling status and related times for previous jobs when J j is considered. Damián Fernández-Cerero, José A. Troyano, A. Jakóbik et al. Journal of King Saud University – Computer and Information Sciences 34 (2022) 3191–3203 3196 In this work, we only employ known information to build our training data set. Due to this, the following two features are incorporated into every row in our training data set, which represent each J j job, to manage the scenarios where unknown information regarding previous jobs is present: (a) F p;sched , that denotes whether the immediate past job is fully scheduled or not. F p;sched is computed as follows to generate the training data set: F p;sched ¼1ift j1;sched 6t j;sub 0 otherwise (b) t p;sched , that represents the known period of scheduling time of the previous job. In our training data set, t p;sched ¼min t j1;sched ;t j;sub  . This very same process is used for job queue and completion times. 4.2. Training and evaluation of the regression model Gradient-boosting regression (Friedman, 2002), which has been used in the past with successful results in many regression problems (Zhang and Haghani, 2015; Cai et al., 2020), is an ensemble learning method that combines the estimates provided by a set of simple regressors to produce a joint prediction. Decision trees are used in this work as base estimators. The R 2 metric is used as the main cross-validation KPI for the evaluation of the developed regression models. The metric R 2 computes a correlation coefficient between the regressor output and the expected real value, with 1 being the best possible result. Fig. 4 shows the evolution of the values of the R 2 metric for the regression models trained with the scheduling performance traces provided by Mesos (two-level) and Omega (shared-state) resourcemanaging models according to the size of the past time window. Values in the range [1–10] are considered to analyse the impact of the size of the past window. The results provided by employing the original scheduling time S j and its logarithmic transformation for model training are shown. The analysis performed provides the following conclusions: The estimation results for both the Mesos (two-level) and Omega (shared-state) resource management models are good, resulting in R 2 values higher than 0:8. The logarithmic transformation is the key to achieving successful results, increasing the value of R 2 by approximately 0:3 for the two-level resource management model (see Fig. 4 and by more than 0:1 for the shared-state resource management model (see Fig. 4b). The size of the past window time is also a key to improving the estimation results, especially for Omega, where the value of R 2 increases by approximately 0:1 when using a window size of at least 3, as shown in Fig. 4b. Fig. 5 shows the learning curves for the two-level and sharedstate resource management models. The analysis of learning Fig. 4. R 2 score evolution according to the size of the time window. Fig. 5. Resulting learning curves from the regression model training process. Damián Fernández-Cerero, José A. Troyano, A. Jakóbik et al. Journal of King Saud University – Computer and Information Sciences 34 (2022) 3191–3203 3197 curves is an essential tool for the detection of overfitting. In this figure, where the R 2 score is presented versus the size of the data set, it becomes clear that the trained regression models do not overfit, since the gap between the training and validation curves strongly decreases as soon as the size of the data set increases. The stabilisation of the gap trend towards the end of the learning curves clearly shows that the size of the training data set is large enough. Fig. 6 shows the Kernel Density Estimation (KDE) error distribution for the regression models trained with the Omega and Mesos scheduling performance result traces, respectively. KDE is defined as the difference, in seconds, between the estimations provided by the regression models and the expected real values. These error values are computed through cross-validation, resulting in a distribution skewed toward a small value of approximately 2 s for Mesos and Omega resource managers. The gradient boosting technique provides a set of quality indicators for each feature during the regression model training process. The feature importance must be highlighted among the aforementioned indicators, since it denotes the contribution of each feature to the predictive capacity of the regression model. In the gradient boosting model, the feature importance is calculated on the basis of the number of appearances of each feature in the set of decision trees trained for the base models, along with the position of each feature in those decision trees. The feature importance indicator may be used, in addition to the implementation of feature selection algorithms, as a first and simple step for model explanation, since some hints about how the model performs the predictions may be obtained. Fig. 7 shows the 10 features with the highest feature importance for the prediction of the scheduling time S j of a given job J j for the regression models trained with the scheduling results of the two-level (see Fig. 7 and shared-state (see Fig. 7b) resource management models, respectively. It should be noted that, even if similar, the feature importance values are different for both regression models. The following two hints may be highlighted since they represent that the model was successful in detecting the differences in terms of behaviour between the strategies of the considered models. The two-level resource management model (Mesos) gives more importance to the scheduling time of the previous job S j1 , while the shared-state model (Omega) takes into account only whether or not the previous job J j1 has been completed. This is due to the pessimistic blocking of two-level resource manager agents (Mesos), which perform batch scheduling processes that prevent other scheduling agents from making scheduling decisions in parallel. The feature task scheduling attempts P n jA i¼1 n j;uns is more critical for Omega (shared-state model) than for Mesos (two-level model). This is due to shared-state resource managers being more prone to perform conflicting scheduling actions, which Fig. 6. Kernel density estimation (KDE) error distribution. Fig. 7. Feature importance for the prediction of the scheduling time S j of a given job J j . Damián Fernández-Cerero, José A. Troyano, A. Jakóbik et al. Journal of King Saud University – Computer and Information Sciences 34 (2022) 3191–3203 3198 need to be retried and worsen the data-centre performance, when a higher number of task scheduling decisions are made in the same period of time. 5. Empirical Evaluation 5.1. Experimentation framework In this work, we used the SCORE simulator Fernández-Cerero et al. (2018), which has been widely used (Fernández-Cerero et al., 2018; D. Fernández-Cerero et al., 2018; Fernández-Cerero et al., 2020; Fernández-Cerero et al., 2020; Fernández-Cerero et al., 2021) to analyse the behaviour of realistic hyper-scale data centres, usually composed of thousands of machines, both in terms of energy consumption and scheduling performance. The following key performance indicators, previously explained in Section 3, are studied to analyse the performance impact of the proposed data-centre resource management model: (a) Measured average inter-arrival time Dt sub;W . In this work, two main workloads are considered, Batch (B) and Service (Service) workloads, that is, W¼B;Ser v ice fg .1)Batch workload. Batch jobs are usually composed of a large number of tasks that consume relatively few resources for a short period and have a fixed end. Map-reduce jobs are a good example of this kind of workload. We analyse three different scale parameters k B for the Weibull distribution used to generate the inter-arrival times for the Batch workload 1 k B ¼120;100;80 ½ s.2)Service workload. Service jobs are usually of a reduced number of tasks, which consume a high amount of resources for a longer time period and have no determined end. Web servers or DBMS jobs are a good example of this kind of workload. The arrival rate of this workload is set to be 10 times less frequent than that of the Batch workload, that is, k Ser v ice ¼10 k B . The Batch workload usually represents almost 90% of scheduled jobs, as the workload used is modelled following industry usage patterns (Tirmazi et al., 2020; Lu et al., 2017). Due to the prevalence and stronger impact on the scheduling system of this workload, the results for the Batch workload, such as D t sub;B instead of global D t sub;W , are presented in the tables for the sake of clarity. The results for the Service workload can be found as supplementary material. (b) Queue times. Both Q 1B , which denotes the average time that each J j job in the Batch workload W B waits in queue until its first task is scheduled, and Q nB , which, in turn, represents the average time that the J B jobs in the Batch workload wait in queue until its last task n j is scheduled, can be found in the tables. The 90 percentiles of these values, Q 1;90%B and Q n;90%B , respectively, are also presented so that the long-tailed behaviour, if present, is shown. (c) Scheduling time. The average scheduling time of Batch jobs S B , and the 90 percentile value S 90%B , are analysed. (d) Makespan. The average completion time of the jobs that make up the Batch workload C B , and the 90 percentile value C 90%B are shown. In addition to these indicators, there is a set of useful indicators related to the behaviour of each resource management model studied. The following indicators are analysed and presented in the tables: (a) Timed-out jobs.J to denotes the percentage of jobs that could not be completed due to a time-out in the scheduling process. This event occurs when scheduling bottlenecks appear of 1,000 consecutive unsuccessful task scheduling attempts or 100 consecutive unsuccessful job scheduling attempts. J to is only shown in the tables when the percentage of timed-out jobs is greater than zero in any of the experiments presented. (b) Locked resources. Due to their pessimistic blocking strategy, both the two-level and Boost resource management models present the parameter R lock , which represents the average percentage of resources unavailable due to a scheduling agent performing a scheduling operation, which prevents any other scheduling agent from performing scheduling operations in parallel. (c) Conflicted tasks. Due to their optimistic blocking nature, both Shared-state and Boost resource management models make conflicting task scheduling attempts T conf , which must be retried. Energy consumption Eis also shown in the tables to analyse the impact of Boost in terms of energy efficiency. The value of the scheduling algorithm times, both at the job and task level, K J and K T , is set following the trends of the industrial literature, in 1000 and 100 ms, respectively (Schwarzkopf et al., 2013). The execution time of the Boost estimation process K D has been established through empirical measurements performed on an M1 Macbook Pro, which, after 500 runs, results in an average time of 100 ms. Each experiment simulates 15 days of operation of a data centre composed of 1,000 homogeneous machines. All experiment configurations are run 25 times. Tables 2–4 show the average performance and energy results for Boost, Mesos, and Omega, respectively. The following statistical tests were performed in SPSS to verify the statistical significance of a ¼0:05: (1) Iglewicz and Hoaglin robust test for multiple outliers (two-sided test) with Z¼3:5. (2) Levene test to check the non–homogeneity of variances with a significance level of a ¼0:05. (3) Kruskal–Wallis test for independent samples, to analyse the difference in the results between pairs of resource management models with a significance level of a ¼0:05. The results of the tests performed are provided as supplementary material. 5.2. Simulation evaluation Fig. 8 shows the evolution of the selection of the resource management strategy performed by Boost depending on the workload inter-arrival time. The red line denotes the period where the twolevel strategy is used (the same strategy employed by Mesos), whilst the blue line represents the period where the shared-state strategy is used (the same strategy employed by Omega). Although the behaviour of Boost’s selection process suggests homogeneity between Fig. 8a and 8b, a main trend is present: when the workload pressure of the data centre increases (see Fig. 8b), the twolevel strategy is used more frequently, while in periods of lower utilisation rates, Boost employs the shared-state strategy more frequently (see Fig. 8a). The results in terms of performance and energy consumption for Boost, Mesos, and Omega are shown in Tables 2–4, respectively. It must be kept in mind that the value presented in each cell is the average of 25 runs of the same experiment configuration, without outliers. Each table shows 3 rows, corresponding to a workload whose inter-arrival time is generated by means of a Weibull a ¼0:5 with three different mean inter-arrival rates: k B ¼120;100;80½s, which represent low, medium, and high utilisation rates, respectively. The experimentally measured average inter-arrival time D t sub;B for each workload is shown in the first column. Table 2 also shows four groups, each representing one of the configurations for the decision period of the Boost resource management selection process: The first shows the results when the resource management selection process is performed on each job scheduling action, D D ¼fA ji  ;A ji 2A; the following three show the results of making resource management selection decisions on a background agent in every given period: D D = [30,60,120]s. The results of the average scheduling time S B , the main scheduling performance indicator, can be analysed to expose the main trends in the Boost behaviour. Although our resource management model improves the performance of the data centre in all scenarios, Boost excels as soon as the decision period for the Damián Fernández-Cerero, José A. Troyano, A. Jakóbik et al. Journal of King Saud University – Computer and Information Sciences 34 (2022) 3191–3203 3199