Short-term load forecasting models: A review of challenges, progress, and the road ahead
Abstract
Short-term load forecasting (STLF) is critical for the energy industry. Accurate predictions of future electricity demand are necessary to ensure power systems’ reliable and efficient operation. Various STLF models have been proposed in recent years, each with strengths and weaknesses. This paper comprehensively reviews some STLF models, including time series, artificial neural networks (ANNs), regression-based, and hybrid models. It first introduces the fundamental concepts and challenges of STLF, then discusses each model class’s main features and assumptions. The paper compares the models in terms of their accuracy, robustness, computational efficiency, scalability, and adaptability and identifies each approach’s advantages and limitations. Although this study suggests that ANNs and hybrid models may be the most promising ways to achieve accurate and reliable STLF, additional research is required to handle multiple input features, manage massive data sets, and adjust to shifting energy conditions.
Full text
Citation: Akhtar, S.; Shahzad, S.; Zaheer, A.; Ullah, H.S.; Kilic, H.; Gono, R.; Jasi´nski, M.; Leonowicz, Z. Short-Term Load Forecasting Models: A Review of Challenges, Progress, and the Road Ahead. Energies 2023, 16, 4060. https://doi.org/10.3390/ en16104060 Academic Editor: Young Gyu Jin Received: 13 March 2023 Revised: 8 May 2023 Accepted: 10 May 2023 Published: 12 May 2023 Copyright: © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). energies Review Short-Term Load Forecasting Models: A Review of Challenges, Progress, and the Road Ahead Saima Akhtar 1, Sulman Shahzad 2,* , Asad Zaheer 3, Hafiz Sami Ullah 4, Heybet Kilic 5, Radomir Gono 6,* , Michał Jasi´nski 7and Zbigniew Leonowicz 6 1Department of Computer Science, National Textile University, Faisalabad 37610, Pakistan; engr[email protected] 2Department of Electrical Engineering, Islamia University of Bahawalpur, Bahawalpur 63100, Pakistan 3Department of Electrical Engineering, NFC Institute of Engineering & Technology, Multan 60000, Pakistan; [email protected] 4National Transmission and Despatch Company Ltd., Lahore 54000, Pakistan; [email protected] 5Department of Electric Power and Energy Systems, Dicle University, 21280 Diyarbakır, Turkey; [email protected] 6Department of Electrical Power Engineering, Faculty of Electrical Engineering and Computer Science, VSB-Technical University of Ostrava, 708-00 Ostrava, Czech Republic 7 Department of Electrical Engineering Fundamentals, Faculty of Electrical Engineering, Wroclaw University of Science and Technology, 50-370 Wroclaw, Poland *Correspondence: [email protected] (S.S.); radomir[email protected] (R.G.) Abstract: Short-term load forecasting (STLF) is critical for the energy industry. Accurate predictions of future electricity demand are necessary to ensure power systems’ reliable and efficient operation. Various STLF models have been proposed in recent years, each with strengths and weaknesses. This paper comprehensively reviews some STLF models, including time series, artificial neural networks (ANNs), regression-based, and hybrid models. It first introduces the fundamental concepts and challenges of STLF, then discusses each model class’s main features and assumptions. The paper compares the models in terms of their accuracy, robustness, computational efficiency, scalability, and adaptability and identifies each approach’s advantages and limitations. Although this study suggests that ANNs and hybrid models may be the most promising ways to achieve accurate and reliable STLF, additional research is required to handle multiple input features, manage massive data sets, and adjust to shifting energy conditions. Keywords: short-term load forecasting; neural networks; time series; autoregression; deep learning; artificial intelligence; support vector machines; hybrid models; exponential smoothing; data quality; random forest; decision tree; ensemble methods 1. Introduction The efficient operation and planning of the power system require accurate load forecasting. Short-term load forecasting (STLF) is critical in power system applications, as it allows the optimal scheduling of energy resources and the efficient management of energy storage systems [ 1 ]. Short-term load forecasting (STLF) is a crucial task for the energy industry, as accurate predictions of future electricity demand are necessary to ensure power systems’ reliable and efficient operation [ 2 ]. STLF models, shown in Figure 1, involve predicting the electricity demand for the next few hours, days, or weeks, typically up to a maximum of one month. Accurate load forecasting is essential for efficient energy planning, scheduling, and dispatch, as it helps to balance supply and demand, minimize production costs, and avoid power outages [ 3 ]. Various STLF models have been proposed in recent years, each with strengths and weaknesses [ 4 ]. These models include time series, artificial neural networks (ANNs), regression-based, and hybrid models. Time series models use historical load data to identify patterns and trends in the load profile and make predictions Energies 2023,16, 4060. https://doi.org/10.3390/en16104060 https://www.mdpi.com/journal/energies
Energies 2023,16, 4060 2 of 29 based on statistical and mathematical techniques such as autoregressive integrated moving averages (ARIMA) and exponential smoothing [ 5 ]. ANNs are machine learning models that can capture complex nonlinear relationships between input and output variables and can be trained on large and diverse datasets using backpropagation algorithms [ 6 ]. Regression-based models use linear or nonlinear regression analysis to establish causal relationships between load and environmental factors such as temperature, humidity, and time of day [ 7 ]. Hybrid models combine the strengths of multiple model classes, such as ANNs and time series models, to achieve better accuracy and robustness [ 8 ]. Despite the growing interest in STLF, several challenges and limitations are associated with each model class. Time series models may be limited by their inability to capture long-term trends and non-stationary load data and their sensitivity to data outliers and missing values. ANNs may be limited by their computational complexity, interpretability, and overfitting, as well as their susceptibility to noise and bias in the training data [ 9 ]. Regression-based models may be limited by their assumptions of linearity and additivity in the input–output relationships and their difficulty in handling categorical or non-numeric data [ 10 ]. Hybrid models may be limited by their complexity, parameter tuning, and scalability, as well as their potential overfitting and data redundancy [ 11 ]. To address these challenges, it is essential to compare and evaluate the performance of different STLF models on real-world load data using a variety of metrics and validation techniques. Energies.2023,16,xFORPEERREVIEW2of29 modelsusehistoricalloaddatatoidentifypatternsandtrendsintheloadprofileandmake predictionsbasedonstatisticalandmathematicaltechniquessuchasautoregressiveinte‐ gratedmovingaverages(ARIMA)andexponentialsmoothing[5].ANNsaremachine learningmodelsthatcancapturecomplexnonlinearrelationshipsbetweeninputandout‐ putvariablesandcanbetrainedonlargeanddiversedatasetsusingbackpropagational‐ gorithms[6].Regression‐basedmodelsuselinearornonlinearregressionanalysistoes‐ tablishcausalrelationshipsbetweenloadandenvironmentalfactorssuchastemperature, humidity,andtimeofday[7].Hybridmodelscombinethestrengthsofmultiplemodel classes,suchasANNsandtimeseriesmodels,toachievebetteraccuracyandrobustness [8].DespitethegrowinginterestinSTLF,severalchallengesandlimitationsareassociated witheachmodelclass.Timeseriesmodelsmaybelimitedbytheirinabilitytocapture long‐termtrendsandnon‐stationaryloaddataandtheirsensitivitytodataoutliersand missingvalues.ANNsmaybelimitedbytheircomputationalcomplexity,interpretability, andoverfitting,aswellastheirsusceptibilitytonoiseandbiasinthetrainingdata[9]. Regression‐basedmodelsmaybelimitedbytheirassumptionsoflinearityandadditivity intheinput–outputrelationshipsandtheirdifficultyinhandlingcategoricalornon‐nu‐ mericdata[10].Hybridmodelsmaybelimitedbytheircomplexity,parametertuning, andscalability,aswellastheirpotentialoverfittinganddataredundancy[11].Toaddress thesechallenges,itisessentialtocompareandevaluatetheperformanceofdifferentSTLF modelsonreal‐worldloaddatausingavarietyofmetricsandvalidationtechniques. Figure1.STLFmodelworking. Giventhefast‐pacedprogressinmachinelearning,artificialintelligence,anddata processingmethods,thereisacontinuousneedforupdatedreviewsencompassingthe latestdevelopmentsinshort‐termloadforecasting(STLF)models.Thisreviewpaperaims tocovertheserecentadvancementsandprovideinsightsintostate‐of‐the‐artmodels, whichmightnotbecoveredinolderreviews.Astheworldmovestowardamoresustain‐ ableenergyfuture,accurateloadforecastingisbecomingevenmorecriticalinmanaging powersystemsʹincreasingcomplexityandvariability.Table1liststhecontributionsand limitationsofthelatestpublishedarticlesonthetopicofshort‐termloadforecasting. Table1.Contributionsandlimitationsofrecentpublications. Refs.ContributionsLimitations [12]1. Acomprehensivereviewofdeeplearningtechniques 2. DetailedanalysisofCNNs,LSTMs,andGRUs 1. Limitedtodeeplearningtechniques 2. Lackscomparisonwithtraditionalorhybridmethods [13]1. Comparativestudyofhybridmodels 2. Clearanalysisofmodelperformance 1. Limitedtospecifichybridcombinations 2. Lacksanalysisofindividualmodelcomponents [14]1. Meta‐analysisofensemblelearningtechniques 2. Thoroughdiscussionofbagging,boosting,andstacking 1. Limitedtoensembletechniques 2. Doesnotconsiderstandalonemodels [15]1. Casestudyonfeatureselectioninloadforecasting 2. Demonstratesthepotentialoffeatureselection 1. Limitedtoonecasestudy 2. Lacksgeneralizationtootherscenarios [16]1. ExploresBayesianneuralnetworksforforecasting. 2. Explanationofmethodologyandadvantages 1. Limitedtoprobabilisticmethods 2. Lackscomparisonwithdeterministicapproaches Figure 1. STLF model working. Given the fast-paced progress in machine learning, artificial intelligence, and data processing methods, there is a continuous need for updated reviews encompassing the latest developments in short-term load forecasting (STLF) models. This review paper aims to cover these recent advancements and provide insights into state-of-the-art models, which might not be covered in older reviews. As the world moves toward a more sustainable energy future, accurate load forecasting is becoming even more critical in managing power systems’ increasing complexity and variability. Table 1lists the contributions and limitations of the latest published articles on the topic of short-term load forecasting. Table 1. Contributions and limitations of recent publications. Refs. Contributions Limitations [12]1. A comprehensive review of deep learning techniques 2. Detailed analysis of CNNs, LSTMs, and GRUs 1. Limited to deep learning techniques 2. Lacks comparison with traditional or hybrid methods [13]1. Comparative study of hybrid models 2. Clear analysis of model performance 1. Limited to specific hybrid combinations 2. Lacks analysis of individual model components [14]1. Meta-analysis of ensemble learning techniques 2. Thorough discussion of bagging, boosting, and stacking 1. Limited to ensemble techniques 2. Does not consider standalone models
Energies 2023,16, 4060 3 of 29 Table 1. Cont. Refs. Contributions Limitations [15]1. Case study on feature selection in load forecasting 2. Demonstrates the potential of feature selection 1. Limited to one case study 2. Lacks generalization to other scenarios [16]1. Explores Bayesian neural networks for forecasting. 2. Explanation of methodology and advantages 1. Limited to probabilistic methods 2. Lacks comparison with deterministic approaches This review paper highlights the challenges in the field of short-term load forecasting and suggests suitable solutions. The latest development in statistical, hybrid, and intelligent STLF models is explored with mathematical analysis. The various features of deep learning strategies are analyzed. This in-depth analysis is valuable for researchers and practitioners, enabling them to make informed decisions when selecting and applying appropriate forecasting models. Our paper presents a critical analysis of the models’ advantages and limitations, offering insights into potential improvements and suggesting new approaches that can enhance the performance of STLF models. This contribution aids in advancing the field by addressing existing challenges and identifying areas for future research. 2. Challenges and Solutions One of the critical challenges in developing accurate STLF models is dealing with the dynamic nature of load data. Weather, economic conditions, and consumer behavior affect load data, which can change over time [ 17 ]. Therefore, STLF models need to be able to adapt to changes in the underlying data-generating process. One approach to dealing with non-stationary data is using time-varying models, such as autoregressive moving average (TVARMA) models [ 13 ]. TVARMA models allow autoregressive coefficients and moving average terms to vary over time, allowing the model to capture changes in the underlying data-generating process. TVARMA models have been shown to outperform stationary models such as ARIMA in some STLF applications. Another approach to dealing with non-stationary data is to use machine learning models, such as deep learning models [ 18 ]. Deep learning models, such as deep neural networks (DNNs), can automatically learn complex patterns and relationships and adapt to changes in the data-generating process [ 18 ]. However, training and implementing deep learning models require much data and computational resources. In addition to dealing with non-stationary data, another challenge in STLF is uncertainty and risk management. STLF models provide point forecasts of the future load values but do not provide information on the uncertainty or risk associated with the forecasts. Therefore, there is a need for models that can provide probabilistic forecasts, such as quantile regression models [ 19 ]. Quantile regression models provide estimates of the conditional quantiles of the load distribution, allowing for the construction of prediction intervals and risk assessment. Quantile regression models are effective in providing probabilistic forecasts in STLF applications. One more strategy to deal with uncertainty and risk management is scenario-based forecasting. Scenario-based forecasting involves generating multiple scenarios of the future load values based on the different assumptions of the underlying data-generating process [ 20 ]. Scenario-based forecasting allows for assessing the risk associated with different scenarios and can inform decision-making under uncertainty. STLF models must also be scalable and adaptable to different power systems and operating conditions. Power systems vary in size, complexity, and generation mix, and STLF models must be able to accommodate these variations [ 21 ]. Transfer learning is one approach to developing scalable and adaptable STLF models. Transfer learning involves using a pre-trained model on one power system and adapting it to another. Transfer learning can reduce the data and computational resources required to train a new model and enable the transfer of knowledge and expertise across different power systems. Using metalearning is a different strategy for developing scalable and adaptable STLF models. Meta-
Energies 2023,16, 4060 4 of 29 learning involves learning the optimal model and hyperparameters for a new power system based on the characteristics of the power system and the available data. Meta-learning can reduce the time and resources required to develop a new STLF model and enable the development of models tailored to the power system’s specific characteristics [22]. Another challenge in STLF is dealing with the uncertainty and variability of renewable energy sources, such as wind and solar. Renewable energy sources are highly variable and uncertain, and their integration into power systems can affect the accuracy of STLF models [ 23 ]. One method of handling the variability of renewable energy sources is to use hybrid models that combine STLF and renewable energy forecasting models. Renewable energy forecasting models use weather data and other predictors to forecast the output of renewable energy sources, such as wind and solar. Hybrid models can combine the forecasts of the STLF models with the forecasts of the renewable energy forecasting models, improving the load forecasts’ accuracy [ 24 ]. Another strategy to cope with the variability of renewable energy sources is the use of demand response and energy storage. Demand response involves incentivizing consumers to reduce their electricity consumption during periods of high demand or low renewable energy output, while energy storage involves storing excess renewable energy for later use. Demand response and energy storage can reduce the variability and uncertainty of renewable energy sources and improve STLF model accuracy. Another difficulty in STLF is addressing the high-dimensional and complex nature of the data. STLF models often require many predictors, such as weather data, holiday schedules, and economic indicators [ 25 ]. However, using too many predictors can lead to overfitting and reduced accuracy. One strategy for handling the high-dimensional and complex nature of the data is to use feature selection techniques. Feature selection techniques involve selecting a subset of the most relevant predictors for the forecasting task [ 26 ]. This strategy can reduce the number of predictors used in the model, reduce overfitting, and improve the accuracy of the forecasts. Another method for handling the high-dimensional and complex nature of the data is to use dimensionality reduction techniques. Dimensionality reduction techniques involve transforming the data into a lower-dimensional space while preserving the essential information. This technique can reduce the number of predictors used in the model, reduce the computational resources required, and improve the accuracy of the forecasts [27]. STLF also faces the difficulty of the non-Gaussian and heavy-tailed nature of the load data. Load data often exhibits non-Gaussian and heavy-tailed distributions, which can violate the assumptions of many STLF models [ 28 ]. One method to deal with the nonGaussian and heavy-tailed nature of the load data is to use robust STLF models. Robust STLF models are designed to be less sensitive to outliers and heavy-tailed distributions and can provide more accurate forecasts in these scenarios. Another technique to deal with the non-Gaussian and heavy-tailed nature of the load data is to use distributional STLF models [ 29 ]. Distributional STLF models model the entire distribution of the load data rather than just the mean, allowing for the estimation of quantiles, prediction intervals, and risk assessment. A further challenge in STLF is dealing with the nonlinear and non-monotonic relationships between the load data, weather data, and other predictors [ 30 ]. STLF models often require nonlinear and non-monotonic models to capture these relationships. One strategy to cope with nonlinear and non-monotonic relationships is to use non-parametric models, such as decision trees or random forests [ 31 ]. Non-parametric models can handle nonlinear and non-monotonic relationships and automatically learn complex patterns and relationships in the data. One more approach to dealing with nonlinear and non-monotonic relationships is to use kernel-based models, such as kernel regression or support vector machines [ 32 ]. Kernel-based models can handle nonlinear and non-monotonic relationships and be more computationally efficient than non-parametric models.
Energies 2023,16, 4060 5 of 29 Another critical challenge in STLF is dealing with the spatial dimension of load data. Load data varies over time and space, and STLF models must account for spatial dependencies and heterogeneity. One way to deal with the spatial dimension of load data is to use spatial–temporal models. Spatial–temporal models account for the spatial and temporal dependencies of the load data and can provide more accurate forecasts by incorporating information from neighboring locations [ 33 ]. Another strategy to handle the spatial dimension of load data is to use clustering and spatial interpolation techniques. Clustering techniques involve grouping similar places based on their load patterns. By contrast, spatial interpolation techniques involve estimating the load values at unobserved locations based on the load values at nearby observed locations. In addition to the challenges mentioned above, STLF models must be transparent and interpretable. STLF models are used in critical decision-making processes, such as load shedding and demand response, and power system operators and decision-makers need to understand their results quickly. One method of developing transparent and interpretable STLF models is to use explainable artificial intelligence (XAI) techniques [ 34 ]. XAI techniques involve developing models that explain their decisions and predictions. This can increase the transparency and trustworthiness of the models and enable power system operators and decision-makers to better understand the underlying factors that contribute to the load forecasts. Another way to develop transparent and interpretable STLF models is to use causal inference techniques [ 35 ]. Causal inference techniques involve developing models that can identify the causal relationships between the load data and other predictors and provide insights into the underlying drivers of load demand. Finally, another challenge in STLF is dealing with the lack of data and data quality issues. STLF models require a large amount of data to train and validate, and the data quality can affect the forecasts’ accuracy [ 36 ]. Data augmentation techniques are one method to deal with the lack of data and data quality issues. Data augmentation techniques involve generating new data from the existing data by adding noise, perturbing the data, or generating synthetic data. Data augmentation can increase the data available for training and improve the model’s generalization. Another way to deal with the lack of data and data quality issues is to use transfer learning and domain adaptation techniques [ 37 ]. These techniques can reduce the data required to train a new model and enable the transfer of knowledge and expertise across different power systems. 3. Development in STLF Models Traditional STLF models use historical load, weather, and calendar data as input features. However, the availability of new data sources, such as social media data and smart meter data, presents an opportunity to develop more accurate and robust STLF models. Advanced machine learning techniques, such as deep learning and reinforcement learning, have shown great potential in improving the accuracy of STLF models [ 38 ]. Deep learning techniques can automatically learn complex patterns and relationships in the data, while reinforcement learning can learn to optimize actions based on feedback from the environment [ 39 ]. Probabilistic forecasting provides a measure of uncertainty around the point forecast, which can help power system operators make more informed decisions in the face of uncertainty. Various STLF models have been developed, but they can be broadly classified into statistical, intelligent, and hybrid [ 40 ]. Figure 2shows three main types of STLF models: statistical, intelligent, and hybrid models.
Energies 2023,16, 4060 6 of 29 Energies.2023,16,xFORPEERREVIEW5of29 dimensionofloaddataistouseclusteringandspatialinterpolationtechniques.Clustering techniquesinvolvegroupingsimilarplacesbasedontheirloadpatterns.Bycontrast,spa‐ tialinterpolationtechniquesinvolveestimatingtheloadvaluesatunobservedlocations basedontheloadvaluesatnearbyobservedlocations. Inadditiontothechallengesmentionedabove,STLFmodelsmustbetransparentand interpretable.STLFmodelsareusedincriticaldecision‐makingprocesses,suchasload sheddinganddemandresponse,andpowersystemoperatorsanddecision‐makersneed tounderstandtheirresultsquickly.Onemethodofdevelopingtransparentandinterpret‐ ableSTLFmodelsistouseexplainableartificialintelligence(XAI)techniques[34].XAI techniquesinvolvedevelopingmodelsthatexplaintheirdecisionsandpredictions.This canincreasethetransparencyandtrustworthinessofthemodelsandenablepowersystem operatorsanddecision‐makerstobetterunderstandtheunderlyingfactorsthatcontribute totheloadforecasts.AnotherwaytodeveloptransparentandinterpretableSTLFmodels istousecausalinferencetechniques[35].Causalinferencetechniquesinvolvedeveloping modelsthatcanidentifythecausalrelationshipsbetweentheloaddataandotherpredic‐ torsandprovideinsightsintotheunderlyingdriversofloaddemand. Finally,anotherchallengeinSTLFisdealingwiththelackofdataanddataquality issues.STLFmodelsrequirealargeamountofdatatotrainandvalidate,andthedata qualitycanaffecttheforecasts’accuracy[36].Dataaugmentationtechniquesareone methodtodealwiththelackofdataanddataqualityissues.Dataaugmentationtech‐ niquesinvolvegeneratingnewdatafromtheexistingdatabyaddingnoise,perturbing thedata,orgeneratingsyntheticdata.Dataaugmentationcanincreasethedataavailable fortrainingandimprovethemodelʹsgeneralization.Anotherwaytodealwiththelackof dataanddataqualityissuesistousetransferlearninganddomainadaptationtechniques [37].Thesetechniquescanreducethedatarequiredtotrainanewmodelandenablethe transferofknowledgeandexpertiseacrossdifferentpowersystems. 3.DevelopmentinSTLFModels TraditionalSTLFmodelsusehistoricalload,weather,andcalendardataasinputfea‐ tures.However,theavailabilityofnewdatasources,suchassocialmediadataandsmart meterdata,presentsanopportunitytodevelopmoreaccurateandrobustSTLFmodels. Advancedmachinelearningtechniques,suchasdeeplearningandreinforcementlearn‐ ing,haveshowngreatpotentialinimprovingtheaccuracyofSTLFmodels[38].Deep learningtechniquescanautomaticallylearncomplexpatternsandrelationshipsinthe data,whilereinforcementlearningcanlearntooptimizeactionsbasedonfeedbackfrom theenvironment[39].Probabilisticforecastingprovidesameasureofuncertaintyaround thepointforecast,whichcanhelppowersystemoperatorsmakemoreinformeddecisions inthefaceofuncertainty.VariousSTLFmodelshavebeendeveloped,buttheycanbe broadlyclassifiedintostatistical,intelligent,andhybrid[40].Figure2showsthreemain typesofSTLFmodels:statistical,intelligent,andhybridmodels. Figure2.MaintypesofSTLFmodels. Figure 2. Main types of STLF models. 3.1. Statistical Models Statistical models are based on time series analysis and can capture the temporal patterns of load demand. Figure 3shows common statistical models for STLF including autoregressive integrated moving averages (ARIMA), seasonal ARIMA (SARIMA), exponential smoothing (ES), and generalized linear models (GLM) [ 41 ]. ARIMA models assume that the current value of the load demand is a function of its past values and the random error term. SARIMA models incorporate seasonal patterns in the data, which can be helpful in power system applications where load demand exhibits daily, weekly, or monthly cycles. ES models use a weighted average of the past load demand values to predict future values [ 42 ]. Statistical models are relatively simple and require low computational resources. However, they may not be able to capture the nonlinear relationships and complex dynamics of power system systems, which can result in lower forecasting accuracy [ 43 ]. Short-term load forecasting (STLF) is a critical component of energy management systems (EMS) for power system applications. Accurate STLF is essential for optimizing power system energy supply and demand. It can improve energy efficiency, reduce costs, and enhance reliability [44]. Energies.2023,16,xFORPEERREVIEW6of29 3.1.StatisticalModels Statisticalmodelsarebasedontimeseriesanalysisandcancapturethetemporalpat‐ ternsofloaddemand.Figure3showscommonstatisticalmodelsforSTLFincludingau‐ toregressiveintegratedmovingaverages(ARIMA),seasonalARIMA(SARIMA),expo‐ nentialsmoothing(ES),andgeneralizedlinearmodels(GLM)[41].ARIMAmodelsas‐ sumethatthecurrentvalueoftheloaddemandisafunctionofitspastvaluesandthe randomerrorterm.SARIMAmodelsincorporateseasonalpatternsinthedata,whichcan behelpfulinpowersystemapplicationswhereloaddemandexhibitsdaily,weekly,or monthlycycles.ESmodelsuseaweightedaverageofthepastloaddemandvaluesto predictfuturevalues[42].Statisticalmodelsarerelativelysimpleandrequirelowcompu‐ tationalresources.However,theymaynotbeabletocapturethenonlinearrelationships andcomplexdynamicsofpowersystemsystems,whichcanresultinlowerforecasting accuracy[43].Short‐termloadforecasting(STLF)isacriticalcomponentofenergyman‐ agementsystems(EMS)forpowersystemapplications.AccurateSTLFisessentialforop‐ timizingpowersystemenergysupplyanddemand.Itcanimproveenergyefficiency,re‐ ducecosts,andenhancereliability[44]. Figure3.Typesofstatisticalmodels. 3.1.1.AutoregressiveIntegratedMovingAverage(ARIMA)Models ARIMAmodelsarewidelyusedinSTLFapplicationsduetotheirsimplicityandabil‐ itytocapturethetemporaldependenceoftheloaddata.ARIMAmodelsarebasedon threecomponents:theautoregressive(AR)component,theintegrated(I)component,and themovingaverage(MA)component[45].TheARcomponentmodelsthedependenceof theloadonitspastvalues,theIcomponentmodelsthetrendintheloaddata,andtheMA componentmodelsthedependencyonpasterrors.ARIMAmodelscanbecustomizedby adjustingtheparametersofthethreecomponents,suchastheorderoftheARandMA componentsandthedegreeofdifferencingintheIcomponent[46].However,ARIMA modelsassumethattheloaddatafollowsastationaryprocess,whichmaynotalwaysbe validforpowersystemapplications. Furthermore,ARIMAmodelsmaynotcapturethenonlinearrelationshipsbetween theloadandotherfactorsthatinfluencetheload,suchasweatherandoccupancy.The algorithmofARIMAmodelsisshowninFigure4,TheARIMAmodelisdenotedas ARIMA(p,d,q),wherepistheorderoftheARmodel,distheorderofdifferencing,andq istheorderoftheMAmodel.Theseparametersarechosenbasedonthecharacteristicsof thetimeseriesbeinganalyzedandcanbeestimatedusingstatisticalmethodssuchas maximumlikelihoodestimation.Oncetheparametershavebeenevaluated,theARIMA modelcanbeusedtoforecastfuturevaluesofthetimeseries.Themodelworksbyusing pastobservationstogenerateapredictionofthenextvalueintheseries,basedontheAR andMAcomponentsofthemodel,andthenusingthispredictedvaluetoupdatetheerror termandmakeanewpredictionforthefollowingtimestep.Thisprocessisrepeatedre‐ cursivelytogenerateaforecastforaspecifiednumberoftimestepsintothefuture. TheARcomponentassumesthatthecurrentvalueofthetimeseriesdependsonits previousvalues.AnARmodeloforderpisrepresentedasAR(p),andtheequationis: Figure 3. Types of statistical models. 3.1.1. Autoregressive Integrated Moving Average (ARIMA) Models ARIMA models are widely used in STLF applications due to their simplicity and ability to capture the temporal dependence of the load data. ARIMA models are based on three components: the autoregressive (AR) component, the integrated (I) component, and the moving average (MA) component [ 45 ]. The AR component models the dependence of the load on its past values, the I component models the trend in the load data, and the MA component models the dependency on past errors. ARIMA models can be customized by adjusting the parameters of the three components, such as the order of the AR and MA components and the degree of differencing in the I component [ 46 ]. However, ARIMA models assume that the load data follows a stationary process, which may not always be valid for power system applications. Furthermore, ARIMA models may not capture the nonlinear relationships between the load and other factors that influence the load, such as weather and occupancy. The algorithm of ARIMA models is shown in Figure 4, The ARIMA model is denoted as ARIMA(p,d,q), where p is the order of the AR model, d is the order of differencing, and q is the order of the MA model. These parameters are chosen based on the characteristics
Energies 2023,16, 4060 7 of 29 of the time series being analyzed and can be estimated using statistical methods such as maximum likelihood estimation. Once the parameters have been evaluated, the ARIMA model can be used to forecast future values of the time series. The model works by using past observations to generate a prediction of the next value in the series, based on the AR and MA components of the model, and then using this predicted value to update the error term and make a new prediction for the following time step. This process is repeated recursively to generate a forecast for a specified number of time steps into the future. Energies.2023,16,xFORPEERREVIEW7of29 𝑌𝑡𝑐𝜑 𝑌𝑡1𝜑 𝑌𝑡2……..𝜑 𝑌𝑡𝑝 𝜀𝑡 (1) where: Y(t)isthevalueofthetimeseriesattimet; cisaconstant; φ₁,φ₂,...,φₚaretheautoregressivecoefficients; pistheorderoftheARmodel; ε(t)istheerrortermattimet. TheparameterddenotesdifferencingintheARIMAmodel.Thefirst‐orderdiffer‐ encingofatimeseriesis: ∆𝑌𝑡𝑌 𝑡𝑌 𝑡1(2) Higher‐orderdifferencingcanbeappliedbydifferencingthealreadydifferencedse‐ riesmultipletimes.Forexample,thesecond‐orderdifferencingis: ∆ 𝑌𝑡𝑌′ 𝑡𝑌′ 𝑡1(3) TheMAcomponentrepresentsthedependencyofthecurrentvalueofthetimeseries onthepasterrorterms.AnMAmodeloforderqisrepresentedasMA(q),andtheequa‐ tionis: 𝑌𝑡𝑐𝜀 𝑡𝜃 𝜀𝑡1𝜃 𝜀𝑡2⋯𝜃 𝜀𝑡𝑞 (4) where: Y(t)isthevalueofthetimeseriesattimet; cisaconstant; ε(t)istheerrortermattimet; θ₁,θ₂,…,θ q arethemovingaveragecoefficients; qistheorderoftheMAmodel. CombiningtheAR,I,andMAcomponents,wecanrepresentanARIMA(p,d,q)model as: ∆ 𝑌𝑡𝑐𝜑 ∆ 𝑌𝑡1⋯.𝜑 ∆ 𝑌𝑡𝑝 𝜀 𝑡𝜃 𝜀𝑡1⋯𝜃 𝜀𝑡𝑞 𝑡∆𝑌 𝑡∆𝑌 𝑡1(5) Figure4.AlgorithmoftheARIMAmodel. Figure 4. Algorithm of the ARIMA model. The AR component assumes that the current value of the time series depends on its previous values. An AR model of order p is represented as AR(p), and the equation is: Y(t)=c+ϕ1Y(t−1)+ϕ2Y(t−2).........+ϕpY(t−p)+ε(t)(1) where: Y(t) is the value of the time series at time t; cis a constant; ϕ1,ϕ2, ..., ϕpare the autoregressive coefficients; pis the order of the AR model; ε(t) is the error term at time t. The parameter d denotes differencing in the ARIMA model. The first-order differencing of a time series is: ∆Y(t)=Y(t)−Y(t−1)(2) Higher-order differencing can be applied by differencing the already differenced series multiple times. For example, the second-order differencing is: ∆2Y(t)=Y0(t)−Y0(t−1)(3) The MA component represents the dependency of the current value of the time series on the past error terms. An MA model of order qis represented as MA(q), and the equation is: Y(t)=c+ε(t)−θ1ε(t−1)−θ2ε(t−2)−. . . −θqε(t−q)(4)
Energies 2023,16, 4060 8 of 29 where: Y(t) is the value of the time series at time t; cis a constant; ε(t) is the error term at time t; θ1,θ2, . . . , θqare the moving average coefficients; qis the order of the MA model. Combining the AR, I, and MA components, we can represent an ARIMA(p,d,q) model as: ∆dY(t)=c+ϕ1∆dY(t−1)+. . . ϕp∆dY(t−p)+ε(t)−θ1ε(t−1)−. . . −θqε(t−q)(t)=∆Y(t)−∆Y(t−1)(5) 3.1.2. Seasonal Autoregressive Integrated Moving Average (SARIMA) Models SARIMA models are extensions of ARIMA models that can capture the seasonal patterns in the load data. SARIMA models include additional parameters to model the seasonal variation in the load data, such as the seasonal period and the order of the AR, I, and MA components [ 47 ]. SARIMA models are helpful for STLF applications in the power system that exhibit strong seasonal patterns, such as peak load periods during the day or week. However, SARIMA models suffer from the same limitations as ARIMA models, such as the assumption of a stationary process and the inability to capture nonlinear relationships between the load and other factors [ 48 ]. SARIMA models may also require a large amount of historical data to estimate the seasonal parameters accurately, which may not be available for new power system installations. The algorithm of SARIMA models is shown in Figure 5, The SARIMA model is specified using three main parameters: p, d, and q for the non-seasonal component, and P, D, and Q for the seasonal component. The p parameter represents the order of autoregression, the d parameter represents the degree of differencing, and the q parameter represents the order of moving average for the non-seasonal component. Energies.2023,16,xFORPEERREVIEW8of29 3.1.2.SeasonalAutoregressiveIntegratedMovingAverage(SARIMA)Models SARIMAmodelsareextensionsofARIMAmodelsthatcancapturetheseasonalpat‐ ternsintheloaddata.SARIMAmodelsincludeadditionalparameterstomodelthesea‐ sonalvariationintheloaddata,suchastheseasonalperiodandtheorderoftheAR,I, andMAcomponents[47].SARIMAmodelsarehelpfulforSTLFapplicationsinthepower systemthatexhibitstrongseasonalpatterns,suchaspeakloadperiodsduringthedayor week.However,SARIMAmodelssufferfromthesamelimitationsasARIMAmodels, suchastheassumptionofastationaryprocessandtheinabilitytocapturenonlinearrela‐ tionshipsbetweentheloadandotherfactors[48].SARIMAmodelsmayalsorequirea largeamountofhistoricaldatatoestimatetheseasonalparametersaccurately,whichmay notbeavailablefornewpowersysteminstallations.ThealgorithmofSARIMAmodelsis showninFigure5,TheSARIMAmodelisspecifiedusingthreemainparameters:p,d, andqforthenon‐seasonalcomponent,andP,D,andQfortheseasonalcomponent.The pparameterrepresentstheorderofautoregression,thedparameterrepresentsthedegree ofdifferencing,andtheqparameterrepresentstheorderofmovingaverageforthenon‐ seasonalcomponent. Similarly,theP,D,andQparametersrepresenttheorderofautoregression,thede‐ greeofdifferencing,andtheorderofmovingaveragesfortheseasonalcomponent.The algorithmforfittingaSARIMAmodelinvolvesseveralsteps.First,themodelparameters areestimatedusingmaximumlikelihoodestimation.Thisprocessinvolvesselectingthe valuesofp,d,q,P,D,andQthatmaximizethelikelihoodoftheobserveddata[49].Once theparametersareestimated,themodelisfittedtothedatausingforecasting.Thisstrat‐ egyinvolvesusingthemodeltopredictfutureperiodsbasedonhistoricaldata. Figure5.AlgorithmofSARIMAmodel. TheequationsfortheSARIMAmodelcomponentsareasfollows: Autoregressive(AR)component: 𝜑𝐵𝑋𝑐𝜗 𝐵𝜀(6) where𝜑𝐵istheautoregressiveoperator,Bisthebackshiftoperator,Xtisthetimeseries attimet,cisaconstant,𝜗𝐵isthemovingaverageoperator,andεtistheerrorterm. Differencing(I)component: 1𝐵 𝑋𝑌 (7) where(1−B)disthedifferencingoperator,andYtisthedifferencedtimeseries. Figure 5. Algorithm of SARIMA model. Similarly, the P, D, and Q parameters represent the order of autoregression, the degree of differencing, and the order of moving averages for the seasonal component. The algorithm for fitting a SARIMA model involves several steps. First, the model parameters are estimated using maximum likelihood estimation. This process involves selecting the values of p, d, q, P, D, and Q that maximize the likelihood of the observed data [49]. Once the parameters are estimated, the model is fitted to the data using forecasting. This strategy involves using the model to predict future periods based on historical data.
Energies 2023,16, 4060 9 of 29 The equations for the SARIMA model components are as follows: Autoregressive (AR) component: ϕ(B)Xt=c+ϑ(B)εt(6) where ϕ(B) is the autoregressive operator, Bis the backshift operator, X t is the time series at time t,cis a constant, ϑ(B)is the moving average operator, and εtis the error term. Differencing (I) component: (1−B)dXt=Yt(7) where (1 −B)dis the differencing operator, and Ytis the differenced time series. Moving average (MA) component: Xt=µ+εt+θ1e(t−1)+. . . +θqe(t−q)(8) where µis the series’ mean and θiare the MA coefficients. Seasonal autoregressive (SAR) component: ∅s(Bs)Yt=ϑs(Bs)εt(9) where ∅s(Bs) is the seasonal autoregressive operator and ϑs(Bs) is the seasonal moving average operator. Seasonal differencing (SI) component: (1−Bs)DYt=Zt(10) where (1 − B s ) D is the seasonal differencing operator, and Z t is the seasonally differenced time series. Seasonal moving average (SMA) component: Yt=εt+θsε(t−s)+. . . +θsQε(t−Qs)(11) where θsi are the seasonal MA coefficients. Combining these components, the SARIMA model equation can be represented as: ϕ(B)ϕs(Bs)(1−B)d(1−Bs)DXt=c+ϑ(B)ϑs(Bs)εt(12) 3.1.3. Exponential Smoothing (ES) Models ES models are time series models that use an exponentially weighted average of past observations to forecast future values. ES models can be customized by adjusting the smoothing parameter, which controls the weights assigned to past observations. ES models include several variants, such as simple exponential smoothing (SES), Holt’s linear exponential smoothing (Holt), and Holt–Winters seasonal exponential smoothing (HW) [ 49 ]. SES models use a single smoothing parameter to forecast the load data based on past values. Holt models include an additional trend component to capture the linear trend in the load data. HW models include both a trend and a seasonal component to capture the seasonal patterns in the load data. HW models are helpful for STLF applications in the power system that exhibit trend and seasonal patterns [50]. ES models are computationally efficient and require less historical data than ARIMA and SARIMA models. ES models can also capture the nonlinear relationships between the load and other factors that influence the load, such as weather and occupancy. However, ES models assume that the load data follows a stationary process and may not perform well for power systems with non-stationary load data [ 51 ]. The algorithm of ES models is shown in Figure 6. The ES model is specified using two main parameters: alpha and beta. The alpha parameter controls the weight given to the most recent observation, while the
Energies 2023,16, 4060 16 of 29 have been used successfully in power system applications to forecast the load based on temperature, humidity, and solar radiation. Figure 10 shows the working of a random forest algorithm. The m features are selected from the incoming instances. The different numbers of trees are made, leading to unique prediction classes. The majority vote determines the final class [66]. Energies.2023,16,xFORPEERREVIEW15of29 Figure9.Blockdiagramofadecisiontree. 3.2.3.RandomForestandGradientBoosting Randomforest(RF)isanextensionofDTsthatcanhandleoverfittingandimprove theaccuracyoftheloadforecasts.RFsconstructmultipledecisiontreesusingbootstrap samplesofthetrainingdataandaveragingthepredictionsoftheindividualtrees[66].RFs havebeenusedsuccessfullyinpowersystemapplicationstoforecasttheloadbasedon temperature,humidity,andsolarradiation.Figure10showstheworkingofarandom forestalgorithm.Themfeaturesareselectedfromtheincominginstances.Thedifferent numbersoftreesaremade,leadingtouniquepredictionclasses.Themajorityvotedeter‐ minesthefinalclass[66]. Figure10.Principleofrandomforest. Gradientboosting(GB)isanensembleMLmodelthatcanimprovetheaccuracyof loadforecasts[67].GBworksbysequentiallyaddingdecisiontreestothemodel,which correctstheerrorsoftheprevioustrees,resultinginafinalmodelthatcancapturethe Figure 10. Principle of random forest. Gradient boosting (GB) is an ensemble ML model that can improve the accuracy of load forecasts [ 67 ]. GB works by sequentially adding decision trees to the model, which corrects the errors of the previous trees, resulting in a final model that can capture the nonlinear relationships between the load and other factors that influence the load. GB has been used successfully in power system applications to forecast the load based on weather conditions, occupancy, and time of day [ 68 ]. Figure 11 is a block diagram of the gradient boosting algorithm. The data training determines the weak learners used to make a more accurate prediction. Energies.2023,16,xFORPEERREVIEW16of29 nonlinearrelationshipsbetweentheloadandotherfactorsthatinfluencetheload.GBhas beenusedsuccessfullyinpowersystemapplicationstoforecasttheloadbasedonweather conditions,occupancy,andtimeofday[68].Figure11isablockdiagramofthegradient boostingalgorithm.Thedatatrainingdeterminestheweaklearnersusedtomakeamore accurateprediction. Figure11.Blockdiagramofthegradientboostingalgorithm. Thestep‐wiseworkingofrandomforestandgradientboostingalgorithmsisbelow. 1. Thefirststepinvolvescollectinghistoricalelectricityloaddataandrelevantexoge‐ nousvariablessuchasweatherdata,dayoftheweek,andtimeoftheday.Thesedata arecleanedandpreprocessedtoremoveinconsistencies,outliers,ormissingvalues. Featurescalingisgenerallynotrequiredforrandomforestasdecisiontrees,itsbase learners,arelesssensitivetothescaleofinputfeatures. 2. Themostrelevantfeaturesfortheforecastingtaskareselectedtoensurethatirrele‐ vantorredundantfeaturesdonotnegativelyimpactthemodel.Althoughrandom foresthasaninherentabilitytohandlemanyfeaturesandautomaticallyestimate featureimportance,usingdomainknowledgeortechniquessuchasrecursivefeature elimination(RFE)andcorrelationanalysiscanhelpfurtherimprovemodelperfor‐ mance. 3. Withthepreprocesseddataandselectedfeatures,therandomforestmodelistrained. Thealgorithmcreatesmultipledecisiontrees,andeachtreeistrainedonadifferent bootstrapsampleoftheoriginaldataset(samplingwithreplacement).Additionally, arandomsubsetoffeaturesisconsideredateachsplitinthetreeconstructionpro‐ cess,whichintroducesfurtherdiversityamongthetreesandreducesoverfitting. 4. Thecross‐validationtechniqueensuresthattherandomforestmodelperformswell onunseendata.Thedatasetisdividedintotrainingandvalidationsubsets,withthe modelbeingtrainedononesubsetandtestedontheother.Thisprocesshelpsassess themodelʹsperformanceandgeneralizability. 5. Randomforesthasseveralhyperparameters,suchasthenumberoftrees(n_estima‐ tors),themaximumdepthofthetrees,andtheminimumnumberofsamplesrequired tosplitanode.Thesehyperparameterscanbetunedusingtechniquessuchasgrid orrandomsearchandcross‐validationtofindthebestcombinationforthespecific STLFproblem. Figure 11. Block diagram of the gradient boosting algorithm.
Energies 2023,16, 4060 17 of 29 The step-wise working of random forest and gradient boosting algorithms is below. 1. The first step involves collecting historical electricity load data and relevant exogenous variables such as weather data, day of the week, and time of the day. These data are cleaned and preprocessed to remove inconsistencies, outliers, or missing values. Feature scaling is generally not required for random forest as decision trees, its base learners, are less sensitive to the scale of input features. 2. The most relevant features for the forecasting task are selected to ensure that irrelevant or redundant features do not negatively impact the model. Although random forest has an inherent ability to handle many features and automatically estimate feature importance, using domain knowledge or techniques such as recursive feature elimination (RFE) and correlation analysis can help further improve model performance. 3. With the preprocessed data and selected features, the random forest model is trained. The algorithm creates multiple decision trees, and each tree is trained on a different bootstrap sample of the original dataset (sampling with replacement). Additionally, a random subset of features is considered at each split in the tree construction process, which introduces further diversity among the trees and reduces overfitting. 4. The cross-validation technique ensures that the random forest model performs well on unseen data. The dataset is divided into training and validation subsets, with the model being trained on one subset and tested on the other. This process helps assess the model’s performance and generalizability. 5. Random forest has several hyperparameters, such as the number of trees (n_estimators), the maximum depth of the trees, and the minimum number of samples required to split a node. These hyperparameters can be tuned using techniques such as grid or random search and cross-validation to find the best combination for the specific STLF problem. 6. Once the random forest model has been trained, validated, and tested, it can make short-term load forecasts based on new input data. The model takes in the relevant features for the desired forecasting period and produces a prediction from each decision tree. The final prediction is the average of the individual tree predictions, which provides a more accurate and stable forecast. 3.2.4. Multilayer Perceptron Model A multilayer perceptron (MLP) is an artificial neural network consisting of multiple layers of interconnected nodes, also known as neurons or perceptrons. It is widely used in supervised learning tasks, such as classification and regression. An MLP is shown in Figure 12, it has the following components [69]: 1. Input layer: This is the first layer of the MLP model, receiving the input data (e.g., numbers, images, and text). Each node in this layer corresponds to a single input data feature. 2. Hidden layers: These are the layers between the input and output layers. They consist of neurons that learn to represent and process the data. The more hidden layers and neurons per layer, the more complex patterns the model can learn. 3. Output layer: The last layer in the MLP model produces the final results or predictions. The number of nodes in this layer depends on the problem one is trying to solve. For example, if images are classified into ten categories, the output layer will have ten nodes. 4. Neurons: Each neuron in the MLP model receives input from other neurons, processes it using an activation function, and sends the output to other neurons in the next layer. The activation function introduces non-linearity, which enables the MLP to learn complex patterns in the data. 5. Weights and biases: Each connection between neurons has a weight that determines the strength of the association. The weights are adjusted during training to minimize the difference between the predicted and actual values. Biases are additional constants that help shift the activation function, improving the model’s learning ability.
Energies 2023,16, 4060 18 of 29 6. Training: MLP models are trained using a backpropagation algorithm, which adjusts the weights and biases by minimizing the error between the predicted and actual values. The process is iterative, involving multiple passes through the data to fine-tune the model. 7. Loss function: This is a measure of how well the MLP model is performing. A lower value indicates better performance. During training, the goal is to minimize the loss of function. Energies.2023,16,xFORPEERREVIEW17of29 6. Oncetherandomforestmodelhasbeentrained,validated,andtested,itcanmake short‐termloadforecastsbasedonnewinputdata.Themodeltakesintherelevant featuresforthedesiredforecastingperiodandproducesapredictionfromeachde‐ cisiontree.Thefinalpredictionistheaverageoftheindividualtreepredictions, whichprovidesamoreaccurateandstableforecast. 3.2.4.MultilayerPerceptronModel Amultilayerperceptron(MLP)isanartificialneuralnetworkconsistingofmultiple layersofinterconnectednodes,alsoknownasneuronsorperceptrons.Itiswidelyused insupervisedlearningtasks,suchasclassificationandregression.AnMLPisshownin Figure12,ithasthefollowingcomponents[69]: 1. Inputlayer:ThisisthefirstlayeroftheMLPmodel,receivingtheinputdata(e.g., numbers,images,andtext).Eachnodeinthislayercorrespondstoasingleinputdata feature. 2. Hiddenlayers:Thesearethelayersbetweentheinputandoutputlayers.Theyconsist ofneuronsthatlearntorepresentandprocessthedata.Themorehiddenlayersand neuronsperlayer,themorecomplexpatternsthemodelcanlearn. 3. Outputlayer:ThelastlayerintheMLPmodelproducesthefinalresultsorpredic‐ tions.Thenumberofnodesinthislayerdependsontheproblemoneistryingto solve.Forexample,ifimagesareclassifiedintotencategories,theoutputlayerwill havetennodes. 4. Neurons:EachneuronintheMLPmodelreceivesinputfromotherneurons,pro‐ cessesitusinganactivationfunction,andsendstheoutputtootherneuronsinthe nextlayer.Theactivationfunctionintroducesnon‐linearity,whichenablestheMLP tolearncomplexpatternsinthedata. 5. Weightsandbiases:Eachconnectionbetweenneuronshasaweightthatdetermines thestrengthoftheassociation.Theweightsareadjustedduringtrainingtominimize thedifferencebetweenthepredictedandactualvalues.Biasesareadditionalcon‐ stantsthathelpshifttheactivationfunction,improvingthemodelʹslearningability. 6. Training:MLPmodelsaretrainedusingabackpropagationalgorithm,whichadjusts theweightsandbiasesbyminimizingtheerrorbetweenthepredictedandactual values.Theprocessisiterative,involvingmultiplepassesthroughthedatatofine‐ tunethemodel. 7. Lossfunction:ThisisameasureofhowwelltheMLPmodelisperforming.Alower valueindicatesbetterperformance.Duringtraining,thegoalistominimizetheloss offunction. Figure12.Multilayerperceptronmodel. Figure 12. Multilayer perceptron model. Mathematically, the output of a neuron can be represented as [66]: aj=f∑(wij ∗xi+bj)(30) where: ajis the output (activation) of neuron j; fis the activation function; wij is the weight connecting input ito neuron j; xiis the input value for input i; bjis the bias term for neuron j. For each layer in the MLP, this equation can be applied in a matrix form: A=f(WX +B)(31) where: Ais the activation matrix (each column represents the activation of a neuron); fis the activation function applied element-wise; Wis the weight matrix; Xis the input matrix (each column represents an input feature vector); Bis the bias matrix. After computing the activations for all layers, the output layer produces the final prediction. For classification tasks, a softmax function is typically used in the output layer to convert the activations into probabilities: softmax(ai) = exp(ai)/Σ(exp(aj)) (32) where: aiis the activation of output neuron i, ajis the activation of output neuron j, softmax(ai) is the probability for class i.
Energies 2023,16, 4060 19 of 29 3.2.5. Deep Learning Models Deep learning (DL) is a class of ML models that can capture complex nonlinear relationships between the load and other factors that influence the load [ 70 ]. DL models consist of multiple layers of interconnected nodes that process and transmit information through weighted connections [ 71 ]. One popular DL model for STLF is the convolutional neural network (CNN), which can capture the spatial and temporal patterns in the load data. CNNs have been used successfully in power system applications to forecast the load based on weather conditions, occupancy, and time of day [ 72 ]. Another critical DL model for STLF is the recurrent neural network (RNN), which can capture the temporal dependencies in the load data. RNNs use feedback connections to allow information to be passed from one time step to the next, enabling the model to capture the dynamics of the load over time. One type of RNN is the long short-term memory (LSTM) network, which is well-suited for STLF as it can capture both short-term and long-term dependencies in the load data [73]. 3.2.6. Ensemble Models Ensemble models are machine learning models that combine multiple models’ predictions to produce a final prediction [ 74 ]. The basic idea behind ensemble models is to use the strengths of different models and combine their predictions to create a more accurate and robust forecast. Ensemble models can be used in various machine learning tasks, including classification, regression, and clustering [ 75 ]. One of the main advantages of ensemble models is their ability to reduce variance and overfitting. Variance is a measure of how much the predictions of the models in the ensemble vary from each other. Overfitting occurs when a model is too complex and fits the training data too closely, leading to poor performance on new, unseen data. Ensemble models can reduce variance and overfitting by combining the predictions of multiple models, thereby reducing the overall variance and producing a more robust forecast [ 76 ]. One of the main disadvantages of ensemble models is their complexity. Ensemble models can be more complex to implement and interpret than single models and may require more computational resources [77]. Additionally, the performance of ensemble models can depend on the specific combination of models used, and finding the optimal combination can be challenging. Another disadvantage of ensemble models is their sensitivity to the quality of the models in the ensemble [ 78 ]. Poor-quality models can negatively affect ensemble models, such as overfitting or underfitting the data. It is essential to carefully select the models in the ensemble and ensure that they are high quality. Several techniques for creating ensemble models include bagging, boosting, and stacking shown in Figure 13. Each technique has its strengths and limitations, and the choice of method depends on the specific application and dataset. A. Bragging. Bagging, or bootstrap aggregating, is a technique that involves training multiple models on different subsets of the training data and then combining their predictions using a weighted average. Bagging can be used with any model and can reduce variance and overfitting. The basic idea behind bagging is to create multiple copies of the original dataset, each with a different subset of the data [ 79 ]. The models are then trained on each of these copies, combining their predictions to produce a final prediction. By combining the predictions of multiple models, bagging can produce a more accurate and robust forecast. B. Boosting. Boosting is a technique that involves training multiple models sequentially, with each model focusing on the examples that the previous model misclassified. Boosting can improve the accuracy of the forecast but may be more prone to overfitting than bagging. The basic idea behind boosting is to start with a simple model and then sequentially add more complex models that focus on the examples that the previous model misclassified [ 80 ]. The models are combined using a weighted sum, with weights that depend on their accuracy.
Energies 2023,16, 4060 20 of 29 By focusing on the examples that the previous model misclassified, boosting can produce a more accurate and robust forecast. C. Stacking. Stacking, or stacked generalization, is a technique that involves training multiple models on the training data and then using their predictions as input to a higher-level model. The higher-level model learns to combine the predictions of the lower-level models to produce a final prediction. Stacking can improve the accuracy and robustness of the forecast but may be more complex to implement and interpret than bagging or boosting [ 81 ]. The basic idea behind stacking is to train multiple models on the training data and then use their predictions as input to a higher-level model. The higher-level model learns to combine the predictions of the lower-level models to produce a final prediction. By combining the predictions of multiple models, stacking can create a more accurate and robust forecast. Energies.2023,16,xFORPEERREVIEW19of29 overfittingbycombiningthepredictionsofmultiplemodels,therebyreducingtheoverall varianceandproducingamorerobustforecast[76].Oneofthemaindisadvantagesof ensemblemodelsistheircomplexity.Ensemblemodelscanbemorecomplextoimple‐ mentandinterpretthansinglemodelsandmayrequiremorecomputationalresources [77]. Additionally,theperformanceofensemblemodelscandependonthespecificcom‐ binationofmodelsused,andfindingtheoptimalcombinationcanbechallenging.An‐ otherdisadvantageofensemblemodelsistheirsensitivitytothequalityofthemodelsin theensemble[78].Poor‐qualitymodelscannegativelyaffectensemblemodels,suchas overfittingorunderfittingthedata.Itisessentialtocarefullyselectthemodelsintheen‐ sembleandensurethattheyarehighquality.Severaltechniquesforcreatingensemble modelsincludebagging,boosting,andstackingshowninFigure13.Eachtechniquehas itsstrengthsandlimitations,andthechoiceofmethoddependsonthespecificapplication anddataset. Figure13.Typesoftheensemblemethod. A.Bagging Bagging,orbootstrapaggregating,isatechniquethatinvolvestrainingmultiplemodels ondifferentsubsetsofthetrainingdataandthencombiningtheirpredictionsusingaweighted average.Baggingcanbeusedwithanymodelandcanreducevarianceandoverfitting.The basicideabehindbaggingistocreatemultiplecopiesoftheoriginaldataset,eachwithadif‐ ferentsubsetofthedata[79].Themodelsarethentrainedoneachofthesecopies,combining theirpredictionstoproduceafinalprediction.Bycombiningthepredictionsofmultiplemod‐ els,baggingcanproduceamoreaccurateandrobustforecast. B.Boosting Boostingisatechniquethatinvolvestrainingmultiplemodelssequentially,witheach modelfocusingontheexamplesthatthepreviousmodelmisclassified.Boostingcanimprove theaccuracyoftheforecastbutmaybemorepronetooverfittingthanbagging.Thebasicidea behindboostingistostartwithasimplemodelandthensequentiallyaddmorecomplexmod‐ elsthatfocusontheexamplesthatthepreviousmodelmisclassified[80].Themodelsarecom‐ binedusingaweightedsum,withweightsthatdependontheiraccuracy.Byfocusingonthe examplesthatthepreviousmodelmisclassified,boostingcanproduceamoreaccurateand robustforecast. Figure 13. Types of the ensemble method. 3.3. Hybrid Models Hybrid models combine the advantages of both statistical and machine learning models. The most used hybrid models for STLF are ARIMA–SVR and ES–ANN. ARIMA– SVR integrates the ARIMA and SVR models and can capture the temporal patterns and nonlinear relationships of load demand [ 82 ]. ES–ANN combines ES and ANN models and can capture the seasonal and nonlinear patterns of load demand. Hybrid models can improve the accuracy and interpretability of STLF models for power system applications. However, they require more computational resources than statistical models and may be more challenging to implement than machine learning models [ 83 ]. Hybrid models, which combine different modeling techniques, have been proposed to improve the accuracy of load forecasts [84]. The following are some of the essential hybrid models: 3.3.1. ARIMA–ANN Hybrid Model The autoregressive integrated moving average (ARIMA) model is a classical timeseries forecasting method that captures linear dependencies in the data. Artificial neural networks (ANNs) are capable of learning complex nonlinear patterns. By combining the linear forecasting ability of ARIMA with the nonlinear forecasting ability of ANNs, this hybrid model can capture both linear and nonlinear dependencies in the load data, resulting in improved STLF accuracy [85]. 3.3.2. Wavelet-Transform-Based Hybrid Models Wavelet transform is a technique that decomposes a time series into different frequency components, which can be analyzed separately. The high-frequency components represent
Energies 2023,16, 4060 21 of 29 noise and sudden changes, while low-frequency components capture the underlying trends. Wavelet transform can be combined with various forecasting techniques, such as ANN, support vector machines (SVM), or long short-term memory (LSTM) networks, to create a hybrid model. It can handle the frequency components separately, leading to improved forecast accuracy [86]. 3.3.3. EEMD–ANN Hybrid Model The ensemble empirical mode decomposition (EEMD) is an advanced signal processing technique that decomposes a non-stationary time series into a set of intrinsic mode functions (IMFs). Combining EEMD with ANN allows this hybrid model to handle nonstationary and nonlinear load data more effectively. The EEMD preprocesses the load data by extracting the IMFs, and the ANN is trained on these IMFs to generate forecasts. The forecasts are then combined to produce the final STLF [87]. 3.3.4. Fuzzy-Logic-Based Hybrid Models Fuzzy logic is a mathematical approach that deals with uncertainty and imprecision in data. It can be combined with other forecasting techniques such as ANN, SVM, or regression models to create a hybrid model that handles the uncertainty in load data more effectively. Fuzzy logic can preprocess the input data, model the uncertainties in the forecasting model, or fuse the forecasts from different models [88]. 3.3.5. Deep-Learning-Based Hybrid Models Deep learning techniques, such as convolutional neural networks (CNN) and LSTMs, have shown great potential in STLF due to their ability to learn hierarchical and temporal features in the data. These deep learning models can be combined with other forecasting techniques, such as statistical models, wavelet transform, or fuzzy logic, to create hybrid models that leverage the strengths of both approaches for improved STLF accuracy [89]. 3.4. Performance Comparison of STLF Models The performance of STLF models depends on various factors, such as the size and quality of the data, the forecasting horizon, and the complexity of the underlying relationships between the variables. Intelligent models outperform statistical models, and hybrid models outperform both statistical and intelligent models [ 90 ]. However, the models’ relative performance can vary depending on the application and dataset. For example, statistical models may perform well when the data are stationary and linear and when limited data are available. Intelligent models may perform well when the data are nonlinear and non-stationary and when a large amount of data is available. Hybrid models may perform well when there are both linear and nonlinear relationships between the variables and when the data are noisy or missing. Table 2summarizes the findings. Table 2. Performance comparison of STLF Models. Refs. Model Contributions Applications Limitations [91] ARIMA Suitable for modeling time series data with trend and seasonality Forecasting of various economic and financial data, weather forecasting, and sales forecasting Assumes stationarity and requires careful selection of model parameters [92] SARIMA Incorporates seasonal factors into ARIMA model Forecasting of time series data with seasonal patterns, such as sales data during holiday seasons Requires careful selection of model parameters [93]Exponential Smoothing Simple yet effective method for time series forecasting Used in various industries, such as finance, supply chain management, and marketing Assumes no trend or seasonality in data
Energies 2023,16, 4060 22 of 29 Table 2. Cont. Refs. Model Contributions Applications Limitations [94]Generalized Linear Model Extends linear regression to accommodate non-normal response variables Used in various fields, such as medical research, environmental science, and social sciences Assumes linearity between predictors and response variable [95]Support Vector Machine Nonlinear classification and regression method Used in various fields, such as finance, biology, and image recognition Requires careful selection of model parameters [96] Decision Tree Non-parametric method for classification and regression Used in various fields, such as finance, marketing, and healthcare Prone to overfitting and requires careful selection of hyperparameters [97] Random Forest Ensemble method that uses multiple decision trees for classification and regression Used in various fields, such as finance, marketing, and healthcare Prone to overfitting and requires careful selection of hyperparameters [98]Gradient Boosting Ensemble method that uses multiple weak learners to improve predictions Used in various fields, such as finance, healthcare, and image recognition Prone to overfitting and requires careful selection of hyperparameters [99] Deep Learning Neural-network-based models for complex data analysis Used in various fields, such as image recognition, natural language processing, and speech recognition Requires large amounts of training data and computational resources [100]Ensemble Methods Combine multiple models to improve prediction accuracy Used in various fields, such as finance, marketing, and healthcare Requires careful selection of models and hyperparameters [101]Multilayer Perceptron Neural-network-based models for classification and regression Used in various fields, such as finance, healthcare, and image recognition Prone to overfitting and requires careful selection of hyperparameters [102] Hybrid Models Combine multiple models or methods to improve prediction accuracy Used in various fields, such as finance, marketing, and healthcare Requires careful selection of models and hyperparameters 4. The Road Ahead Short-term load forecasting (STLF) is critical in the energy industry. It enables power system operators to make informed decisions about resource allocation, power generation, and grid stability. Over the years, numerous STLF models have been developed and evaluated, but there is still room for improvement. Some potential future research directions in the development of STLF models are shown in Figure 14. 1. Incorporating new data sources: Traditional STLF models rely on historical load, weather, and calendar data as input features. However, the availability of new data sources, such as social media data and smart meter data, presents an opportunity to develop more accurate and robust STLF models [ 103 ]. Future studies can examine the application of machine learning algorithms to identify the most relevant data sources for predicting electricity demand and how best to incorporate these data sources into STLF models. 2. Development of hybrid models: Hybrid models combine different models or techniques to address specific challenges or achieve specific goals. Hybrid models can combine traditional STLF models with models for predicting renewable energy production or demand-side management (DSM) models [ 104 ]. Future research can examine the development of more advanced hybrid models that can handle multiple input features, uncertainty quantification, and other challenges in STLF modeling. 3. Integration of advanced machine learning techniques: Advanced machine learning techniques, such as deep learning and reinforcement learning, have shown great potential in improving the accuracy of STLF models. Deep learning techniques can automatically learn complex patterns and relationships in the data, while reinforcement learning can learn to optimize actions based on feedback from the environment [ 105 ]. Future work can investigate the creation of more sophisticated machine learning
Energies 2023,16, 4060 23 of 29 models that can manage vast volumes of data, combine numerous input features, and adjust to shifting energy system conditions. 4. Handling of non-stationary and nonlinear load data: Traditional STLF models assume that the load data is stationary and linear. However, the load data can be nonstationary and nonlinear due to changes in consumer behavior, the introduction of new technologies, and other factors [ 106 ]. Future research can explore the development of STLF models that can handle non-stationary and nonlinear load data, either through advanced machine learning techniques or more flexible statistical models. 5. Integration of probabilistic forecasting: Probabilistic forecasting measures uncertainty around the point forecast. It can help power system operators make more informed decisions in the face of uncertainty [ 107 ]. Future work can support the development of STLF models that can provide point forecasts and uncertainty estimates, such as prediction intervals or probabilistic forecasts. These uncertainty estimates can help to identify potential risks and improve the overall reliability of the power system. 6. Integration of online learning: Online learning is a type of machine learning that can adapt to changing conditions in the energy system in real-time [ 108 ]. Online learning algorithms can learn from new data as it becomes available and adjust the forecast accordingly. Future studies may look toward creating STLF models that employ online learning algorithms to increase forecast precision and timeliness [109]. 7. Development of interpretable models: Interpretable models are models that can provide insights into the factors that are driving the forecast. Interpretable models can help power system operators understand the underlying patterns and relationships in the data and make more informed decisions about resource allocation and power generation [ 110 ]. Additional studies may examine the creation of STLF models that are easier to understand, either via the use of advanced machine learning techniques or more straightforward statistical models [111]. 8. Integration of ensemble methods: Ensemble methods, such as bagging and boosting, have shown great potential in improving the accuracy and robustness of STLF models. Ensemble methods can combine and overfit them and be used to select the best model for a given dataset [ 112 ]. Future work could investigate the creation of more sophisticated ensemble models capable of handling various input features, uncertainty quantification, and other difficulties in STLF modeling [113]. 9. Handling of data quality issues: Data quality issues, such as missing data, outliers, and measurement errors, can have a significant impact on the accuracy of STLF models [ 114 ]. Future studies could assist in creating STLF models that can deal with problems with data quality by prediction models or more sophisticated statistical models that can directly deal with missing data. 10. Integration of domain knowledge: Domain knowledge, such as knowledge about consumer behavior, the energy system, and the environment, can provide valuable insights into the factors driving electricity demand [ 115 ]. Future studies can create STLF models that incorporate domain knowledge into the modeling process, either through expert systems or sophisticated machine learning methods that can add domain knowledge as different input characteristics. 11. Development of adaptive models: The energy system constantly changes, and the factors driving the electricity demand can vary over time [ 116 ]. Future studies may examine the creation of STLF models that can modify their predictions in response to changing energy system conditions, either using online learning algorithms or more adaptable statistical models. 12. Handling multiple time scales: The electricity demand can exhibit patterns on various time scales, such as daily, weekly, and seasonal patterns [ 117 ]. Future studies can develop STLF models that can handle multiple time scales by combining models trained on various time scales or utilizing more sophisticated machine-learning methods. 13. Integration of uncertainty information: Uncertainty information, such as information about input data reliability or model accuracy, can provide valuable insights into the
Energies 2023,16, 4060 24 of 29 quality of the forecast [ 118 ]. Future studies may explore creating STLF models that can incorporate uncertainty data into the modeling process through probabilistic models or more sophisticated statistical models that can calculate forecast uncertainty [119]. 14. Development of models for distributed energy resources: The increasing use of distributed energy resources, such as rooftop solar panels and energy storage systems, has introduced new challenges for STLF models [ 120 ]. Future studies may develop STLF models that can account for distributed energy resources, either through models that forecast the production of renewable energy or models that forecast the effects of distributed energy resources on electricity consumption [121,122]. Energies.2023,16,xFORPEERREVIEW24of29 Figure14.TheroadaheadforSTLFmodels. 5.Conclusions STLFmodelsareanessentialcomponentoftheenergyindustry,astheyenable powersystemoperatorstomakeinformeddecisionsaboutresourceallocation,power generation,andgridstability.Thisreviewarticlehaspresentedanoverviewofthestate‐ of‐the‐artSTLFmodelsforpowersystemapplications,includingstatistical,intelligent, andhybridmodels.Thesemodelshavetheirstrengthsandlimitations.Thechoiceof modeldependsonvariousfactors,suchasthesizeandqualityofthedata,theforecasting horizon,andthecomplexityoftheunderlyingrelationshipsbetweenthevariables.Power systemoperatorsshouldcarefullyevaluatetheperformanceofdifferentmodelsandcon‐ siderthesefactorswhenselectingthemostappropriatemodelfortheirapplication.The developmentofSTLFmodelsisanongoingresearcharea,andfutureadvancesinmachine learning,dataanalytics,andcomputationalresourcesareexpectedtoimprovetheaccu‐ racyandrobustnessofSTLFmodels. AuthorContributions:Conceptualization,S.A.andS.S.;methodology,S.S.andM.J.;formalanaly‐ sis,A.Z.andH.S.U.;validation,Z.L.andR.G.;visualization,H.K.;investigation,allauthors;writ‐ ing—originaldraftpreparation,S.A.andS.S.;writing—reviewandediting,A.Z.,H.S.U.,M.J.,H.K., R.G.andZ.L.;supervision,H.K.andM.J.;projectadministration,H.K.;funding,R.G.Allauthors havereadandagreedtothepublishedversionofthemanuscript. Funding:ThisworkreceivedfundingfromanSGSGrantfromVSB‐TechnicalUniversityofOstrava undergrantnumberSP2023/005. DataAvailabilityStatement:Notapplicable. ConflictsofInterest:Theauthorsdeclarenoconflictofinterest. Figure 14. The road ahead for STLF models. 5. Conclusions STLF models are an essential component of the energy industry, as they enable power system operators to make informed decisions about resource allocation, power generation, and grid stability. This review article has presented an overview of the state-of-the-art STLF models for power system applications, including statistical, intelligent, and hybrid models. These models have their strengths and limitations. The choice of model depends on various factors, such as the size and quality of the data, the forecasting horizon, and the complexity of the underlying relationships between the variables. Power system operators should carefully evaluate the performance of different models and consider these factors when selecting the most appropriate model for their application. The development of STLF models is an ongoing research area, and future advances in machine learning, data analytics, and computational resources are expected to improve the accuracy and robustness of STLF models.
Energies 2023,16, 4060 25 of 29 Author Contributions: Conceptualization, S.A. and S.S.; methodology, S.S. and M.J.; formal analysis, A.Z. and H.S.U.; validation, Z.L. and R.G.; visualization, H.K.; investigation, all authors; writing— original draft preparation, S.A. and S.S.; writing—review and editing, A.Z., H.S.U., M.J., H.K., R.G. and Z.L.; supervision, H.K. and M.J.; project administration, H.K.; funding, R.G. All authors have read and agreed to the published version of the manuscript. Funding: This work received funding from an SGS Grant from VSB-Technical University of Ostrava under grant number SP2023/005. Data Availability Statement: Not applicable. Conflicts of Interest: The authors declare no conflict of interest. References 1. Cheng, L.; Yu, T. A new generation of AI: A review and perspective on machine learning technologies applied to smart energy and electric power systems. Int. J. Energy Res. 2019,43, 1928–1973. [CrossRef] 2. Ding, Y.; Zhu, Y.; Feng, J.; Zhang, P.; Cheng, Z. Interpretable spatio-temporal attention LSTM model for flood forecasting. Neurocomputing 2020,403, 348–359. [CrossRef] 3. Zakaria, A.; Ismail, F.B.; Lipu, M.H.; Hannan, M. Uncertainty models for stochastic optimization in renewable energy applications. Renew. Energy 2020,145, 1543–1571. [CrossRef] 4. Fu, H.; Baltazar, J.-C.; Claridge, D.E. Review of developments in whole-building statistical energy consumption models for commercial buildings. Renew. Sustain. Energy Rev. 2021,147, 111248. [CrossRef] 5. Lu, S.; Li, Q.; Bai, L.; Wang, R. Performance predictions of ground source heat pump system based on random forest and back propagation neural network models. Energy Convers. Manag. 2019,197, 111864. [CrossRef] 6. Ganaie, M.; Hu, M.; Malik, A.; Tanveer, M.; Suganthan, P. Ensemble deep learning: A review. Eng. Appl. Artif. Intell. 2022 , 115, 105151. [CrossRef] 7. Altan, A.; Karasu, S.; Zio, E. A new hybrid model for wind speed forecasting combining long short-term memory neural network, decomposition methods and grey wolf optimizer. Appl. Soft Comput. 2021,100, 106996. [CrossRef] 8. Alexander, M.; Beushausen, H. Durability, service life prediction, and modelling for reinforced concrete structures—Review and critique. Cem. Concr. Res. 2019,122, 17–29. [CrossRef] 9. Kurani, A.; Doshi, P.; Vakharia, A.; Shah, M. A Comprehensive Comparative Study of Artificial Neural Network (ANN) and Support Vector Machines (SVM) on Stock Forecasting. Ann. Data Sci. 2023,10, 183–208. [CrossRef] 10. Dagoumas, A.S.; Koltsaklis, N.E. Review of models for integrating renewable energy in the generation expansion planning. Appl. Energy 2019,242, 1573–1587. [CrossRef] 11. Zhang, R.; Chen, Z.; Chen, S.; Zheng, J.; Büyüköztürk, O.; Sun, H. Deep long short-term memory networks for nonlinear structural seismic response prediction. Comput. Struct. 2019,220, 55–68. [CrossRef] 12. Lindberg, K.B.; Seljom, P.; Madsen, H.; Fischer, D.; Korpås, M. Long-term electricity load forecasting: Current and future trends. Util. Policy 2019,58, 102–119. [CrossRef] 13. Koponen, P.; Ikäheimo, J.; Koskela, J.; Brester, C.; Niska, H. Assessing and Comparing Short Term Load Forecasting Performance. Energies 2020,13, 2054. [CrossRef] 14. Trierweiler Ribeiro, G.; Cocco Mariani, V.; dos Santos Coelho, L. Enhanced ensemble structures using wavelet neural networks applied to short-term load forecasting. Eng. Appl. Artif. Intell. 2019,82, 272–281. [CrossRef] 15. Moradzadeh, A.; Moayyed, H.; Zakeri, S.; Mohammadi-Ivatloo, B.; Aguiar, A.P. Deep Learning-Assisted Short-Term Load Forecasting for Sustainable Management of Energy in Microgrid. Inventions 2021,6, 15. [CrossRef] 16. Almalaq, A.; Edwards, G. A Review of Deep Learning Methods Applied on Load Forecasting. In Proceedings of the 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), Cancun, Mexico, 18–21 December 2017; pp. 511–516. [CrossRef] 17. Li, L.-L.; Zhao, X.; Tseng, M.-L.; Tan, R.R. Short-term wind power forecasting based on support vector machine with improved dragonfly algorithm. J. Clean. Prod. 2020,242, 118447. [CrossRef] 18. Carvalho, T.P.; Soares, F.A.A.M.N.; Vita, R.; Francisco, R.D.P.; Basto, J.P.; Alcalá, S.G.S. A systematic literature review of machine learning methods applied to predictive maintenance. Comput. Ind. Eng. 2019,137, 106024. [CrossRef] 19. Chen, J.; Ran, X. Deep Learning with Edge Computing: A Review. Proc. IEEE 2019,107, 1655–1674. [CrossRef] 20. Nespoli, A.; Ogliari, E.; Leva, S.; Pavan, A.M.; Mellit, A.; Lughi, V.; Dolara, A. Day-Ahead Photovoltaic Forecasting: A Comparison of the Most Effective Techniques. Energies 2019,12, 1621. [CrossRef] 21. Mahzarnia, M.; Moghaddam, M.P.; Baboli, P.T.; Siano, P. A Review of the Measures to Enhance Power Systems Resilience. IEEE Syst. J. 2020,14, 4059–4070. [CrossRef] 22. Ahmad, T.; Zhang, D.; Huang, C.; Zhang, H.; Dai, N.; Song, Y.; Chen, H. Artificial intelligence in sustainable energy industry: Status Quo, challenges and opportunities. J. Clean. Prod. 2021,289, 125834. [CrossRef] 23. Ruano, A.; Hernandez, A.; Ureña, J.; Ruano, M.; Garcia, J. NILM Techniques for Intelligent Home Energy Management and Ambient Assisted Living: A Review. Energies 2019,12, 2203. [CrossRef]