Application of neural network predictive control methods to solve the shipping container sway control problem in quay cranes

Received March 19, 2021, accepted April 24, 2021, date of publication May 26, 2021, date of current version June 3, 2021. Digital Object Identifier 10.1109/ACCESS.2021.3083928 Application of Neural Network Predictive Control Methods to Solve the Shipping Container Sway Control Problem in Quay Cranes SERGEJ JAKOVLEV 1,2, TOMAS EGLYNAS1, AND MIROSLAV VOZNAK 1,2, (Senior Member, IEEE) 1Marine Research Institute, Klaipeda University, 92294 Klaipeda, Lithuania 2Department of Telecommunications, Faculty of Electrical Engineering and Computer Science, VSB–Technical University of Ostrava, 708 00 Ostrava, Czech Republic Corresponding author: Sergej Jakovlev (s.jakovlev[email protected]) This work was supported by the European Regional Development Fund through the Research Council of Lithuania (LMTLT) under Project 01.2.2-LMT-K-718-03-0001. ABSTRACT Smart control systems are mostly applied in industry to control the movements of heavy machinery while optimizing overall operational efficiency. Major shipping companies use large quay cranes to load and unload containers from ships and still rely on the experience of on-site operators to perform transportation control procedures using joysticks and visual contact methods. This paper presents the research results of an EU-funded project for the Klaipeda container terminal to develop a novel container transportation security and cargo safety assurance method and system. It was concluded that many risks arise during the container handling procedures performed by the quay cranes and operators. To minimize these risks, the authors proposed controlling the sway of the spreader using a model predictive control method which applies a multi-layer perceptron (MLP) neural network (NN). The paper analyzes current neural network architectures and case studies and provides the engineering community with a unique case study which applies real operation statistical data. Several key training algorithms were tested, and the initial results suggest that the Levenberg–Marquardt (LM) algorithm and variable learning rate backpropagation perform better than methods which use the multi-layer perceptron neural network structure. INDEX TERMS Neural nets, data mining, control systems. I. INTRODUCTION The logistics sector is constantly looking for a better way to decrease the costs associated with port handling operations, including maintenance costs, long operational delays, and human error during technical procedures. Most containers are handled by heavy machinery and operated under even less optimal standardized procedures and control systems, including those applied at the Klaipeda container terminal. Factors such as different container weights, wind gusts, operator experience and control routines often cause containers to sway chaotically during their transportation from ships. It is increasingly challenging to stabilize containers carried by quay crane spreaders because of their large weights and sizes [1]. Engineering and industry communities have dedicated much attention to the examination and improvement of The associate editor coordinating the review of this manuscript and approving it for publication was Xiaosong Hu . mechanical crane stabilization systems, construction of new components and development of new smart control systems for movement and speed prediction sub-routines. However, less attention has been paid to the full autonomy of crane control operations [2]. New control sub-routines must take into account the full specter of the nature of the container swinging problem [3]. Experienced on-site operators who perform container-handling procedures have years of experience and are ready to respond to continuous changes. However, due to a lack of knowledge of some crucial elements of control, these operators tend to learn false information. Visual feedback is used to aid in positioning the container below the quay crane. Therefore, it is necessary to control sudden changes in acceleration of the container during its transportation along with the movement of the crane, which results in a change in the high amplitude acceleration. The following section addresses the application of predictive control and neural network (NN) methods and systems. VOLUME 9, 2021 This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/ 78253 S. Jakovlev et al.: Application of NN Predictive Control Methods II. LITERATURE REVIEW A. RECENT ADVANCEMENTS IN CRANE SPREADER AND CARGO STABILIZATION Recent advances in artificial intelligence, AI machine learning [4], [5], and deep learning [6] have presented new opportunities for use in the industry to solve complex control and scheduling problems. Many engineering problems from the past can now be solved using advanced knowledge extraction methods and big data (BD) analytics tools. Many academic authors rely solely on the theoretical achievements in the relevant field, while others present strong practical applications in the area of transportation, dealing with delay optimization and various pattern predictions [7]–[11]. We analyzed recent methods for compensating the sway of containers during movement [12] (from ship to shore and shore to ship via quay cranes) and minimizing the overall transportation process [13]. We examined some general methodologies, including the use of key on-site safety and security regulations. The theoretical part of the presented model was based on research ideas from the following authors. Zhang et al. [14] developed a modelindependent control method, called proportional-derivative sliding mode control, for 3D overhead crane systems to achieve simultaneous trolley positioning and payload swing suppression for quay cranes. Golovin and Palis [15] presented a robust control-based approach for active damping of elastic structural vibrations in gantry cranes. Yongming et al. [16] analyzed the influence of vertical deformation in cranes on anti-sway control; they created a novel three-mass threedegree-of-freedom elastic dynamic model of the trolley system to solve the sway problem. Ileš et al. [17] presented an asymptotically stabilizing sequential distributed model predictive control (MPC) of a 3D tower crane. Abdullahi et al. [18] proposed a new online adaptive outputbased command shaping (AOCS) technique for effective payload sway reduction in an overhead crane. Smoczek and Szpytko [19] developed a new evolutionary-based algorithm for a fuzzy logic-based data-driven predictive model of time between failures (TBF) for an adaptive crane control system. Finally, Maghsoudi et al. [20] presented an improved unity magnitude zero vibration (UMZV) shaper for payload sway reduction of an underactuated 3D overhead crane with hoisting effects. B. RECENT ADVANCEMENTS IN CRANE PREDICTIVE CONTROL Authors Nelson and Johnson [21] developed two model predictive control approaches for optimizing microgrid dispatch. Yang et al. [22] proposed a model predictive control system with adaptive machine-learning-based building models for building automation and control applications. Zeng et al. [23] aimed to increase the control performance of a selective catalytic reduction (SCR) denitrification system through modeling and disturbance rejection. Beckenbach et al. [24] analyzed model-based predictive controllers used to manage control tasks with constraints on the state. Bünning et al. [25] developed a model predictive control method for room temperature control in buildings. Li et al. [26] explored a hierarchical model predictive controlbased energy management strategy for fuel cell hybrid construction vehicles. Liu et al. [27] presented a novel finite control-set model predictive control (FCS-MPC) strategy for solving the well-known challenges in predictive control regulated NNPC. De León Puig et al. [28] proposed a simple adaptive-predictive control scheme for a DC-DC buck converter. Hu et al. [29] developed generalized predictive control (GPC) to suppress the effect of time-varying delay and parameter identification error during robot-assisted cardiovascular surgery. Wang et al. [30] attempted to optimize control performance of the mean value model of a fuelpowered aircraft engine; the authors designed an adaptive fuzzy radial basis function (RBF) neural network to perform predictive air-fuel ratio control to optimize performance and reduce exhaust emissions in fuel-powered unmanned aerial vehicles (UAVs). Maraoui and Bouzrara [31] proposed a distributed model predictive control based on a game theory framework for nonlinear systems with nonlinearly coupled dynamics. Oyama and Durand [32] developed economic model predictive control (EMPC) which integrates process control and economic optimization and can potentially allow time-varying operating policies to maximize economic performance. Yin et al. [33] presented a data-driven multi-objective predictive control approach to increase power production and reduce fatigue loads in wind farms using evolutionary optimization. Shi et al. [34] analyzed the autonomous vehicle trajectory planning methods and constructed an adaptive model predictive control (AMPC) trajectory tracking system which considers disturbances in the path curvature. Gao et al. [35] proposed a data-driven predictive control strategy for a nonlinear system, tested on a continuous stirred tank heater (CSTH) benchmark. Mazar and Rezaiezadeh [36] employed a model predictive controller (MPC) to minimize boiler activation time. Pozzi et al. [37] addressed battery pack management and developed non-linear model predictive control (NMPC). Ramirez et al. [38] also developed a fast model-based predictive control (MPC), designed to control active and reactive power exchanged by a grid-connected MMC, providing a fast dynamic response, low current THD, and constant switching frequency. Vallianos et al., [39] presented an experimental and numerical study of predictive control of a hybrid ventilation system in an institutional building, with an emphasis on thermal comfort. Wang and Wang [40] discussed the possibilities to adopt MPC in the automotive industry. Yin et al. [41] presented a reliability aware multi-objective predictive control strategy for wind farms based on machine learning and heuristic optimizations. C. JUSTIFICATION OF THE SELECTED PREDICTION METHOD A deeper analysis of the scientific publications revealed positive results from the adoption of certain artificial neural network (ANN) training algorithms to solve 78254 VOLUME 9, 2021 S. Jakovlev et al.: Application of NN Predictive Control Methods prediction problems via time-series forecasting, especially, a modified Levenberg–Marquardt (LM) algorithm by Garoosiha et al. [42]. Two authors developed this algorithm independently: Levenberg [43] and Marquardt [44]. Other research teams found LM useful and efficient enough to solve complex tasks in time-constrained situations [45]. Billah et al. [46] proposed an improved Levenberg–Marquardt (LM) training algorithm for artificial neural networks to predict the possible day-end closing stock price. Keong et al. [47] developed an LM back-propagation artificial neural network model to predict floods. Authors Multazam et al. [48] discussed new trends and sources of renewable energy and proposed a wind speed prediction backpropagation neural network (BPNN) with the Levenberg–Marquardt algorithm for weight updates. Qiao et al. [49] developed an adaptive LM-algorithm-based echo state network (ALM-ESN) for chaotic time-series predictions. Zhang and Behera [50] proposed a predictive model based on recurrent neural networks trained with the LM backpropagation learning algorithm to forecast solar radiation. Mammadli [51] proposed the use of artificial neural networks using the LM optimization algorithm for the prediction of financial time series. Finally, Shi et al. [52] proposed an improved recursive Levenberg–Marquardt algorithm (RLM) to more efficiently train multi-layer neural networks and achieve reliable efficiency in engineering tasks. LM algorithms in combination with advanced neural network architectures can bring positive results to predict the sway of the spreader during movement and minimize the time delays during live control operations using MPC, by converging more often and accelerating training to be quick enough in real scenarios, under real control operations. D. WORKING HYPOTHESIS Based on a review of the current trends in AI and sway control, we propose a technological solution capable of solving the problem of container sway caused by lack of experience in on-site crane operators, by developing an MPC-based method to predict spreader speed during unloading operations. In this paper, we present a generic work hypothesis consisting of two statements: – Despite their adaptability in solving only generic curvefitting problems and finding only a local minimum, LM algorithms can be used in machinery control operations where this minimum is sufficient to perform transportation operations more efficiently. – By estimating speed factors and predicting cargo sway scenarios for the predictive control model, LM is more robust and can more efficiently converge [53]. III. DESCRIPTION OF THE STATISTICAL DATA ACQUISITION SYSTEM AND THE DATA ACQUISITION PROCESS A. DESCRIPTION OF THE EQUIPMENT USED TO COLLECT THE TRAINING DATA To acquire the necessary statistical data to train the algorithm, we developed a sensory system and applied it to real trans-shipment operations using quay cranes, spreader systems, AGVs, and trucks [54]. The system’s design and experimental background were modeled according to the methods presented by Harrison et al. [55]. In the experimental section, a DL1 PRO data logger/analyzer acquires and transfers the data. It uses a threeaxis accelerometer (6 g) to detect the movement vector. Accelerometer working parameters: 3-axis work modes with a guaranteed 6 gminimum full scale on all axes, and maximum resolution of 0.005 g. We examined the dynamic characteristics of the containers with different masses during movement and placement on AGVs/trucks. The acceleration data proved to be very interesting and informative for the analytical section. The research team also examined the spreader and its position above the AGVs /trucks during container handling operations. The movement detection speed was calculated up to 100 times per second due to several technology limitation factors. The data logging accuracy of the experimental hardware system was set to 1 % due to possible irregularities in the electronics. Figure 1 shows the data logger’s hardware, and Figure 2 indicates its placement on the spreader during the experiment. This data logging technology (DL1 PRO data logger) was selected for several key parameters. First, it recorded accurate data references with exact time stamps and three-dimensional positions in space during movement. Second, it possesses sufficient technological compatibilities with other information and communication technologies (ICT). The logger itself has IP50 environmental protection. The experiment was performed at the Klaipeda port container terminal – LKAB ‘‘Smelte’’. During the data acquisition experiment, the necessary equipment was mounted on the spreader of the quay crane (Fig. 2). The quay crane performed container transportation procedures from the ship on top of the AGV (Ship-to-Shore operations). During the spreader’s movements, the acceleration data from the accelerometer were collected by the system and stored on an SD card. The container and other statistical information were collected, including the mass of the container and the operators who controlled the movements of the spreader. The entire data acquisition process took nine hours to complete due to battery usage limitations. This data was used for the present research (in total, 200 separate operations were recorded). B. DESCRIPTION OF THE ACQUIRED DATA Figure 3 plots measured data from the experimental case study of a shipping container unloading procedure performed by a quay crane at the Klaipeda container port. Each measurement taken during the case studies had its own statistical and operational deviations and technological irregularities, mainly due to the strict rules presented in the operations manual for crane operators. Each ‘‘best control choice’’ scenario was performed without operational problems. Each container varied in mass, although deviations from the 20,000 kg lifting VOLUME 9, 2021 78255 S. Jakovlev et al.: Application of NN Predictive Control Methods FIGURE 1. DL-1 PRO Datalogger with GPS antenna (the special protective case was designed to withstand a harsh physical environment). FIGURE 2. Placement of the data acquisition sensory hardware on the quay crane spreader. operational standard were minimal; it did not affect the quality of the measurements. Figures 4 depicts key stages during transportation/unloading: – Container raising with hooking – Vertical raising of the container – Bias raising of the container – Horizontal transportation of container – Bias lowering of the container – Vertical lowering of the container – Container placement on the transport means (truck or AGV) Figure 3 presents the sway speed of the spreader and the container. This information is very important because higher values correlate with the actual speed of the cargo during transportation and ship unloading at a designated port. The data indicate that the overall transportation process is FIGURE 3. Comparison of four cases with detected spreader speed across the X-axis during container transportation from the ship to the AGV. FIGURE 4. Quay crane with spreader movement along three axes. prolonged due to compensation for sway, thereby creating time delays. Figure 4 depicts quay crane cargo transportation along all axes, the X-axis used for prediction priority due to high sway speeds. The literature highlights the efficiency of several algorithms used to train neural networks also used in various predictive control scenarios, i.e. variable learning rate backpropagation, scaled conjugate gradient, recursive prediction error methods, and the Levenberg–Marquardt methods. We discuss these algorithms in the following sections and apply them to predict the speed of the spreader along the X-axis using the MLP network structure. To set the input values, we collected data from ten statistically similar procedures using containers of similar mass and the same crane operator. In the modeling phase, we assume that all containers are transported from a single point in space (from the ship), without deviations in their actual placement from each another. In practice, planning is performed with a delay in mind, thus minimizing the efficiency of the terminal [56]. Most operations are also synchronized with the on-site operator’s actions for truck and AGV secure movements and CO2regulations [57]. Such delays occur daily, and we aim to demonstrate the problem to the academic environment, to the engineers working in this field, and to show the capabilities of new AI prediction methods to solve the possible sway control problems on-site. 78256 VOLUME 9, 2021 S. Jakovlev et al.: Application of NN Predictive Control Methods Jakovlev et al. [58] presented the initial results, which suggested the following: – Quay crane operators did not maintain uniform horizontal movement speed of the container. The ladder shape presented in Figure 3 indicates the actual decrease in speed. – Operators initiated sudden control movements of the joystick to stop the container transportation process for short periods. – Operators initiated corrections to movements and thereby prolonged the container transportation process. – Lack of experience among operators and low efficiency in the technological synchronization and planning methods for quay cranes, trucks and AGVs led to additional unnecessary oscillations of the cargo, up to the final seventh stage of the transportation [59]. – Because the maximum speed of the spreader is regulated by ISO operational standards, each container was transported with an average of 8.1 seconds delay for the entire period of measurement, the average container transportation speed being 40.4 seconds. According to the operational manual, the working efficiency of these procedures achieved a mere 80 %. – Each quay crane is limited only by the human factor, therefore prompting new advancements in the relevant area of engineering [60]. It is worth mentioning that the experimental statistical data which describes the underlying network and the sway processthroughoutitsoperatingrangewasassignedasuitable sampling frequency in advance, while also eliminating errors. IV. METHODOLOGY FOR THE PREDICTION MODEL A. INTRODUCTION OF THE MODEL PREDICTIVE CONTROL (MPC) STRATEGY For the spreader sway control strategy, we propose the application of an MPC which uses an internal system model to make single-step predictions of the system behavior and compute the container sway over a predefined prediction period, taking into account the operational constraints of the crane. MPC can adequately measure the dynamics of energy consumption devices (motors) and the characteristics of the mechanical components used to move the spreader. New measurements of the system and new predictions are added continuously to the system. Figure 5 schematically illustrates the main components of centralized model predictive control (CMPC), detailing the optimization criteria for the model in Figure 6. MPC is a multivariable control algorithm which uses: •An internal dynamic model of the spreader control process •A history of past spreader movements (speed values) •Optimization criteria and the function J, which collaborate over the receding prediction horizon and apply the best AI functions and methods. General MPCs are based on iterative, finite-horizon optimization models. In the current state, at time tn, the spreader FIGURE 5. Description of the MPC control strategy for the quay crane (system) to detect the speed of the spreader along the X-axis during container transportation from the ship to AGV and to predict the optimal speed for this procedure with time steps tk+1[61]. state (speed) is sampled and a cost-minimizing control strategy is computed (via a numerical minimization algorithm with the optimization block shown in Fig. 5 and Equations 1–3) for a relatively short time horizon in the future for the sampling period T. Specifically, an AI enriched prediction strategy calculation is used to explore spreader speed deviations which emanate from the control current state and find a cost-minimizing control strategy until time for computational period Tc. After the spreader speed control strategy is applied, the spreader speed is sampled again and the calculations are repeated, yielding a new control and new predicted speed. The prediction horizon is continuously shifted forward, and for this reason, MPC is also called Receding Horizon Control [62]. Although this approach is not optimal, in practice it has given very good results [63]. The defined single controller determines the system’s inputs. First, at each control step, the CMPC controller measures the current state of the quay crane control system (measures the speed of the spreader) and determines which value control inputs (speed control indicators) to provide to the system using pre-defined numerical optimization models. The model consists of prediction algorithms based on a neural network structure, defined in the following section. Prediction values (desired speed of the spreader) are linked to the objective function to minimize the overall transportation time of the cargo by the spreader. These optimum values determine the actions which provide the best-predicted performance according to a given objective function minJ: min J (vT,T)=XT=n T=1 ST vT ,(1) where Jis the total minimized transportation time, Tis the desired period of the control procedure to take effect (for each prediction step tk+1), nis the number of periods in a transportationprocessfromtheship to the AGV,Sisthetransportation distance, and vis the transportation speed (obtained by the system from the proposed data acquisition unit). Therefore, the effectiveness of the MPC can be described VOLUME 9, 2021 78257 S. Jakovlev et al.: Application of NN Predictive Control Methods FIGURE 6. Schematic of the MPC Controller optimization algorithm with the optimization objective for the control action, i.e., to increase/decrease the cargo transportation speed. as (2) and (3): min MPC (T,Tc)=(min J,min C),(2) min C (Tc)=XT=n T=1CTc,(3) where Tcis the computation period for each new prediction. This strategy includes the control variables determined for the first prediction step applied to the system, after which the cycle repeats. The goal of this control strategy is to determine the optimal control values so that the spreader stabilization time delay and energy consumption costs are minimized. Hence, the predictive control problem is stated as determining the input values so that the control objectives are achieved optimally while satisfying the overall mechanical and electrical quay crane control system constraints. The proposed MPC solution does not have real-time properties. Delays occurring due to the computational times for each period Tdecrease the real-time properties, i.e., an increase in the number of periods increases the overall computational time for the entire operation; however, it can decrease the transportation time for each container. The heart of the controller is a model M( ), parameterized by a data set (consisting of time-series data samples for spreader speed momentum values zk– input data samples for the neural network for each input node. These are vectors FIGURE 7. Schematic of the ANN MLP structure used in this study. of the appropriate dimensions used to predict the movement speed of the spreader. Here, kis the number of desired input nodes. Optimization is subject to the constraints of the controller variables (CVs). The effectiveness of the MPC system in these control tasks is from the advantages of the implicit closed-loop control law. Only the first of the set of control values, i.e., , is transmitted to the prediction model, after which the complete optimization and prediction procedure is repeated using the current speed value output, thereby improving the results at each iteration, after each period T. B. MODEL STRUCTURE SELECTION The aim of this study is to analyze and discuss the performance of an artificial neural network, i.e., a multi-layer perceptron (MLP), in the prediction and estimation of container sway during a quay crane unloading process. The modeling problem was defined as a time series forecasting function approximation problem. MLP can model complex functional relationships and approximate any complex nonlinear function. To test the functionality and efficiency of the LM algorithm, we compared it to several other conventional algorithms: – Recursive prediction error method – Scaled conjugate gradient – Variable learning rate backpropagation The network inputs were the spreader speeds from the presented case studies. At each new phase, the network is trained using the previous ‘‘best-case’’ experience from the data set , adding the new node zn, upgrading the data set θ and eliminating the ‘‘worst-case’’ scenarios, while keeping the number of parameters and the size of the data set θconstant. We designed a fully connected two-layer feedforward MLP-network containing four inputs for each speed value, along with a varying number of hidden units, and a single output unit, for the prediction of X-axis spreader sway speed (Fig. 7). 78258 VOLUME 9, 2021 S. Jakovlev et al.: Application of NN Predictive Control Methods FIGURE 8. Implementation of the Levenberg–Marquardt algorithm in NNSYSID. The network output is the predicted speed of the spreader according to the crane operator’s control actions. ANN training data ZNconsists of inputs to the network u(t)and FIGURE 9. Investigated indices for system orders from 1 to 6. corresponding desired outputs y(t): ZN={[u(t),y(t)]|i=1. . . N}.(4) The defined MLP network consists of a hidden layer and hyperbolic tangent sigmoid activation functions (f,F): ˆyi(w,W)=FiXq jWij fjX2 l=1wjlzl+wj0+Wi0. (5) In this case, the MLP hidden layer f(.)for each hidden neuron zj,wjis presented as: fjzj,wj=2 1+e−2P2 j=1zj·wji+wj0 −1.(6) The output of the neural network with nnodes in a hidden layer is defined as: yiyij,Wij=FXi=1yij ·Wij +W0j.(7) The weights (specified by the vector θand by the matrices wand W) are the adjustable parameters of the MLP network determined through the neural network training procedure. The objective of the training is then to determine a mapping from the set of training data to the set of possible weights: ZN→ˆ θ. (8) MLP produces the predictions ˆyi(t). MLP uses the prediction error approach, which measures closeness in terms of a mean square error criterion: VNθ, ZN=1 2·NXN i=1yi− ˆyi|θT yi− ˆyi|θ.(9) The weights for the predictions are: ˆ θ=arg min θVNθ, ZN.(10) By an iterative minimization scheme: θi+1=θi+µi·fi,(11) VOLUME 9, 2021 78259 S. Jakovlev et al.: Application of NN Predictive Control Methods FIGURE 10. Investigated indices for system orders from 1 to 6 for past inputs and past outputs. FIGURE 11. One-step ahead prediction for an NN with 10 hidden neurons and a 3rd order model using the LM algorithm. where θispecifies the current iterate, and fiis the search direction with the step size µi. The selected MLP neural network structure and mathematical definition is the most-often considered choice for prediction problems [64] which use only one hidden layer. In particular, we selected the LM version of Baptista and Morgado-dias [53] for training since it proved effective in solving similar tasks. The difference between the Marquardt [44] and the current iteration of LM is in the following adjustments by Fletcher [65]. The size of the elements FIGURE 12. Prediction errors for an NN with 10 hidden neurons and a 3rd order model using the LM algorithm. of the diagonal matrix added to the Gauss–Newton Hessian is according to the size of the ratio between actual decrease and predicted decrease: ri=VNθi,ZN−VNθi+fji,ZN VNθi,ZN−Liθi+fji,(12) Lθi+fji=VNθi,ZN+fTB(θi)+ +1 2fTB(θi)fji, (13) where Bdenotes the gradient of the criterion concerning the weights, and Ris the so-called Gauss-Newton approximation to the Hessian. The following NNSYSID algorithm was applied to the LM (Fig. 8): A. Select an initial parameter vector θ0and an initial value δ0; B. Determine the search direction from [R(θi+δi·I)]· fji = −G(θi), where Iis a unit matrix; C. If the predicted decrease is close to the actual decrease, let the search direction approach the Gauss–Newton search direction while increasing step size, ri>0.75 →δi=δi 2; D. If a predicted decrease is far from the actual decrease, let the search direction approach the gradient direction while decreasing step size, ri<0.25 →δi=2·δi; E. If VNθi+fji,ZN<VNθi,ZN, then θi+1=θi+fji as a new iterate and let δi+1=δi,i=i+1. F. If the stopping criterion is not satisfied, go to 2). The weights are adjustable, and they are updated through network training. The objective of the training is then to 78260 VOLUME 9, 2021 S. Jakovlev et al.: Application of NN Predictive Control Methods FIGURE 13. One-step ahead prediction for an NN with 10 hidden neurons and a 3rd order model using the recursive prediction error method. determine a mapping from the set of training data to the set of possible weights. Many methods are available to select the number of nodes in a hidden layer [66], however no general rule exists which meets every case of neural network design. Therefore, we designed several NN structures: 5, 10, 15, and 20 hidden neurons in each case. We defined the normal distribution for random initial weights for each trial and partitioned the training and test data. Training was performed on the first 70 % of the sequence and the test on the final 30 %. Each algorithm was programmed to train until the squared error threshold was less than 0.005. V. COMPUTATIONAL RESULTS The neural network was trained using MATLAB software. First, the training set was scaled to a zero mean and a variance of one, and then the test set was scaled with the same constants. Next, we addressed the problem of finding the order of the system (Fig. 9). It is difficult to conclude anything certain from this figure. The added measurement noise in all four data samples corrupted the measurements. Figure 10 charts the order index correlation with past inputs and outputs. We can assume that the system can be modeled by a 3rd order model since the slope of the curve decreases for model orders ≥3. Figures 11 to 18 indicate the efficiency of the training algorithms for one-step-ahead predictions. A comparison of the plots for the training and test sets for all four algorithms was satisfactory. The computational FIGURE 14. Prediction errors for an NN with 10 hidden neurons and a 3rd order model using the recursive prediction error method. FIGURE 15. One-step ahead prediction for an NN with 10 hidden neurons and 3rd order model using scaled conjugate gradient. results suggest that the networks do not overfit the data in each case. Therefore, we conclude that the selected MLP VOLUME 9, 2021 78261