A PD-type state-dependent Riccati equation with iterative learning augmentation for mechanical systems

Preprint version of the article: S. R. Nekoo, J. A. Acosta, G. Heredia, and A. Ollero, “A PD-Type State-Dependent Riccati Equation with Iterative Learning Augmentation for Mechanical Systems,” IEEE/CAA Journal of Automatica Sinica, 2022. doi: 10.1109/JAS.2022.105533 1  Abstract—This work proposes a novel proportional-derivative (PD)-type state-dependent Riccati equation (SDRE) approach with iterative learning control (ILC) augmentation. On the one hand, the PD-type control gains could adopt many useful available criteria and tools of conventional PD controllers. On the other hand, the SDRE adds nonlinear and optimality characteristics to the controller, i.e. increasing the stability margins. These advantages with the ILC correction part deliver a precise control law with the capability of error reduction by learning. The SDRE provides a symmetric-positive-definite distributed nonlinear suboptimal gain 𝐊(𝐱) for the control input law 𝐮= −𝐑−𝟏(𝐱)𝐁𝑻(𝐱)𝐊(𝐱)𝐱. The sub-blocks of the overall gain 𝐑−𝟏(𝐱)𝐁𝑻(𝐱)𝐊(𝐱), are not necessarily symmetric positive definite. A new design is proposed to transform the optimal gain into two symmetric-positive-definite gains like PD-type controllers as 𝐮= −𝐊𝐒𝐏(𝐱)𝐞−𝐊𝐒𝐃(𝐱)𝐞󰇗. The new form allows us to analytically prove the stability of the proposed learning-based controller for mechanical systems; and presents guaranteed uniform boundedness in finite-time between learning loops. The symmetric PD-type controller is also developed for the state-dependent differential Riccati equation (SDDRE) to manipulate the final time. The SDDRE expresses a differential equation with a final boundary condition, which imposes a constraint on time that could be used for finite-time control. So, the availability of PD-type finite-time control is an asset for enhancing the conventional classical linear controllers with this tool. The learning rules benefit from the gradient descent method for both regulation and tracking cases. One of the advantages of this approach is a guaranteedstability even from the first loop of learning. A mechanical manipulator, as an illustrative example, was simulated for both regulation and tracking problems. Successful experimental validation was done to show the capability of the system in practice by the implementation of the proposed method on a variable-pitch rotor benchmark. Index Terms—SDRE; SDDRE; Symmetric; PD-type; Closedloop; Iterative Learning Control. I. INTRODUCTION HE proportional derivative (PD) control has wellestablished mathematics, so many well-known tools, assessment and tuning methods, etc. It has been accepted by the industrial platforms which prefer the simplicity of the control This Research is funded by the European Commission H2020 Programme under HYFLIERS project contract 779411, AERIAL-CORE project contract number 871479 and the ARTIC (RTI2018-102224-B-I00) project, funded by the Spanish Agencia Estatal de Investigación. law and straightforward analysis. What if someone wants a finite-time PD controller? We intend to gain all the benefits of the PD control with the additional power of nonlinearity and finite-time regulation. The state-dependent Riccati equation (SDRE) has been defined as an optimal control design for nonlinear systems. Optimality, robustness, design flexibility, and systematic procedure are some advantages of the SDRE [1]. The differential form of the SDRE uses a final boundary condition to impose an extra penalty on final states near the end of time, the method is so-called the state-dependent differential Riccati equation (SDDRE) [2]. Both SDRE and SDDRE were widely used in aerospace [3], approximate dynamic programming framework [4], cancer treatment modeling [5], etc. Either of SDRE or SDDRE method provides a suboptimaldistributed symmetric-positive-definite gain 𝐊(𝐱) for the standard form of Riccati control law 𝐮=−𝐗(𝐱)𝐱 in which 𝐗(𝐱)=𝐑−1(𝐱)𝐁𝑇(𝐱)𝐊(𝐱). This work proposes a new control structure to transform the distributed gain into two symmetricpositive-definite gains for the error vector and its derivative 𝐮=−𝐊SP(𝐱)𝐞−𝐊SD(𝐱)𝐞󰇗. The proposed structure possesses many advantages such as independent control of error of the system, PD shape, the structure of a finite-time PD controller, etc.; however, the main objective for the transformation is to release a new design to analytically guarantee the stability of a novel SDRE design, augmented by iterative learning control (ILC) approach. Using PD-like control seems more practical and widely used, and easier for implementation and mathematical derivation. The motivation for using the SDRE controller is the nonlinear optimal structure and finite-time characteristics of the controller. The optimality deviates since the symmetric structure changes the original gain 𝐊(𝐱); nonetheless, the nonlinearity and finite-time control remained as the advantages of the proposed method. Iterative learning control uses previous data to update the next control loop [6]. A proper ILC converges the error of a system towards zero in each control loop [7-9]. One of the main advantages of the ILC is a compensation of the modeling uncertainty by the learning process [10-12]. This is more critical when dynamic modeling is complex, for example, Authors are with the GRVC Robotics Lab., Depto de Ingeniería de Sistemas y Automática, Escuela Técnica Superior de Ingeniería, Universidad de Sevilla, Seville, Spain, emails: [email protected], (jaar, guiller, aollero)@us.es A PD-Type State-Dependent Riccati Equation with Iterative Learning Augmentation for Mechanical Systems Saeed Rafee Nekoo, José Ángel Acosta, Guillermo Heredia (Member, IEEE) and Anibal Ollero (Fellow, IEEE) T Preprint version of the article: S. R. Nekoo, J. A. Acosta, G. Heredia, and A. Ollero, “A PD-Type State-Dependent Riccati Equation with Iterative Learning Augmentation for Mechanical Systems,” IEEE/CAA Journal of Automatica Sinica, 2022. doi: 10.1109/JAS.2022.105533 2 the behavior of the flapping wing flying robots [13]; or piezoelectric actuators with hysteretic nonlinearity [14]. Shen and Xu presented an adaptive ILC for systems with randomly varying iteration lengths [15]. Robotics has been an attractive field for the implementation of ILC. Roveda et al. presented the iterative learning control approach with reinforcement peculiar to high-accuracy force tracking in robotics [16]. Oh et al. researched a regulation problem for iterative learning model predictive control [17]. Pan and Yu developed a composite learning robot control possessing guaranteed parameter convergence [18]. The iterative learning control improves trajectory tracking by several trials; Schollig and D’Andrea employed the learning for tracking systems with state and input constraints [19]. The dynamic of a brush-bot is a challenging topic, especially in terms of actuation; hence, one efficient way is to use learning methods. Barrier-certified adaptive control was implemented for the navigation of a brush-bot as a reinforcement learning method [20]. Vision-based learning was used for racing drones in a tracking problem with moving uncertain trajectories [21]. The high level of uncertainty was handled by a deep neural network learning tool. In reinforcement learning, a gradient descent method is a powerful approach for training the models, also capable of modification for computation enhancement [22]. The closed-loop ILCs, with PD- or PID-based structures were reported [23, 24]; however, the ILC was usually employed to skip the unknown closed-loop dynamic in feedforward methods [25, 26]. It was also common to consider ILC based on a core hypothesis that repetitiveness of task and model was satisfied; however, robustness with ILC was shown using a high-order internal model [27-29]. In other words, the learning process helps to deliver an ideal controller without engaging the design with offline dynamic modeling; and using the online data of each step for the performance enhancement of the next loop. The open-loop approach demonstrated the nature of the learning by showing unexpected behavior at primary loops might result in bad trajectories, collisions, saturations, or destruction of the prototypes. So, open-loop learning is preferable for stable systems; i.e. a quadrotor with an internal stable loop [26]. The closed-loop ILC could possess stable primary loops at the beginning. This is useful for dynamically unstable systems. Here in this work, the presented learningbased controller is closed-loop, applicable for a dynamically unstable system. The experimental implementation of this work uses a propeller-type inverted pendulum that, in nature, is unstable at an equilibrium point. So, the PD-type SDDRE generates a primary input signal (stable) for the first loop and the ILC will update and reduce the error in consecutive learning loops. The stability of the proposed ILC is proved by a new Lyapunov-like candidate with a dynamic-scaling factor. From the control application point of view, the best qualities of the new controller are its PD-like structure and finite-time option. We propose a baseline PD controller that is redesigned to gain robustness via dynamic scaling. The error with this controller will converge to zero at infinity, and in finite-time, although bounded, the error might be too large for the application. Thus, the ILC reduces the error in each iteration based on the update in the previous loop. The proposed design guarantees bounded error, 𝜀(𝑘), at 𝑘-th iteration lim 𝑡→𝑡f|𝑒(𝑡)|≤𝜀(𝑘) as it is shown in Fig. 1. For an infinite-time case, the stability of the nonlinear PD-like controller for robot manipulators was investigated and it was shown that this form leads to global asymptotic stability [30]. The integrator part of the control also generated semiglobal stability for saturated linear PID controllers [31]. The current work uses a gradient descent method as a training rule for updating the learning feedforward control law to propose a symmetric gain SDRE/SDDRE control and combining it with the ILC to form a new finite-time iterative learning approach. The stability proof of the control law is presented. The main advantage of this learning system is to employ a nonlinear (suboptimal) input law besides the ILC to guarantee stable regulation and tracking even from the first loop of the learning. The proposed ILC is also implemented on a variable-pitch (VP) rotor benchmark for experimental validation. The VP platform was controlled by the SDRE [32], though here for the first time, the implementation of a learningbased controller has been presented. Fig. 1. The explanation of the ILC operation in error reduction in finite-time. The main contributions of this work are as follows: C1. Presenting a nonlinear finite-time PD-like controller based on SDDRE with the ILC feedforward compensation to gain more precision. C2. Transforming the suboptimal-distributed symmetricpositive-definite gain of the SDRE/SDDRE methodology into novel PD-like gains, which is more suitable for mechanical systems. C3. Proving the stability of the nonlinear controller with ILC. C4. Introducing a novel convex objective function for the regulation training rule of the gradient descent method. C5. Guaranteeing uniform boundedness in finite-time between learning loops allows us to use the approach for unstable mechanical systems. Notations: Let 𝜆max(∙) and 𝜆min(∙) be maximum and minimum eigenvalues of an arbitrary matrix, respectively. ℝ𝑛 is denoted as the 𝑛-dimensional Euclidean space; ℝ𝑛×𝑚 is presented as the set of 𝑛×𝑚 real matrices. (∙)𝑇 shows transpose and diag(∙) means a diagonal matrix. 𝑡f 𝑒(𝑡) 𝑡 ∞ ∞-time convergence lim 𝑡→∞|𝑒(𝑡)|=0 𝜀(𝑘) 1 𝑁 𝑘 ILC iterations Guaranteed Stability finite-time convergence Preprint version of the article: S. R. Nekoo, J. A. Acosta, G. Heredia, and A. Ollero, “A PD-Type State-Dependent Riccati Equation with Iterative Learning Augmentation for Mechanical Systems,” IEEE/CAA Journal of Automatica Sinica, 2022. doi: 10.1109/JAS.2022.105533 3 Section II briefly presents the SDRE/SDDRE controller structure. Iterative control system design has been presented for both regulation and tracking problems in Section III. Learning rules based on the gradient descent method have been expressed in Section IV. Simulations and experiments are presented in Sections V and VI, respectively, and the conclusions in Section VII. II. THE SDRE/SDDRE CONTROL DESIGN Consider a nonlinear system 𝐱󰇗(𝑡)=𝐀(𝐱(𝑡))𝐱(𝑡)+𝐁(𝐱(𝑡))𝐮(𝑡), (1) where 𝐱(𝑡)∈ℝ𝑛 is a state vector, 𝐮(𝑡)∈ℝ𝑚 is an input vector. 𝐀(𝐱(𝑡)):ℝ𝑛→ℝ𝑛×𝑛 and 𝐁(𝐱(𝑡)):ℝ𝑛→ℝ𝑛×𝑚 are statedependent coefficient (SDC) parameterization of a nonlinear system 𝐱󰇗(𝑡)=𝐟(𝐱(𝑡))+𝐠(𝐱(𝑡),𝐮(𝑡)) where 𝐟(𝐱(𝑡)):ℝ𝑛→ ℝ𝑛 and 𝐠(𝐱(𝑡),𝐮(𝑡)):ℝ𝑛×ℝ𝑚→ℝ𝑛 are piecewisecontinuous vector-valued functions that satisfy a local Lipschitz condition. The aim is to minimize the cost function [33]: 𝐽(⋅)=12(𝐱𝑇(𝑡f)𝐅𝐱(𝑡f)+∫{𝐱𝑇𝐐(𝐱)𝐱+𝐮𝑇𝐑(𝐱)𝐮}d𝑡 𝑡f 0), (2) and finish a control task in a finite predefined time 𝑡∈[0,𝑡f]. 𝐐(𝐱(𝑡)):ℝ𝑛→ℝ𝑛×𝑛 and 𝐅∈ℝ𝑛×𝑛 penalize the states in 𝑡∈ [0,𝑡f) and at 𝑡f, respectively (both symmetric positive semidefinite); 𝐑(𝐱(𝑡)):ℝ𝑛→ℝ𝑚×𝑚 penalizes the inputs (symmetric positive definite). Assumption 1. (Controllability). The nonlinear pair of [𝐀(𝐱(𝑡)),𝐁(𝐱(𝑡))] must form a completely controllable parameterization of the original system (1) with the mentioned conditions [1]. Assumption 2. (Observability). The nonlinear pair of [𝐀(𝐱(𝑡)),𝐐12 ⁄(𝐱(𝑡))] must form a completely observable parameterization of the original system (1) with the mentioned conditions [1]. Satisfying Assumptions 1 and 2 was discussed and for more details on checking those conditions, please visit Refs. [34, 35]. The control law representing the SDRE/SDDRE is: 𝐮(𝑡)=−𝐑−1(𝐱(𝑡))𝐁𝑇(𝐱(𝑡))𝐊(𝐱(𝑡))𝐱(𝑡), (3) where the symmetric positive-definite suboptimal gain [𝐊(𝐱(𝑡))]2𝑛×2𝑛, is a solution to the state-dependent Riccati equation [1]: 𝐀𝑇(𝐱)𝐊ss(𝐱)+𝐊ss(𝐱)𝐀(𝐱) −𝐊ss(𝐱)𝐁(𝐱)𝐑−1(𝐱)𝐁𝑇(𝐱)𝐊ss(𝐱) +𝐐(𝐱)=𝟎, (4) or the state-dependent differential Riccati equation [1]: 𝐊󰇗(𝐱)=−{𝐀𝑇(𝐱)𝐊(𝐱)+𝐊(𝐱)𝐀(𝐱) −𝐊(𝐱)𝐁(𝐱)𝐑−1(𝐱)𝐁𝑇(𝐱)𝐊(𝐱) +𝐐(𝐱)}, 𝐊(𝑡f)=𝐅. (5) In LQR design for linear system or an open-loop two-point boundary value problem, there exists optimality; however, in the SDRE or SDDRE, which is the extension of LQR for nonlinear systems, the following terms appear in the derivation such as (𝜕𝐀(𝐱) 𝜕𝐱 𝐱)𝑇, (𝜕𝐁(𝐱) 𝜕𝐱 𝐱)𝑇, etc., for more details Refs. [36- 39] could be visited. Therefore, the SDRE is so-called, a suboptimal closed-loop control design. In the LQR, since the 𝐀,𝐁,𝐐 and 𝐑 are linear and constant, those derivative terms do not exist and the resultant gain 𝐊 is the optimal answer to the linear system. An SDRE controller is suitable for cases that need an optimal design when the finishing time is unimportant and the steadystate behavior of the system is more demanded. An SDDRE controller penalizes the states at a final time which is usually less than the conventional time of operation (when the user requires a faster response). The final time is a function of control effort that the designer invests in the platform. More energy results in a faster response; and the fastest response is limited by the actuator’s saturations and limits. The other aspect that defines the final time is the amplitude of the error in regulation (point-to-point) control and the length and speed of a trajectory in the tracking case. That means if the error in regulation is big, more time is needed and if the error is small, less time. o The question is how to determine the final time? Using the SDRE, without penalization of the final boundary condition delivers an infinite-time optimal control solution as a function of error and the convergence-time is limited to the bound of actuators and tuning of the weighting matrices. In that sense, we realize approximately how much time we need for a specific task in regulation between two set points. o After that how to finish the control task faster? Using the SDDRE with penalizing the final boundary condition. The error at the end of the regulation gets smaller and the amplitude of the input signal decreases which shows an asymptotic behavior in infinite time controllers. Using the finite time approach, the regulation near the end needs to speed up and we see a faster response, which leads to a shorter finishing time. Matrix 𝐅 plays this role and enhances regulation speed. It should be noted that in SDDRE closed-loop optimal control, at the final time, there is an error but the controller equipped with ILC will reduce it, see Fig. 1. The exact final time with zero error is possible by open-loop two-point boundary value problem [40, 41], etc.; however, using closedloop optimal control, the exact value without error is impossible. As a result, the signal of the SDDRE control law increases near the end of the time of operation. The solution to the differential Riccati equation, SDDRE, is a little more complicated than the SDRE (SDRE is an algebraic matrix equation). The more complicated structure was not a limit to the application of this method in control of a super-tanker [42], aircraft [43], wind energy conversion system [44], helicopter [45], etc. Lemma 1. The nonlinear system (1) with its performance index (2) and necessary conditions based on Assumptions 1 and Preprint version of the article: S. R. Nekoo, J. A. Acosta, G. Heredia, and A. Ollero, “A PD-Type State-Dependent Riccati Equation with Iterative Learning Augmentation for Mechanical Systems,” IEEE/CAA Journal of Automatica Sinica, 2022. doi: 10.1109/JAS.2022.105533 4 2 can be stabilized using control law (3), in which 𝐊ss(𝐱(𝑡)) is the positive-definite solution to the SDRE (4) [46]. Lemma 2. The nonlinear system (1) with its cost function (2) and necessary conditions based on Assumptions 1 and 2 can be stabilized using control law (3), in which 𝐊(𝐱(𝑡)) is the positive-definite solution to the SDDRE (5) with the final condition 𝐊(𝑡f)=𝐅 [33]. Remark 1. The control law can employ a solution to the SDRE (4) which generates a steady-state suboptimal gain or the SDDRE (5) which releases a time-varying suboptimal gain with the benefit of finite-time control. The stability proofs of Lemmas 1 and 2 used a conventional Lyapunov function 𝑉= 𝐱𝑇𝐊𝐱. The stability of the proposed PD-type SDRE and ILC law cannot be proved by the common function since it generates extra sign indefinite terms that are difficult to analyze: 𝑉󰇗=−𝐱𝑇(𝐊𝐁𝐑−1𝐁𝑇𝐊+𝐐)𝐱+𝐱𝑇𝐊𝐁𝐇+𝐇𝑇𝐁𝑇𝐊𝐱, where 𝐇 is a learning part. A different Lyapunov-like function has been considered here, stated in Theorem 1. The closed-form solution to the SDDRE (5) was reported in Sections 3.3.1 and 3.3.2 of Ref. [33]. III. SDRE/SDDRE AUGMENTED BY ITERATIVE LEARNING A. Regulation Consider 𝐪(𝑡)∈ℝ𝑁 as a generalized coordinate vector of an arbitrary 𝑁 degree-of-freedom (DoF) mechanical system with an equation of motion: 𝐌(𝐪(𝑡))𝐪󰇘(𝑡)+𝐂(𝐪(𝑡),𝐪󰇗(𝑡))𝐪󰇗(𝑡)+𝐠(𝐪(𝑡))+𝐃f𝐪󰇗(𝑡) =𝐮(𝑡), (6) where 𝐌(𝐪(𝑡)):ℝ𝑁→ℝ𝑁×𝑁 is an inertia matrix, [𝐂(𝐪(𝑡),𝐪󰇗(𝑡))𝐪󰇗(𝑡)]:ℝ𝑁→ℝ𝑁 includes Coriolis and centrifugal terms, 𝐠(𝐪(𝑡)):ℝ𝑁→ℝ𝑁 is a gravity vector, 𝐃f∈ ℝ𝑁×𝑁 is a positive-diagonal matrix (including viscous friction, drag, etc.) and 𝐮(𝑡)∈ℝ𝑁 is an input vector consisting of 𝐹𝑖(𝑡) (N) for force or 𝜏𝑖(𝑡) (Nm) for torque. Property 1. The inertia matrix 𝐌(𝐪(𝑡)) is a symmetric positive-definite matrix such that 𝐌1≤𝐌(𝐪(𝑡))≤𝐌2 where 𝐌1,𝐌2∈ℝ𝑁×𝑁 are constant symmetric positive definite matrices with the same size of 𝐌(𝐪(𝑡)). Property 2. Defining the Coriolis and centrifugal force matrix through Christoffel’s symbols, the matrix [𝐌󰇗(𝐪(𝑡))− 2𝐂(𝐪(𝑡),𝐪󰇗(𝑡))] is skew-symmetric and the following holds 𝛈𝑇(𝑡)[𝐌󰇗(𝐪(𝑡))−2𝐂(𝐪(𝑡),𝐪󰇗(𝑡))]𝛈(𝑡)=0 where 𝛈(𝑡)∈ℝ𝑁. A state vector (including generalized coordinates and the derivative of that) for the system is regarded as [𝐱(𝑡)]2𝑁×1= [𝐪𝑇(𝑡)𝐪󰇗𝑇(𝑡)]𝑇, 𝑛=2𝑁, and 𝑚=𝑁. As a result, the statespace equation for the representation of the mechanical system (6) is formed (leaving gravity out of the state-space equation to be compensated by the control law) [39]: 𝐱󰇗(𝑡)=𝐀(𝐱(𝑡))𝐱(𝑡)+𝐁(𝐱(𝑡))𝐮(𝑡) =[𝟎𝑁×𝑁 𝐈𝑁×𝑁 𝟎𝑁×𝑁 −𝐌−1(𝐱)[𝐂(𝐱)+𝐃f]]𝐱(𝑡) +[𝟎𝑁×𝑁 𝐌−1(𝐱)]𝐮(𝑡). (7) The SDRE is augmented by an iterative learning control law, for the 𝑖-th learning loop as: 𝐮𝑖(𝑡)=𝐮S 𝑖(𝑡)+𝐠𝑖(𝐪(𝑡))+𝐇𝑖(𝑡), (8) where 𝐇𝑖(𝑡) is the learning feedforward term and the feedback part is gravity vector plus SDRE/SDDRE (𝐮S(𝑡)) control law 𝐮S(𝑡)=−𝐑−1(𝐪,𝐪󰇗)𝐁𝑇(𝐪,𝐪󰇗)𝐊(𝐪,𝐪󰇗)[𝐞𝑇,𝐞󰇗𝑇]𝑇, (9) where 𝐞(𝑡)=𝐪(𝑡)−𝐪d and 𝐞󰇗(𝑡)=𝐪󰇗(𝑡)−𝐪󰇗d in which 𝐪d and 𝐪󰇗d are desired position and velocity of states at the final time. The updating term 𝐇𝑖(𝑡) is a predicted feed-forward part. It will be computed by an iterative learning law at each loop. Assumption 3. The updated term 𝐇𝑖(𝑡) of the training rule satisfies the following bound ‖𝐇𝑖(𝑡)‖2≤𝑐0+𝑐1‖𝐞(𝑡)‖2+𝑐2‖𝐞󰇗(𝑡)‖2, (10) with constants 𝑐0,𝑐1,𝑐2>0 and any positive integer 𝑖. Assumption 3 puts constraints on the updating term 𝐇(𝑡) in the input law; any kind of actuator is physically limited. The input law generates a signal to feed one or more actuators, that might be DC motors, a hydraulics valve, a propeller rotor, thermo-electric valves, etc. The physical interpretability of Assumption 3 could be expressed as an internal bound on a learning term. That means 𝐇(𝑡) has a relatively limited capability of learning concerning the PD-type counterpart of the controller, such that stability is guaranteed during the iterations. The gain 𝐊(𝐪(𝑡),𝐪󰇗(𝑡)) is a 2𝑁×2𝑁 symmetric-positive- definite matrix, a nonlinear optimal solution to the SDRE/SDDRE, partitioned into four square blocks: 𝐊(𝐪,𝐪󰇗)=[ 𝐊11(𝐪,𝐪󰇗)| 𝐊12(𝐪,𝐪󰇗) −−−−−− | −−−−−− 𝐊12 𝑇(𝐪,𝐪󰇗)| 𝐊22(𝐪,𝐪󰇗)]. (11) Considering the special form of 𝐁(𝐱(𝑡)) for the mechanical system (7), and using Eq. (11), the control law (9) is changed into: 𝐮S(𝑡)=−𝐗1(𝐪,𝐪󰇗)𝐞−𝐗2(𝐪,𝐪󰇗)𝐞󰇗, (12) where [𝐗1(𝐪,𝐪󰇗)]𝑁×𝑁=𝐑−1(𝐪,𝐪󰇗)𝐌−1(𝐪)𝐊12 𝑇(𝐪,𝐪󰇗), (13) [𝐗2(𝐪,𝐪󰇗)]𝑁×𝑁=𝐑−1(𝐪,𝐪󰇗)𝐌−1(𝐪)𝐊22(𝐪,𝐪󰇗). (14) Equation (12) is another representation of SDDRE (9) preserving nonlinearity, finite-time, and suboptimal characteristics. The matrix 𝐊12(𝐪,𝐪󰇗), in (13), is not symmetric positive definite, hence 𝐗1(𝐪,𝐪󰇗) is not positive definite as well. To prove the stability of the controller in Theorem 1, symmetric positive definiteness of the gains is necessary. We use a transformation to present two sets of symmetric positive definite based on the solution to Riccati equation to preserve Preprint version of the article: S. R. Nekoo, J. A. Acosta, G. Heredia, and A. Ollero, “A PD-Type State-Dependent Riccati Equation with Iterative Learning Augmentation for Mechanical Systems,” IEEE/CAA Journal of Automatica Sinica, 2022. doi: 10.1109/JAS.2022.105533 5 finite-time characteristics and benefits of the SDDRE and at the same time, satisfy the condition for stability proof. The final step is to transform the gains (13) and (14), to symmetric positive-definite matrices 𝐊SP(𝐪,𝐪󰇗)=𝐊12(𝐪,𝐪󰇗)𝐌−1(𝐪)𝐑−1(𝐪,𝐪󰇗)𝐌−1(𝐪)𝐊12 𝑇(𝐪,𝐪󰇗) ‖𝐊12(𝐪,𝐪󰇗)𝐌−1(𝐪)‖2, 𝐊SD(𝐪,𝐪󰇗)=𝐊22 𝑇(𝐪,𝐪󰇗)𝐌−1(𝐪)𝐑−1(𝐪,𝐪󰇗)𝐌−1(𝐪)𝐊22(𝐪,𝐪󰇗) ‖𝐊22(𝐪,𝐪󰇗)𝐌−1(𝐪)‖2, where multiplying 𝐊12(𝐪,𝐪󰇗)𝐌−1(𝐪) and 𝐊22 𝑇(𝐪,𝐪󰇗)𝐌−1(𝐪) from left-hand-side makes 𝐊SP(𝐪,𝐪󰇗) and 𝐊SD(𝐪,𝐪󰇗) symmetric-positive-definite gains, leveled by the division to norms ‖𝐊12(𝐪,𝐪󰇗)𝐌−1(𝐪)‖2 and ‖𝐊22(𝐪,𝐪󰇗)𝐌−1(𝐪)‖2 with respect. It also reshapes the control law (9) to a new symmetric design: 𝐮S(𝑡)=−𝐊SP(𝐪,𝐪󰇗)𝐞−𝐊SD(𝐪,𝐪󰇗)𝐞󰇗. (15) So, control law (15) is transformed and is not equal to (12) and the result of this transformation will be used to cancel two terms in the proof of stability in Theorems 1 and 2, using 𝐪󰇗𝑇𝐊SP(𝐪,𝐪󰇗)𝐞=𝐞𝑇𝐊SP(𝐪,𝐪󰇗)𝐪󰇗, which is not possible without symmetry of 𝐊SP(𝐪,𝐪󰇗). Symmetric design and stability analysis: The idea of the symmetric design of the SDRE/SDDRE controller is to reach a PD-like structure to independently control the error and error velocity of a practical system, and analytically prove the stability of the novel ILC controller. The design also preserves the finite-time control capability of the SDDRE. The new structure helps to guarantee the stability of the nonlinear equation of motion (6) of a mechanical system. The advantages of the symmetric design are the optimality and finite-time characteristic of the controller. Theorem 1. (Regulation) Consider the nonlinear mechanical system dynamics given by (6) satisfying Properties 1 and 2 together with the SDDRE dynamic-state feedback (8)-(15) equipped with the suboptimal gain of Eq. (5) and 𝐑=(𝐑0 𝑟(𝑡))−1, 𝐑0=𝐑0𝑇>0, set through the scaling-factor dynamics as 𝑟󰇗=𝜌1𝑟‖2𝛂−𝛾󰇗𝛾𝛃‖2 𝜆min(𝛃)+𝜌2(𝐞,𝐞󰇗,𝑟), 𝑟(0)>‖𝐑0‖, (16) with 𝜌1>1, any smooth function 𝜌2(𝐞,𝐞󰇗,𝑟) with at most linear growth in 𝑟 and 𝛂(𝐪,𝐪󰇗), 𝛃(𝐪,𝐪󰇗), and 𝛾(𝐪,𝐪󰇗) defined along with the proof. Then, the error trajectories 𝐞(𝑡) are ultimately bounded for any training rule satisfying Assumption 3 for all 0≤𝑡≤𝑡f and any finite final time 𝑡f. Proof. In the regulation case, the desired velocity and acceleration are zero at the final time, 𝐞󰇘(𝑡)=𝐪󰇘(𝑡), 𝐞󰇗(𝑡)= 𝐪󰇗(𝑡) and 𝐞(𝑡)=𝐪(𝑡)−𝐪d where 𝐪d is a constant vector. Substitution of the iterative control law (8) in the equation of motion (6) results in 𝐌(𝐪)𝐪󰇘+[𝐂(𝐪,𝐪󰇗)+𝐃f+𝐊SD(𝐪,𝐪󰇗,𝑟)]𝐪󰇗 +𝐊SP(𝐪,𝐪󰇗,𝑟)𝐞=𝐇. (17) To analyze the stability, a Lyapunov-like candidate is considered 𝑉(𝐪,𝐪󰇗,𝑟)=12{𝐪󰇗𝑇𝐌(𝐪)𝐪󰇗+𝐞𝑇𝐊SP(𝐪,𝐪󰇗,𝑟)𝐞}. (18) It should be noted that Eq. (17) is a non-homogenous second order differential equation that is updated in each loop by the time-varying term 𝐇. 𝐊SP(𝐪,𝐪󰇗,𝑟) is positive definite based on the proposed symmetric structure. Taking time-derivative of (18) provides 𝑉󰇗(𝐪,𝐪󰇗,𝑟)=𝐪󰇗𝑇𝐌(𝐪)𝐪󰇘+12𝐪󰇗𝑇𝐌󰇗(𝐪)𝐪󰇗 +𝐞𝑇𝐊SP(𝐪,𝐪󰇗,𝑟)𝐞󰇗 +12𝐞𝑇𝐊󰇗SP(𝐪,𝐪󰇗,𝑟)𝐞, (19) and replacing 𝐞󰇗=𝐪󰇗 and 𝐪󰇘 from (17) into (19) results in 𝑉󰇗(𝐪,𝐪󰇗,𝑟)=12𝐪󰇗𝑇{𝐌󰇗(𝐪)−2𝐂(𝐪,𝐪󰇗)}𝐪󰇗−𝐪󰇗𝑇[𝐃f+𝐊SD(𝐪,𝐪󰇗,𝑟)]𝐪󰇗 −𝐪󰇗𝑇𝐊SP(𝐪,𝐪󰇗,𝑟)𝐞+𝐞𝑇𝐊SP(𝐪,𝐪󰇗,𝑟)𝐪󰇗 +12𝐞𝑇𝐊󰇗SP(𝐪,𝐪󰇗,𝑟)𝐞+𝐪󰇗𝑇𝐇. It can be easily shown that 𝐪󰇗𝑇𝐊SP(𝐪,𝐪󰇗,𝑟)𝐞= 𝐞𝑇𝐊SP(𝐪,𝐪󰇗,𝑟)𝐪󰇗 because 𝐊SP(𝐪,𝐪󰇗,𝑟) is a symmetric matrix and that 𝐌󰇗(𝐪)−2𝐂(𝐪,𝐪󰇗) is a skew-symmetric matrix (Property 2). Consequently, the derivative of the Lyapunov-like candidate turns into 𝑉󰇗(𝐪,𝐪󰇗,𝑟)=−𝐞󰇗𝑇[𝐃f+𝐊SD(𝐪,𝐪󰇗,𝑟)]𝐞󰇗+12𝐞𝑇𝐊󰇗SP(𝐪,𝐪󰇗,𝑟)𝐞 +𝐞󰇗𝑇𝐇, in which 𝐊󰇗SP(𝐪,𝐪󰇗,𝑟) ={[2d d𝑡(𝐊12𝐌−1)𝐑0 𝑟𝐌−1𝐊12 𝑇 +𝐊12𝐌−1d d𝑡(𝐑0 𝑟)𝐌−1𝐊12 𝑇]‖𝐊12𝐌−1‖2 −tr {(𝐊12𝐌−1)𝑇d d𝑡(𝐊12𝐌−1)} √tr{(𝐊12𝐌−1)𝑇(𝐊12𝐌−1)}(𝐊12𝐌−1𝐑0 𝑟𝐌−1𝐊12 𝑇)} /‖𝐊12𝐌−1‖22, and could be rewritten as 𝐊󰇗SP(𝐪,𝐪󰇗,𝑟)=1 𝑟2𝛾{−𝑟󰇗𝛃+𝑟(2𝛂−𝛾󰇗𝛾𝛃)}, (20) where we have defined 𝛂≔d d𝑡(𝐊12𝐌−1)𝐑0𝐌−1𝐊12 𝑇, 𝛃≔𝐊12𝐌−1𝐑0𝐌−1𝐊12 𝑇, 𝛾≔‖𝐊12𝐌−1‖2, 𝛾󰇗=tr {(𝐊12𝐌−1)𝑇d d𝑡(𝐊12𝐌−1)} √tr{(𝐊12𝐌−1)𝑇(𝐊12𝐌−1)}. Recalling the definition of the scaling-factor dynamics given in Theorem 1, Eq. (16), the following bound for derivative Preprint version of the article: S. R. Nekoo, J. A. Acosta, G. Heredia, and A. Ollero, “A PD-Type State-Dependent Riccati Equation with Iterative Learning Augmentation for Mechanical Systems,” IEEE/CAA Journal of Automatica Sinica, 2022. doi: 10.1109/JAS.2022.105533 6 arises 𝑉󰇗(𝐪,𝐪󰇗,𝑟)≤−𝐞󰇗𝑇[𝐃f+𝐊SD(𝐪,𝐪󰇗) 𝑟]𝐞󰇗−𝑐3 𝑟‖𝐞‖22 −𝜌2 2𝑟2𝛾𝐞𝑇𝛃𝐞+𝐞󰇗𝑇𝐇, (21) where 𝑐3 is a positive constant, whose existence is guaranteed because the 𝜌1-term of (16) dominates 𝑟(2𝛂−𝛾󰇗𝛾𝛃) of (20), and where for compactness we have defined 𝐊SD(𝐪,𝐪󰇗)≔𝐊SD(𝐪,𝐪󰇗,1)>0. By construction, Eq. (16) is linear in 𝑟(𝑡), as a result, 𝑟(𝑡) is not escaping to infinity in finite time, and consequently 𝑟(𝑡)∈ ℒ∞e for any fixed 𝑡f, which is the case in SDDRE (“e” stands for extended space). Therefore, for any training rule satisfying Assumption 3, i.e. bounded as in (10) and using Young’s inequality, an upper bound for (21) becomes 𝑉󰇗(𝐪,𝐪󰇗,𝑟)≤−𝑐4‖𝐞‖22−𝑐5‖𝐞󰇗‖22+𝑐02 2, (22) where the existence of the positive constants 𝑐4,𝑐5 is guaranteed by the positivity of 𝐊SD(𝐪,𝐪󰇗) and 𝑐3>0. Additionally, the free function 𝜌2(𝐞,𝐞󰇗,𝑟) must satisfy the constraint 𝐞𝑇𝛃𝐞 2𝑟2𝛾 |𝜌2(𝐞,𝐞󰇗,𝑟)|<𝜆min(𝐃f) ‖𝐞󰇗‖22, e.g. it could be set to zero (its role is explained in Remark 2). On the other hand, recall that 𝐌(𝐪) is positive definite and 𝑟(𝑡)∈ℒ∞e for a fixed 𝑡f. Thus, for any fixed finite time 𝑡f, the function (18) is positive definite, and therefore, there always exists a constant 𝑐>0 such that (22) yields 𝑉󰇗(𝐪,𝐪󰇗,𝑟)≤−𝑐𝑉(𝐪,𝐪󰇗,𝑟)+𝑐02 2. (23) Finally, from (23) for 𝑉≥𝑐02 2𝑐 then 𝑉󰇗≤0 and the ultimate boundedness along the trajectories of (17) is guaranteed, for 0≤𝑡≤𝑡f, i.e. 𝐞,𝐞󰇗∈ℒ∞𝑒. ■ Corollary 1. Suppose that all the conditions of Theorem 1 hold. Then, if 𝑐0=0 in (10) of Assumption 3, the error trajectories 𝐞(𝑡) are uniformly ultimately bounded for all 0≤ 𝑡≤𝑡f and any finite final time 𝑡f. Remark 2. Notice that, the function 𝜌2(𝐞,𝐞󰇗,𝑟) is not needed for the stability proof, and it could be set to zero. This function modulates the size of 𝑟(𝑡) in practice. Recall that the mechanism of the scaling factor is to grow and dominate the undesired nonlinearities. Thus, in practice, it is very important to add a stabilizing term to the 𝑟(𝑡) dynamics to modulate its size in such a way that we prevent the computer hardware from an overflow in digital implementations. In the simulation section, we discuss thoroughly this and propose a form for the function 𝜌2(𝐞,𝐞󰇗,𝑟) while preserving stability. Remark 3. Selection of 𝐑0 and 𝑟(0) defines the performance of the system. Regarding the definition of 𝐑(0)=(𝐑0 𝑟(0))−1, one should note that 𝐑(0) must be defined as the proper selection of the input weighting matrix for systems. Large values for 𝐑(0) result in an improper performance. 𝑟(0) also affects the solution to 𝑟(𝑡), and finally, 𝑟(𝑡) must be positive. B. Tracking Adding and subtracting the desired dynamic to Eq. (6) change the equation of motion to (subscript “d” represents “desired”): [𝐌(𝐪)−𝐌(𝐪d)](𝐪󰇘−𝐪󰇘d) +[𝐂(𝐪,𝐪󰇗)−𝐂(𝐪d,𝐪󰇗d)](𝐪󰇗−𝐪󰇗d) +𝐠(𝐪)+𝐃f(𝐪󰇗−𝐪󰇗d) =𝐮−𝐒d(𝐪󰇘d,𝐪󰇗d,𝐪d), (24) where 𝐒d(𝐪󰇘d,𝐪󰇗d,𝐪d)=𝐌(𝐪d)𝐪󰇘d+𝐂(𝐪d,𝐪󰇗d)𝐪󰇗d+𝐃f𝐪󰇗d. (25) Defining the error and error velocity vector of the system as 𝐞(𝑡)=𝐪(𝑡)−𝐪d(𝑡) and 𝐞󰇗(𝑡)=𝐪󰇗(𝑡)−𝐪󰇗d(𝑡), respectively, and substituting ILC law (8) in (24) result in 𝐌 (𝐪,𝐪d)𝐞󰇘+[𝐂(𝐪,𝐪󰇗,𝐪d,𝐪󰇗d)+𝐃f+𝐊SD(𝐪,𝐪󰇗)]𝐞󰇗 +𝐊SP(𝐪,𝐪󰇗)𝐞=𝐇−𝐒d(𝐪󰇘d,𝐪󰇗d,𝐪d), (26) where 𝐌 (𝐪,𝐪d)=𝐌(𝐪)−𝐌(𝐪d), and 𝐂(𝐪,𝐪󰇗,𝐪d,𝐪󰇗d)= 𝐂(𝐪,𝐪󰇗)−𝐂(𝐪d,𝐪󰇗d). Property 3. The desired dynamic in trajectory tracking is a copy of the original mechanical structure, i.e. 𝐌(𝐪) and 𝐂(𝐪,𝐪󰇗). Thus, defining the Coriolis and centrifugal force matrix 𝐂(𝐪d,𝐪󰇗d) through Christoffel’s symbols, the property 𝛈𝑇[𝐌 󰇗(𝐪,𝐪d)−2𝐂(𝐪,𝐪󰇗,𝐪d,𝐪󰇗d)]𝛈=0 holds, as in Property 2. Assumption 4. The predefined trajectories for tracking, 𝐪d(𝑡), are uniformly continuous and bounded in 0≤𝑡≤𝑡f, twice continuously differentiable and with 𝐪󰇗d(𝑡) and 𝐪󰇘d(𝑡), also bounded. The final time of the trajectory 𝑡f is fixed. Theorem 2. (Tracking) Consider the nonlinear dynamics represented by (24) in the trajectory tracking case, satisfying Properties 1 and 3 together with the SDDRE dynamic-state feedback (8)-(15) with the suboptimal gain of Eq. (5) and 𝐑= (𝐑0 𝑟(𝑡))−1, set through the scaling-factor dynamics (16) with same conditions for 𝜌1, 𝜌2(𝐞,𝐞󰇗,𝑟), 𝛂(𝐪,𝐪󰇗), 𝛃(𝐪,𝐪󰇗), and 𝛾(𝐪,𝐪󰇗) as in Theorem 1. Then, the error vector 𝐞(𝑡) is ultimately bounded for any tracking training rule satisfying Assumption 3 and for any trajectory under Assumption 4, for all 0≤𝑡≤𝑡f and any finite final time 𝑡f. Proof. The stability proof is very similar to the regulation case and hence we provide only the necessary steps. In this case, the Lyapunov-like candidate is 𝑉(𝐪,𝐪󰇗,𝐪d,𝐪󰇗d,𝑟)=12{𝐞󰇗𝑇𝐌 (𝐪,𝐪d)𝐞󰇗+𝐞𝑇𝐊SP(𝐪,𝐪󰇗,𝑟)𝐞}. (27) Taking time-derivative of (27) provides 𝑉󰇗(𝐪,𝐪󰇗,𝐪d,𝐪󰇗d,𝑟)=𝐞󰇗𝑇𝐌 (𝐪,𝐪d)𝐞󰇘+12𝐞󰇗𝑇𝐌 󰇗(𝐪,𝐪d)𝐞󰇗 +𝐞𝑇𝐊SP(𝐪,𝐪󰇗,𝑟)𝐞󰇗 +12𝐞𝑇𝐊󰇗SP(𝐪,𝐪󰇗,𝑟)𝐞, (28) Preprint version of the article: S. R. Nekoo, J. A. Acosta, G. Heredia, and A. Ollero, “A PD-Type State-Dependent Riccati Equation with Iterative Learning Augmentation for Mechanical Systems,” IEEE/CAA Journal of Automatica Sinica, 2022. doi: 10.1109/JAS.2022.105533 7 and replacing 𝐞󰇘 from (26) into (28) result in 𝑉󰇗(𝐪,𝐪󰇗,𝐪d,𝐪󰇗d,𝑟)=12𝐞󰇗𝑇{𝐌 󰇗(𝐪,𝐪d)−2𝐂(𝐪,𝐪󰇗,𝐪d,𝐪󰇗d)}𝐞󰇗 −𝐞󰇗𝑇[𝐃f+𝐊SD(𝐪,𝐪󰇗,𝑟)]𝐞󰇗−𝐞󰇗𝑇𝐊SP(𝐪,𝐪󰇗,𝑟)𝐞 +𝐞𝑇𝐊SP(𝐪,𝐪󰇗,𝑟)𝐞󰇗+12𝐞𝑇𝐊󰇗SP(𝐪,𝐪󰇗,𝑟)𝐞+𝐞󰇗𝑇𝐇. Invoking Property 3, the derivative becomes 𝑉󰇗(𝐪,𝐪󰇗,𝐪d,𝐪󰇗d,𝑟)=−𝐞󰇗𝑇{𝐃f+𝐊SD(𝐪,𝐪󰇗,𝑟)}𝐞󰇗 +12𝐞𝑇𝐊󰇗SP(𝐪,𝐪󰇗,𝑟)𝐞+𝐞󰇗𝑇𝐇, (29) where definition of 𝐊󰇗SP(𝐪,𝐪󰇗,𝑟) is similar to (20). Substituting the scaling-factor 𝑟󰇗(𝑡), Eq. (16), into (29) results in 𝑉󰇗(𝐪,𝐪󰇗,𝐪d,𝐪󰇗d,𝑟)≤−𝐞󰇗𝑇{𝐃f+𝐊SD(𝐪,𝐪󰇗) 𝑟}𝐞󰇗−𝑐3 𝑟‖𝐞‖22 −𝜌2 2𝑟2𝛾𝐞𝑇𝛃𝐞+𝐞󰇗𝑇𝐇. that is analogous to (21). Therefore, under the trajectory boundedness Assumption 4, a similar result follows from (21) of Theorem 1, concluding the proof. ■ Remark 4. We would like to underscore that the training rules of the regulation and tracking cases are quite different. Focusing on Assumption 3, while in the regulation case 𝑐0=0 as considered in Corollary 1, in the tracking case and under Assumption 4, the classical training rule in the literature satisfies Eq. (25) hence 𝑐1=𝑐2=0, in Assumption 3. IV. LEARNING RULE A. Regulation training rule The gradient descent method is used to define a training rule for updating the iterative learning control 𝐇𝑖(𝑡). Regarding that the regulation case seeks the minimum error at the final time, a performance index is regarded as 𝐽LR =12∑‖𝛼𝐇𝑖(𝑡)−1𝛼𝐖𝑖(𝐪,𝐪󰇗)‖2 𝑁I 𝑖=1 , (30) where 𝑁I is the total number of iterations, 𝐽LR is convex, 0< 𝛼≪1 and 𝐖𝑖(𝐪,𝐪󰇗)=−𝐊SP 𝑖(𝐪,𝐪󰇗)[𝐪𝑖−1(𝑡f)−𝐪d]− 𝐊SD 𝑖(𝐪,𝐪󰇗)[𝐪󰇗𝑖−1(𝑡f)−𝐪󰇗d] in which 𝐪󰇗d and 𝐪d are desired velocity and position of generalized coordinates, respectively; 𝐪󰇗𝑖(𝑡f) and 𝐪𝑖(𝑡f) are actual ones at a final time of (𝑖−1)-th loop since the final error, 𝐪(𝑡f), of the previous loop is accessible; 𝐊SP 𝑖(𝐪,𝐪󰇗) and 𝐊SD 𝑖(𝐪,𝐪󰇗) are the same symmetric gains in (15). Applying the gradient descent method on (30), the training rule is found: 𝐇𝑖=𝐇𝑖−1−𝛽R𝜕𝐽LR 𝜕𝐇𝑖 =𝐇𝑖−1 −𝛽R𝛼{𝛼𝐇𝑖−1(𝑡)−1𝛼𝐖𝑖−1(𝐪,𝐪󰇗)}, (31) in which 0<𝛽R<1 is a constant-scalar training factor for regulation (point-to-point motion control). The training rule of the regulation, (31), includes the error 𝐞(𝑡f) and error velocity 𝐞󰇗(𝑡f) in 𝐖𝑖−1(𝐪,𝐪󰇗), therefore the norm of regulation training could be considered bounded as ‖𝐇(𝑡)‖2≤𝑐1‖𝐞(𝑡f)‖2+ 𝑐2‖𝐞󰇗(𝑡f)‖2≤𝑐1‖𝐞(𝑡)‖2+𝑐2‖𝐞󰇗(𝑡)‖2, stated in Assumption 3. The gradient descent method converges if 𝛻𝐽LR is Lipschitz continuous, the step size is 1𝐿 ⁄ where ‖𝛻𝐽LR(𝑣)−𝛻𝐽LR(𝑤)‖< 𝐿‖𝑣−𝑤‖ and 𝐽LR is bounded from below. The first condition for 𝛻𝐽LR =𝛼𝐇𝑖−1(𝑡)−1𝛼𝐖𝑖−1(𝐪,𝐪󰇗) is satisfied since 𝐖(𝐪,𝐪󰇗) possesses continuously differentiable gains and 𝐇(𝑡) is also a time-varying vector that follows 𝐖(𝐪,𝐪󰇗). The selection of step size is a matter of tuning and will be regarded in simulation. The last condition is also satisfied with the convex form of 𝐽LR. B. Tracking training rule The performance index of the tracking problem is expressed based on the non-homogenous part of Eq. (24) as [23]: 𝐽LT =12∑‖𝐇𝑖(𝑡)−𝐒d 𝑖(𝑡)‖2 𝑁I 𝑖=1 , (32) where 𝐒d 𝑖(𝑡) is the desired dynamics at 𝑖-th loop, Eq. (25). Implementation of the gradient descent method on (32) results in the following training rule: 𝐇𝑖=𝐇𝑖−1−𝛽T𝜕𝐽LT 𝜕𝐇𝑖−1=𝐇𝑖−1−𝛽T{𝐇𝑖−1−𝐒d 𝑖−1}, (33) in which 0<𝛽T<1 is a constant-scalar training factor for the trajectory tracking problem. The training rule of the tracking, (33), includes the desired dynamics in which a bounded trajectory updates the terms. As a result, the norm of tracking training could be considered bounded as ‖𝐇(𝑡)‖2≤𝑐0, stated in Assumption 3. Remark 5. Consider the continuous counterpart of the training rule evolution, which is defined through the stable linear (ordinary differential equation) ODE as 𝐇󰇗(𝑡)=−𝛽𝐇(𝑡)+𝛽𝚽, 0≤𝑡≤𝑡f, where 𝚽 plays the role of 𝐖𝑖. Thus, for any bounded 𝚽 and 𝛽>0, the solutions to this ODE can be bounded by a positive constant 𝛿 as follows |𝐇(𝑡f)|≤|𝐇(0)|+max 0≤𝑡≤𝑡f|𝚽|(1−exp(−𝛽𝑡f)) ≤|𝐇(0)|+max 0≤𝑡≤𝑡f|𝚽|≤𝛿, where |∙| denotes any norm. This means that, if the controller can maintain the internal stability at each iteration, then 𝚽 is bounded and so is 𝐇(𝑡), which are the results of the Theorems. Therefore, coming back to the tracking example 𝚽 reads 𝐒d 𝑖(𝑡), that is bounded by a constant. V. SIMULATIONS A. Regulation Consider a three-DoF spherical manipulator [47]. The parameters of the manipulator are presented in Table 1. The initial and final points are 𝐴=(0.7,0,0.7)m and 𝐵= (0,−0.4,0.3)m with respect. The final time is 4 seconds and 30 Preprint version of the article: S. R. Nekoo, J. A. Acosta, G. Heredia, and A. Ollero, “A PD-Type State-Dependent Riccati Equation with Iterative Learning Augmentation for Mechanical Systems,” IEEE/CAA Journal of Automatica Sinica, 2022. doi: 10.1109/JAS.2022.105533 8 iterations are considered. The following control parameters are set: 𝜌1=1.5, 𝛽=0.3, 𝛼=0.01, 𝐑0=𝐈3×3, 𝑟0=1.1×|𝐑0|, 𝐐=10×diag(𝟏1×3,𝟐1×3) and 𝐅=5×𝐐. The 𝜌2(𝐞,𝐞󰇗,𝑟) term is defined by: 𝜌2(𝐞,𝐞󰇗,𝑟)≔−𝜌22𝛾 1+𝐞𝑇𝛃𝐞𝜆min(𝐃f)‖𝐞󰇗‖22𝑟−𝑟0 𝑟, (34) where 𝑟0:=𝑟(0)=‖𝐑0‖2>1, 𝜌1>1 and 0<𝜌2<27 4𝑟02. The simulation results in successful end-effector error reduction. The configuration of the manipulator is shown in Fig. 2-a and the magnified view of the end-effector in Fig. 2-b. The end-effector error in Cartesian coordinates is also presented in Fig. 3. The end-effector error is defined as 𝐸≔ √(𝑥e−𝑥d)2+(𝑦e−𝑦d)2+(𝑧e−𝑧d)2, where {𝑥e,𝑦e,𝑧e}(m) is the Cartesian position of the end-effector of the manipulator, and {𝑥d,𝑦d,𝑧d}(m) is the desired one. Here the role of 𝜌2 is studied in terms of end-effector error. The proposed value is 𝜌2= 27 4𝑟02~8.16. The change in 𝜌2 affects error and different values were simulated to analyze the trend, presented in Table 2. To justify the increase near the end of 𝑟(𝑡), we should emphasize that the SDDRE is a differential equation with final boundary condition, 𝐊(𝑡f)=𝐅. So, near the end, we penalize the error to increase the precision, by weighting matrix 𝐅. 𝑟(𝑡) is a scaling/forcing term that guarantees stability. It manipulates the weighting matrix 𝐑=(𝐑0 𝑟(𝑡))−1. So, since the control law tries to increase the signal near the end, 𝑟(𝑡) provides the balance by increasing the amplitude of 𝐑. As it was explained in Remark 2, the nature of 𝑟(𝑡) in Eq. (34) is growing near the end of the final time and, hence, the extra 𝜌2 term decreases that raises at the end (notice the negative sign of the term). This raise at the end directly increases the input weighting matrix 𝐑= (𝐑0 𝑟(𝑡))−1 near the end of the simulation where the error is negligible. This is against the general tuning policy of SDRE/SDDRE control design which states: smaller 𝐑 results in less error. So, an increase in 𝜌2 corrects the tuning. TABLE 1. THE PARAMETERS OF THE SPHERICAL MANIPULATOR. parameter value unit definition 𝑑1,𝑑c1 0.6,0.3 m length and CoM 𝑎2,𝑎c2 0.5,0.25 m length and CoM 𝑎3,𝑎c3 0.4,0.2 m length and CoM 𝑚1,2,3,𝑚p 2,0.25 kg mass of links, load 𝐷f,1,2,3 0.025 kgm/s viscous friction 𝐼xx1,𝐼yy1,𝐼zz1 6.25×10−4,6.25 ×10−4,0.06 kgm2 moment of inertia, 1st link 𝐼xx2,𝐼yy2,𝐼zz2 0.0417,6.25 ×10−4,6.25×10−4 kgm2 moment of inertia, 2nd link 𝐼xx3,𝐼yy3,𝐼zz3 0.0267,6.25 ×10−4,6.25×10−4 kgm2 moment of inertia, 3rd link TABLE 2. END-EFFECTOR ERROR IN VARIOUS VALUES OF 𝜌2 WITH 𝜌1=1.5. End-effector error (mm) 𝜌2 End-effector error (mm) 𝜌2 0.1268 0 0.0127 7 0.1144 0.5 0.0059 8.16 0.1026 1 0.0057 10 0.0629 3 0.0048 50 Comparative results are presented in Fig. 4. The reason for such involved extra 𝜌2 term is because it has the bound constraint defined in Theorem 1 to guarantee stability. To see the latter, the derivative of the Lyapunov function is illustrated in Fig. 4 (b) where it can be seen that is negative semi-definite. (a) (b) Fig. 2. The configuration of the manipulator; (b) The magnified view of the end-effector error reduction. Fig. 3. End-effector errors in iterations. (a) (b) Fig. 4. (a) Forcing value to prove stability; (b) The derivative of Lyapunov function with different 𝜌2. Preprint version of the article: S. R. Nekoo, J. A. Acosta, G. Heredia, and A. Ollero, “A PD-Type State-Dependent Riccati Equation with Iterative Learning Augmentation for Mechanical Systems,” IEEE/CAA Journal of Automatica Sinica, 2022. doi: 10.1109/JAS.2022.105533 9 As it was pointed out in Remark 2 the simplest way to define this extra term is just 𝜌2=0 but in practice, it might be necessary to have a trade-off to modulate its size. This simulation aims to show the design of the control law based on scaling parameter 𝑟(𝑡). A series of simulations were also provided to check the capability of the proposed approach in various conditions, Fig. 5-a shows the forcing value, and Fig. 5-b shows the derivative of the Lyapunov function. (a) (b) Fig. 5. (a) Forcing value; (b) derivative of Lyapunov function; in a series of different final times. B. Comparison To clarify the machinery of the PD-type SDDRE learning controller, a comparison has been done. We compare the proposed method with a PD-based ILC to highlight the effect of finite-time gain; and also, we compare the method with SDDRE and PD without learning capability to highlight the learning effect on error reduction. The error of the proposed method with similar parameters (30 iterations) in Section V-A was gained 0.0059mm. If we use simple PD + gravity control with 𝐊p=𝐈3×3 and 𝐊d=2×𝐈3×3 gains and the same ILC support, the error will increase to 6.8(mm). Also removing the ILC from the control loop, the conventional SDDRE and PD + gravity resulted in 23.8(mm) and 93.2(mm) errors, respectively. Regarding 4(s) simulation time and 1500mm length of the three links, it is hard for the PD to reach the desired condition though the ILC improved its performance by 92%. So, comparing the results, the role of ILC on the closed-loop control is clearer. The ILC reduces the error by each iteration. The controllers, SDDRE or PD + gravity could indeed be tuned and operate properly, but the power of learning in error reduction and feedforward compensation of unpredicted situations is really helpful, especially in experiments. C. Robustness and uncertainty Uncertainty reduces the performance of the proposed controller and the ideal situation for the SDRE is to have the model as close as possible to the real platform. The source of uncertainty might be the friction (which is hard to estimate and model), the variation of load, and lack of precision in modeling. To assess the performance of the controller, this subsection is presented. We provide uncertainty to the controller by a change in mass of the load, set on the end-effector of the robot. In this scenario, we do not inform the controller about the load, so 𝑚p=0, the mass of the load (kg) is zero in the controller, but in the system we set 𝑚p=0.1kg. This increases the error from 0.0059mm to 1.67mm. Increasing the load more results in failure that shows the weakness towards the uncertainty (the black dotted line in Fig. 6). Remedy. The SDRE could be modified to show robust characteristics such as incorporating the bounds of uncertainty in the SDC matrices, the addition of correction terms such as sliding mode control [48], or using additional features such as neural networks [49]. Here we add robustness through the definition of an upper bound for load in the controller. The SDC matrices will possess 𝑚p,max and the dynamics might have another value, lower than the bound 𝑚p≤𝑚p,max. Considering 𝑚p,max=1.5 and 𝑚p=1.2, and also increasing the gains 𝐐= 100×diag(𝟏1×3,𝟐1×3) and 𝐅=20×𝐐, the end-effector error is gained 0.0054mm (the red dashed line in Fig. 6). The same uncertainty in friction could be addressed by incorporating the bounds in the SDC matrices (the green dashed-dotted line in Fig. 6). The higher error in the initial loop for the uncertainty simulation is due to more load on the end-effector. We emphasize that the focus of the paper is not on robustness and this topic deserves more discussions and a thorough investigation. Robustness of the proposed SDRE + ILC. There are two mechanisms adding robustness: learning and dynamic scaling. The learning would find larger errors at the final time with disturbances, thus forcing a higher correction in the controller for the next iteration. However, this would only be possible if the core controller maintains the internal stability during each iteration, and this is the role of the scaling factor dynamics, dominating the mismatches and eventually disturbances. Fig. 6. Comparison of the results in presence of disturbance ad uncertainty with the ideal case.