Scenario-based defense mechanism against vulnerabilities in Lagrange-based DMPC

Control Engineering Practice 114 (2021) 104879 Contents lists available at ScienceDirect Control Engineering Practice journal homepage: www.elsevier.com/locate/conengprac Scenario-based defense mechanism against vulnerabilities in Lagrange-based DMPC José M. Maestrea,∗, Pablo Velardeb, Hideaki Ishiic, Rudy R. Negenbornd aDepartment of System and Automation Engineering, University of Seville, Spain bFacultad de Ciencias de la Ingeniería e Industrias, Universidad UTE, Quito, Ecuador cDepartment of Computer Science, Tokyo Institute of Technology, Japan dDepartment of Maritime & Transport Technology, Delft University of Technology, The Netherlands ARTICLE INFO Keywords: Model predictive control Distributed control Multi-agent network Cyber-security ABSTRACT In this paper, we present an analysis of the vulnerability of a distributed model predictive control (DMPC) scheme in the context of cyber-security. We consider different types of the so-called insider attacks. In particular, we consider the situation where one of the local controllers sends false information to others to manipulate costs for its own advantage. Then, we propose a popular scenario-based mechanism to protect or, at least, relieve the consequences of the attack in a typical DMPC negotiation process. The theoretical and algorithmic properties of this defense mechanism are also analyzed. A real case study based on a four tank plant is provided to illustrate both the consequences of the attacks and the defense mechanisms. 1. Introduction Model predictive control (MPC) has become a popular control strategy due to the advantages it offers in comparison with other control approaches. An MPC controller can consider explicitly constraints on the manipulated variables and system states, nonlinearities on the model, delays, multiple objectives, etc. For this reason, this technique is widely used in numerous industrial applications, see, e.g., Camacho and Bordons (2004) and references therein. The main idea behind MPC is to calculate a control input sequence by solving a finite-horizon optimization problem (FHOP), based on the system model and its evolution, at each time instant. Only the first component of the control sequence is applied to the system at the current time; then the FHOP is solved again at the next time step (Grosso, Velarde, Ocampo-Martinez, Maestre, & Puig,2017). Nevertheless, geographically disperse systems such as road-traffic, logistics, transportation, water, and electrical networks may not allow to apply centralized MPC due to computational burden, issues with centralized modeling, data collection, and so on Negenborn and Maestre (2014). An alternative to deal with this kind of problems is to divide the whole system into subsystems, each one governed by a local MPC controller (or agent) that takes decisions and exchanges information with the other controllers under a negotiation process to obtain a possibly optimal global solution. This approach is called distributed MPC (DMPC) and has advantages such as ease of implementation, low computational effort in comparison with centralized MPC, modularity ∗Corresponding author. E-mail addresses: [email protected] (J.M. Maestre), [email protected] (P. Velarde), [email protected] (H. Ishii), [email protected] (R.R. Negenborn). of the system, among others. In this regard, there are many types of possible implementations that can be adapted to the specific features of each problem (Negenborn & Maestre,2014), e.g., sequential and parallel solutions, iterative and non-iterative methods, etc. For example, sequential and iterative architectures for DMPC are discussed in Liu, Chen, Muñoz de la Peña, and Christofides (2010). Tutorial reviews including design methods, algorithmic details, and an extensive discussion of the applications of DMPC are given in Christofides, Scattolini, de la Peña, and Liu (2013) and Maestre and Negenborn (2014). A topic that deserves attention is the regular exchange of information during the negotiation process among the local DMPC controllers. Most DMPC schemes have been designed considering a coordinated negotiation process where all controllers work in a reliable way. However, a malicious controller could exploit the vulnerabilities of the network by sharing false information with other controllers, producing an undesirable behavior in the optimization process. At this point, it is possible to speak about cyber-security in the context of DMPC. Well-known examples include Stuxnet (Kushner,2013), which was the first known worm designed to reprogram Siemens programmable logic controllers, forcing various shutdowns at the Natanz (Iran) nuclear plant (Albright, Brannan, & Walrond,2010), and Crash Override, which caused massive blackouts in Ukraine in 2016 (Bindra,2017). For this reason, in recent years, numerous investigations have devoted their efforts to finding solutions to these problems (Cheng, Shi, & Sinopoli,2017). The goal of cyber-security is to protect systems against https://doi.org/10.1016/j.conengprac.2021.104879 Received 23 February 2021; Received in revised form 30 May 2021; Accepted 22 June 2021 Available online 5 July 2021 0967-0661/©2021 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). J.M. Maestre, P. Velarde, H. Ishii et al. Control Engineering Practice 114 (2021) 104879 threat by preserving the confidentiality, availability, and integrity of the exchanged information (Radvanovsky & Brodsky,2016). Classical application areas are protection systems (Sheng, Chan, Li, Xianzhong, & Xiangjun,2007), Internet users (Kritzinger & Von Solms,2010), logistics (Li, Negenborn, & De Schutter,2015;Nabais, Negenborn, Carmona, & Ayala,2015;van Riessen, Negenborn, Lodewijks, & Dekker, 2015), power systems (Barreto, Giraldo, Cardenas, Mojica-Nava, & Quijano,2014;Chakhchoukh & Ishii,2015;Mc Namara, Negenborn, De Schutter, Lightbody, & McLoone,2016;Soudbakhsh, Chakrabortty, & Annaswamy,2017), among others. Control systems are not exempt from possible cyber-attacks (Teixeira, Sou, Sandberg, & Johansson, 2015;Zhu & Basar,2015), with consequences ranging from performance loss to instability. A resilient multivariable control framework for large-scale urban traffic networks subject to several types of cyberattacks is addressed in Mercader and Haddad (2021). Also, Dhal and Roy (2013) presents cyber-security risk assessment for supervisory control and data acquisition (SCADA) and distributed control system networks. In this context, cyber-security does not deal with security from a computer or telecommunications viewpoint, but with providing an additional layer of defense once attackers have already penetrated the control system. By exploiting the knowledge of the system dynamics, controllers can detect and isolate themselves from these threats, or at least mitigate their consequences. For example, in Ananduta, Maestre, Ocampo-Martinez, and Ishii (2020) Bayesian inference is used to detect noncompliant controllers in a distributed MPC setting so that negotiations can be stopped accordingly. In Maestre, Trodden, and Ishii (2018), the issue of noncompliance is addressed using a passive approach that allows agents to be protected against a limited amount of noncompliances in a distributed tube MPC setting. In Braun, Albrecht, and Lucia (2020) and Pierron, Árauz, Maestre, Cetinkaya, and Maniu (2020), stochastic MPC is used to build a tree and improve the controller performance in case of packet losses/attacks. This is also the approach followed in Yang, Li, Dai, and Xia (2019), where denial-of-service (DoS) attacks are considered, although in a setting where only one subsystem is allowed to optimize at each time step. DoS attacks are also dealt with within the framework of Sun, Zhang, and Shi (2019), where conditions are derived to guarantee exponential stability of the closedloop system. A different strategy is followed in Romagnoli, Krogh, and Sinopoli (2019), where the control software is periodically reinstalled to mitigate the impact of cyber agressions. Another interesting line of work is given in Trodden, Maestre, and Ishii (2020), which studies the extent of the control input set that can be gained by an attacker before relevant properties such as the existence of invariant sets is lost. Other proposed methods include encryption and coding schemes as a means to secure communication (Darup, Redder, Shames, Farokhi, & Quevedo,2017;Miao, Zhu, Pajic, & Pappas,2016) and the use of artificial intelligence methods such as neural-networks (Wu et al., 2018) and machine learning (Chen, Wu, & Christofides,2020) to detect cyber-attacks. In this paper, we explore cyber-security issues in Lagrange-based DMPC schemes. Recent studies in this direction include the schemes, e.g., for dual (Velarde, Maestre, Ishii, & Negenborn,2018), and Jacobi– Gauss decomposition (Chanfreut, Maestre, & Ishii,2018), where anomalies can be detected during the negotiation process. Moreover, some works discuss techniques to reject malicious agents during the consensus process (Ananduta et al.,2020;Ishii et al.,2020;Onogawa et al.,2019;Yang et al.,2019). This work analyzes Lagrange based DMPC and, following Velarde, Maestre, Ishii, and Negenborn (2017), shows how a malicious controller can take advantage of the vulnerabilities of the scheme to increase its own benefit at the cost of other controllers. To mitigate the consequences of the attacks, scenario-based techniques are proposed to gain robustness (Calafiore & Fagiano,2013). In particular, scenarios taken from trustworthy historical price information are used to robustify the control network against malicious controllers. As a matter of a fact, scenario-based MPC has been extensively used to deal with external uncertainties affecting control systems. For instance, multi-scenario MPC (MS-MPC) optimizes a single control sequence valid for all potential scenarios taking into account their probability of occurrences, e.g., in the context of water systems (Grosso et al.,2017;van Overloop, Weijs, & Dijkstra,2008; Tian et al.,2019;Velarde, Tian, Sadowska, & Maestre,2019) and smart grids control (Olivares, Lara, Cañizares, & Kazerani,2015). Based on this background, we propose to incorporate this approach in the DMPC formulation as a way to secure dual decomposition DMPC and deal with the internal threats from the distributed network. Also, as a case study to illustrate the proposed defense method, we carry out experiments in a real four tank plant. The remainder of this paper is organized as follows. First, dual decomposition based MPC is briefly introduced in Section 2. Section 3presents a number of threat schemes in the context of Lagrange based DMPC, where a malicious controller can exploit the algorithm. Section 4provides a secure dual decomposition technique based on MS-DMPC to mitigate the impact that an attacker can cause to the other controllers. Section 5analyzes the impact of the proposed scheme regarding the satisfaction of algorithmic and theoretical properties of interest. In Section 6, the aforementioned case study is presented to show the effects of potential attacks and how the proposed mechanisms relieve this issue. Finally, conclusions are drawn in Section 7. A preliminary version of this paper has appeared as Velarde et al. (2017); the current paper analyzes theoretical and algorithmic properties of the scenario based defense mechanism and moreover the results of the experiments based on the four tank plant are presented. 2. Dual decomposition based DMPC In this section, we present a commonly used distributed optimization algorithm based on dual decomposition (Biegel, Stoustrup, & Andersen,2014;Giselsson & Rantzer,2014). Let us consider a distributed system composed of 𝑁subsystems defined by discrete-time linear time-invariant models. The dynamics of subsystem 𝑖are given by 𝑥𝑖[𝑘+ 1] = 𝐴𝑖𝑥𝑖[𝑘] + 𝐵𝑖𝑢𝑖[𝑘],(1) where 𝑥𝑖∈R𝑛𝑥,𝑖 and 𝑢𝑖∈R𝑛𝑢,𝑖 denote the states and input of the system, respectively. 𝐴𝑖∈R𝑛𝑥,𝑖×𝑛𝑥,𝑖 is the state matrix and 𝐵𝑖∈ R𝑛𝑥,𝑖×𝑛𝑢,𝑖 represents the input matrix. Moreover, 𝑛𝑥,𝑖 and 𝑛𝑢,𝑖 represent the number of states and the number of inputs of the subsystem 𝑖, respectively. Each subsystem is subject to convex state and input constraints 𝑥𝑖[𝑘] ∈ 𝑖,∀𝑘∈Z+,(2a) 𝑢𝑖[𝑘] ∈ 𝑖,∀𝑘∈Z+,(2b) where Z+denotes the set of non-negative integer numbers. Let the aggregated vectors of states and inputs be 𝑥[𝑘] = [𝑥1[𝑘]T⋯𝑥𝑁[𝑘]T]T and 𝑢[𝑘] = [𝑢1[𝑘]T⋯𝑢𝑁[𝑘]T]T, respectively, where 𝑥∈R𝑛𝑥,𝑛𝑥= ∑𝑁 𝑖=1 𝑛𝑥,𝑖,𝑢∈R𝑛𝑢, and 𝑛𝑢=∑𝑁 𝑖=1 𝑛𝑢,𝑖. The 𝑁subsystems are also subject to constraints coupling the inputs: 𝐶𝑢[𝑘] = 𝑁 ∑ 𝑖=1 𝐶𝑖𝑢𝑖[𝑘]≤𝑐, (3) where 𝐶∈R𝑛𝑐×𝑛𝑢,𝐶𝑖∈R𝑛𝑐×𝑛𝑢𝑖and 𝑐∈R𝑛𝑐. Note that this formulation can easily cover typical coupling constraints in the states and in the inputs. We assume that a convex stage cost function for each subsystem is given by 𝓁𝑖(𝑥𝑖[𝑘+ 1], 𝑢𝑖[𝑘]).(4) Each subsystem 𝑖is controlled by a local MPC controller. The main idea of (centralized and distributed) MPC is to obtain a control signal by solving, at each time step, an FHOP that takes into account the prediction model of each subsystem. In particular, (1) is used to predict the evolution of the system along a given horizon 𝑁pas a function 2 J.M. Maestre, P. Velarde, H. Ishii et al. Control Engineering Practice 114 (2021) 104879 of the sequence of inputs provided. In this way, it is possible to calculate a control sequence 𝑢∗ 𝑖[𝑘∶𝑘+𝑁p− 1] that optimizes (4) along the horizon. The first component of the optimal control sequence obtained is implemented at the current time step, and the problem is solved at the next time step following a receding horizon strategy. The optimization problem over a fixed time prediction horizon 𝑁p∈Z+can be written as 𝑢∗ 𝑖[𝑘∶𝑘+𝑁p− 1] = arg min 𝑢𝑖[𝑘∶𝑘+𝑁p−1] 𝑘+𝑁p−1 ∑ 𝑗=𝑘 𝓁𝑖(𝑥𝑖[𝑗+ 1], 𝑢𝑖[𝑗]),(5) subject to (1)–(3), assuming that the predicted control actions and states of the rest of the subsystems are known. From an overall perspective, the stage cost function becomes 𝓁(𝑥[𝑘+ 1], 𝑢[𝑘]) = 𝑁 ∑ 𝑖=1 𝓁𝑖(𝑥𝑖[𝑘+ 1], 𝑢𝑖[𝑘]).(6) In this way, the optimization problem, from a global viewpoint, is given by min 𝑢[𝑘∶𝑘+𝑁p−1] 𝑘+𝑁p−1 ∑ 𝑗=𝑘 𝓁(𝑥[𝑗+ 1], 𝑢[𝑗]),(7) subject to (1)–(3). Due to the coupling in (3), controllers have to share information. It is necessary to consider the role played by coupling variables explicitly. Hence, the controllers have to coordinate their actions using a negotiation process. The dual decomposition approach consists of converting the coupled variables into local versions and then incentivize via cost the obtainment of a coordinated value. In this sense, the performance index is reformulated by means of the associated Lagrange multipliers as 𝐿(𝜂[𝑘], 𝛬[𝑘]) = 𝑘+𝑁p−1 ∑ 𝑗=𝑘 (𝓁(𝑥[𝑗+ 1], 𝑢[𝑗]) + 𝜆[𝑗]T(𝐶𝑢[𝑗] − 𝑐)),(8) where 𝜂[𝑘] = [𝑥[𝑘+ 1 ∶ 𝑘+𝑁p]T, 𝑢[𝑘∶𝑘+𝑁p− 1]T]Tis defined as the vector composed of the states and inputs along the horizon 𝑁p,𝜆[𝑗] ∈ R𝑛𝑐are the multipliers associated with the coupling constraints (3), and 𝛬[𝑘] = 𝜆[𝑘∶𝑘+𝑁p−1] is the sequence of the Lagrange multipliers along the horizon. Remark 1. Lagrange multipliers can be interpreted as prices that are used to coordinate the subsystems regarding the fulfillment of coupling constraints (Biegel et al.,2014). The optimal value of the problem for a given sequence of prices is defined as 𝑔(𝛬[𝑘]) = min 𝑢[𝑘∶𝑘+𝑁p−1] 𝑘+𝑁p−1 ∑ 𝑗=𝑘 (𝓁(𝑥[𝑗+ 1], 𝑢[𝑗]) + 𝜆[𝑗]T(𝐶𝑢[𝑗] − 𝑐)),(9) subject to (1) and (2), and allows to deal with (7) in a distributed manner by solving its dual problem maximize 𝑔(𝛬[𝑘]),(10) subject to 𝛬[𝑘]⪰0, by using a distributed gradient search, where ⪰represents componentwise inequality. The distributed control problem solved by dual decomposition is summarized in Algorithm 1(Biegel et al.,2014). Dual decomposition has been used in several applications, e.g., building temperature regulation (Yushen, Shuai, Xie, & Johansson, 2014), coordinating a network of households (Larsen, Van Foreest, & Scherpen,2014), ships (Zheng, Negenborn, & Lodewijks,2017), and logistics (Li, Negenborn, & De Schutter,2017). Algorithm 1 Dual decomposition based DMPC. 1: Each controller initializes its prices (Lagrange multipliers) 𝛬[𝑘]⪰0. All agents have the same initial values for the Lagrange multipliers. 2: repeat 3: Each controller solves its local optimization problem with the current value of 𝛬[𝑘], i.e., min 𝑢𝑖[𝑘∶𝑘+𝑁p−1] 𝑘+𝑁p−1 ∑ 𝑗=𝑘 (𝓁𝑖(𝑥𝑖[𝑗+ 1], 𝑢𝑖[𝑗]) + 𝜆[𝑗]𝑇𝐶𝑖𝑢𝑖[𝑗]),(11a) subject to 𝑥𝑖[𝑗+ 1] = 𝐴𝑖𝑥𝑖[𝑗] + 𝐵𝑖𝑢𝑖[𝑗],∀𝑗∈ [𝑘∶𝑘+𝑁p− 1],(11b) 𝑥𝑖[𝑗] ∈ 𝑖,∀𝑗∈ [𝑘∶𝑘+𝑁p− 1],(11c) 𝑢𝑖[𝑗] ∈ 𝑖,∀𝑗∈ [𝑘∶𝑘+𝑁p− 1].(11d) The solution of the optimization problem is denoted as 𝑥∗ 𝑖[𝑘+ 1 ∶ 𝑘+𝑁p],𝑢∗ 𝑖[𝑘∶𝑘+𝑁p− 1]. Then these values are exchanged with other controllers. 4: Each controller 𝑖determines the violations of the coupling constraints 𝑠[𝑘]≜∑𝑁 𝑖=1 𝐶𝑖𝑢∗ 𝑖[𝑘]−𝑐,𝑆[𝑘] = 𝑠[𝑘∶𝑘+𝑁p−1] ∈ R𝑁p×𝑛𝑐and calculates the new prices along the horizon 𝛬[𝑘] ∶= max[0, 𝛬[𝑘] + 𝛾𝑆[𝑘]], where 𝛾is the step size. 5: until max(𝑆[𝑘]) < 𝜖, where 𝜖is a prespecified threshold, or the maximum number of iterations reached. 6: Each subsystem implements the first component of the control sequence 𝑢∗ 𝑖[𝑘∶𝑘+𝑁p− 1]. 7: Let 𝑘=𝑘+ 1 and return to step 1. 3. Attacks in Lagrange-based DMPC Algorithm 1works in a reliable information exchange setting. If one of the controllers is malicious, the whole system can fail. In particular, we consider situations where one of the controllers is an attacker that uses false information affecting performance. For example, the attacker may manipulate the information shared with others to steer the overall negotiation process. This situation can be seen as a Stackelberg game (Başar & Srikant,2002), where the attacker is the leader, and the rest of the local controllers become followers. Three different ways in which an attacker can take advantage by exchanging false information with other controllers of the subsystems are presented in this section. 3.1. False reference A fake reference attack consists of the strategic use of the reference so as to steer the negotiation process. Any agent performing this attack is setting the local reference in a way that does not correspond to its true preference. Instead, it uses the altered reference as a means to make the controlled signal follow its desired value more closely at the expense of the rest of the controllers. Let us consider that a controller 𝑚∈ {1,…, 𝑁}attacks the rest of the controllers by using a false reference (𝑥𝑚∗ ref ) to bias the negotiation. Therefore, the stage cost function optimized by controller 𝑚is given by 𝓁∗ 𝑚(𝑥𝑚[𝑘+ 1], 𝑢𝑚[𝑘]) = 𝓁𝑚(𝑥𝑚[𝑘+ 1] − 𝑥𝑚∗ ref , 𝑢𝑚[𝑘]).(12) The use of a false reference can steer the negotiation process towards a result that is more beneficial for the attacker. In this sense, there is no incentive for the controllers to be honest regarding their real references because they can be better off in this way from a local perspective. Therefore, other controllers could follow the same type of strategic behavior, i.e., they could hide their preferences to bias the negotiation. The resulting situation would be that of a dynamic 3 J.M. Maestre, P. Velarde, H. Ishii et al. Control Engineering Practice 114 (2021) 104879 non-cooperative game where malicious agents use their references strategically to attain better local costs at the expense of those controllers that set their optimization problems using values that reflect their actual preferences. Therefore, compliant agents would behave in a cooperative manner that would be exploited by malicious agents pursuing their self-interest. Finally, Figs. 1 and 2.b show the case of the false reference attack for simple academic examples. Note that these figures are provided only for illustration purposes and therefore details regarding the numerical values employed are omitted. 3.2. Fake constraints Another way in which the attacking controller 𝑚can take advantage from its neighbor subsystem carrying out the optimization problem is to use fake constraints, e.g., 𝑥𝑚[𝑗] ∈ ∗ 𝑚,∀𝑗∈ [𝑘∶𝑘+𝑁p− 1],(13a) 𝑢𝑚[𝑗] ∈ ∗ 𝑚,∀𝑗∈ [𝑘∶𝑘+𝑁p− 1],(13b) where ∗ 𝑚and ∗ 𝑚are the modified sets with fake constraints. The remaining subsystems optimize their objective functions by considering their original constraints while the attacker uses constraints that steer the negotiation process by reducing its own cost function. Fig. 2.c illustrates the effect of this attack, in terms of the cost using a simple academic example with two agents and two shared variables. As can be seen, the attack has a very strong impact on the negotiation because it limits the set of feasible values of the negotiated variables. 3.3. Fake prices To obtain a better local cost, agent 𝑚can modify its performance index by including a new coefficient denoted as 𝛼by solving min 𝑢𝑚[𝑘∶𝑘+𝑁p−1] 𝑘+𝑁p−1 ∑ 𝑗=𝑘(𝓁𝑚(𝑥𝑚[𝑗+ 1]; 𝑢𝑚[𝑗]) + 𝜆[𝑗]𝑇 𝛼𝐶𝑚𝑢𝑚[𝑗])(14) with 𝛼 > 1, subject to (11)(b)–(11)(d). The solution is biased towards the interests of agent 𝑚because (14) is equivalent to min 𝑢𝑚[𝑘∶𝑘+𝑁p−1] 𝑘+𝑁p−1 ∑ 𝑗=𝑘(𝛼𝓁𝑚(𝑥𝑚[𝑗+ 1]; 𝑢𝑚[𝑗]) + 𝜆[𝑗]𝑇𝐶𝑚𝑢𝑚[𝑗]),(15) so that the global stage cost becomes 𝓁∗(𝑥[𝑘+ 1], 𝑢[𝑘]) = ∑ 𝑗≠𝑚 𝓁𝑗(𝑥𝑗[𝑘+ 1], 𝑢𝑗[𝑘]) + 𝛼𝓁𝑚(𝑥[𝑘+ 1], 𝑢[𝑘]),(16) prioritizing the local cost of agent 𝑚by 𝛼during the execution of Algorithm 1. Again, Fig. 2.d shows the effect of this attack on the trajectory followed by the negotiated variables and the corresponding local costs. As can be seen, the scaling factor 𝛼has a great power to promote the local interest of the malicious agent in the negotiation. 3.4. Attack tuning All the considered attacks correspond to malicious modifications of local optimization problems so as to improve the corresponding utility at the expense of the rest of the system. This is possible because the distributed control algorithm considered, which is very popular due to its simplicity, does not discourage this type of strategical behavior to increase local performance. How to optimize these modifications is problem specific and adjustments may be required depending on the reactions of other subsystems. Nevertheless, the local reference can be increased (decreased) above its current value to force the raise (decrease) of the corresponding controlled variable. Likewise, the constraints can be modified so as to restrict the optimization to certain values which result more beneficial for its local cost function. Finally, whenever the Lagrangian multipliers are divided, the overall optimization will be driven towards the local goal, with increased effects on local performance as 𝛼grows. 4. Secure scenario-based DMPC As seen in the previous section, the negotiation process can be manipulated so that the values of the prices in the coordination mechanism deviate from their optimal values. It is necessary to implement a method that relieves the potential effects of an intentional attack whenever this situation is detected. To this end, we propose a scenario-based approach to robustify the control network against malicious controllers. In particular, trustworthy (possibly audited) price information based on historical data will be used to generate scenarios. Since it is not possible to be certain about the type and number of attacks that can occur, we must first set the target attack levels to apply security measures, and then make sure that the actual attack level of the system to be protected falls within the level of the measure applied with some safety margins. To this end, the detection of attacks to activate the defense mechanism is based on two simple criteria that can be easily implemented. Let 𝛬[𝑘]be the vector of prices calculated at time step 𝑘and  𝛬[𝑘] be a vector of prices with nominal values for the current situation of the system obtained from historical data. The triggers proposed for the activation of the defense mechanism are the following: •Abnormal price values: this condition can be expressed as |𝛬[𝑘] −  𝛬[𝑘]|≥𝜅, (17) where 𝜅is a threshold that establishes a bound on the norm of the deviation of the prices with respect to their expected values. •Abnormal performance: if the performance of the overall system with the current prices is worse than that with nominal prices, the coordination is not working properly. Hence, the following condition can be checked based on (9): 𝑔(𝛬[𝑘]) − 𝑔( 𝛬[𝑘]) >0.(18) Note that (18) can be checked in a distributed fashion. Also, the fact that the local version of (18) is satisfied for many agents can be an indication that the coordination mechanism is under attack. It must be noted that the conditions (17) and (18) are a little conservative and it is possible that some attacks are not detected. For example, it may happen that 𝑔(𝛬[𝑘]) − 𝑔( 𝛬[𝑘]) ≤0(19) with the system being under attack. Such situation corresponds to the case where the attack leads to an increase of cost that is below the increase generated by the use of nominal prices. Therefore, the effect of the attack is not significant enough to be detected. Likewise, any attack whose effect on the prices satisfies |𝛬[𝑘] −  𝛬[𝑘]|< 𝜅, (20) with some constant 𝜅 > 0, will not be detected. 4.1. Scenario generation Scenario generation is necessary to relieve the effects of an attacker inside the network. The availability of audited historical data is demanding for some problem settings, but is also reasonable for systems with periodic exogenous inputs such as power grids and water networks, where clear patterns can be identified depending on factors such as time, day of the week and weather. It might also be performed using a stochastic model to generate synthetic data (Schildbach & Morari,2015), which is the approach we follow. Details are given below. In order to generate different scenario evolutions, we assume a nominal state is available at time step 𝑘for a system with periodic state trajectories. The initial value of the model used for the MPC problem at each step is then perturbed by white noise as  𝐱𝐢[𝑘] = 𝐱𝐢[𝑘] + 𝚗[𝑘],(21) 4 J.M. Maestre, P. Velarde, H. Ishii et al. Control Engineering Practice 114 (2021) 104879 Fig. 1. Result of the negotiation process carried out by two integrators that share the same manipulated variable: 𝑈1and 𝑈2represent the minimizer of the corresponding local cost function, which penalizes quadratically the control effort and the reference tracking error. The use of a false reference by agent 1 modifies its true cost function (red line), displacing it to the left (green dashed line), and the same happens with its minimizer, which becomes 𝑈∗ 1. As a consequence, the negotiated value is steered from 𝑈𝐷𝑀𝑃 𝐶 to 𝑈∗ 𝐹 𝑅 , which is indeed closer to 𝑈1, attaining an improved local cost with respect to its true preferences (red line). (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.) Fig. 2. Local cost level curves and trajectories of negotiated variables for a simple system with two input-coupled agents. The negotiation is performed under different conditions over the manipulated variables 𝑢1and 𝑢2(prediction horizon 𝑁p= 1): (a) standard DMPC (upper-left); (b) false reference (upper-right); (c) fake constraints (lower-left); and (d) fake prices (lower-right). In all the attacks considered, the negotiation outcome is closer to the minimizer of agent 1, resulting in increased costs for agent 2. 5 J.M. Maestre, P. Velarde, H. Ishii et al. Control Engineering Practice 114 (2021) 104879 where  𝐱𝐢[𝑘]represents the perturbed measurement of each state containing noise 𝚗[𝑘] ∼ (𝜇, 𝜎2), i.e., a Gaussian distribution with mean 𝜇 and standard deviation 𝜎. In this way, several experiments are repeated, and the price information 𝜆𝑖[𝑘]is collected as a scenario for each state variable. Starting from the state 𝑥𝑖[𝑘], the FHOP provides us with predictions of the state, input, and Lagrange multipliers. In particular, the scenario generation FHOP is formulated as follows: min 𝑢𝑖[𝑘∶𝑘+𝑁p−1] 𝑘+𝑁p−1 ∑ 𝑙=𝑘 (𝓁𝑖(𝑥𝑖[𝑙+ 1], 𝑢𝑖[𝑙]) + 𝜆[𝑙]𝑇𝐶𝑖𝑢𝑖[𝑙]),(22a) subject to 𝑥𝑖[𝑙+ 1] = 𝐴𝑖𝑥𝑖[𝑙] + 𝐵𝑖𝑢𝑖[𝑙],∀𝑙∈ [𝑘∶𝑘+𝑁p− 1],(22b) 𝑥𝑖[𝑙] ∈ 𝑖,∀𝑙∈ [𝑘∶𝑘+𝑁p− 1],(22c) 𝑢𝑖[𝑙] ∈ 𝑖,∀𝑙∈ [𝑘∶𝑘+𝑁p− 1].(22d) Note that 𝜆𝑖[𝑘∶𝑘+𝑁p−1] is computed at each time instant 𝑘. Hence, the set of trustworthy price scenarios of each controller 𝑖can be expressed as  𝛬𝑖[𝑘]={𝜆1 𝑖[𝑘∶𝑘+𝑁p−1], 𝜆2 𝑖[𝑘∶𝑘+𝑁p−1],…, 𝜆𝑁𝑠 𝑖[𝑘∶𝑘+𝑁p−1]},(23) where 𝑁𝑠is the number of scenarios generated, 𝜆𝑠 𝑖∈R𝑛𝑐corresponds to the 𝑠th scenario of the controller 𝑖for 𝑠∈ [1, 𝑁𝑠]. These scenarios are stored in a database. Note that offline training is carried out to store that information in a database to be used in an online way. Remark 2. Since our method relies on using data from previous executions, it is more suitable for periodic systems and repetitive tasks. Non-periodic-behavior systems could be considered as well if enough information from the past is available so that prices can be analyzed for any possible system state. 4.2. Multi-scenario DMPC (MS-DMPC) MS-DMPC provides robustness by computing a unique control input that ensures the satisfaction of the local constraints for all the potential trajectories determined by the set of scenarios. One issue that deserves special attention is the number of scenarios (𝑁𝑠) that guarantees the robustness of the whole system. A higher number of scenarios results in an over conservative control input and may lead to significant computation burden. The problem formulation of MS-DMPC for each controller 𝑖∈ [1, 𝑁]at each time instant 𝑘is expressed as min 𝑢𝑖[𝑘∶𝑘+𝑁p−1] 𝑁𝑠 ∑ 𝑠=0 𝜌𝑠 𝑖 𝑘+𝑁p−1 ∑ 𝑗=𝑘 (𝓁𝑖(𝑥𝑠 𝑖[𝑗+ 1], 𝑢𝑖[𝑗]) + 𝜆𝑠 𝑖[𝑗]𝑇𝐶𝑖𝑢𝑖[𝑗]),(24a) subject to 𝑥𝑠 𝑖[𝑗+ 1] = 𝐴𝑠 𝑖𝑥𝑠 𝑖[𝑗] + 𝐵𝑖𝑢𝑖[𝑗],(24b) 𝑥𝑠 𝑖[𝑗] ∈ 𝑖,∀𝑗∈ [𝑘∶𝑘+𝑁p− 1],∀𝑠∈ [0, 𝑁𝑠],(24c) 𝑢𝑖[𝑗] ∈ 𝑖,∀𝑗∈ [𝑘∶𝑘+𝑁p− 1],(24d) where 𝜌𝑠 𝑖is the probability of occurrence of each scenario 𝑠, which depends on how frequently it appears in the scenario database. Hence, ∑𝑁𝑠 𝑠=0 𝜌𝑠 𝑖= 1. The set of scenarios for each controller under the defense mechanism is formulated as {𝜆0 𝑖[𝑘∶𝑘+𝑁p− 1]}∪ 𝛬𝑖[𝑘], where 𝜆0 𝑖[𝑘∶𝑘+𝑁p− 1] results from the price update at each inner iteration of the dual decomposition DMPC algorithm. Therefore, the negotiation process when the defense mechanism is active combines at each inner iteration the current value of the sequence 𝜆0 𝑖and the trustworthy price scenarios. Note that the combination is weighted and the weights need not be constant, i.e., they could be updated at each time step to allow for the modulation of the defense mechanism. Table 1 Parameters of the Plant. Parameter Value Parameter Value ℎ0 10.57 m 𝑎11.31 × 10−4 m2 ℎ0 20.62 m 𝑎21.507 × 10−4 m2 ℎ0 30.58 m 𝑎39.627 × 10−5 m2 ℎ0 40.64 m 𝑎48.31 × 10−5 m2 𝑞0 𝐴1.63 m3∕h 𝛾𝑎0.30 𝑞0 𝐵2.00 m3∕h 𝛾𝑏0.40 𝑆{1,2,3,4} 0.03 m2 Therefore, the weight 𝜌𝑠 𝑖assigned to the sequence of current prices 𝜆0 𝑖[𝑘∶𝑘+𝑁p− 1] can be reduced as long as abnormal behavior persists or restored as current prices converge again to their expected values. In particular, 𝜌𝑠 𝑖= 0 reflects that current prices cannot be trusted whereas 𝜌𝑠 𝑖= 1 indicates that the coordination system is working as expected. In this work, we consider an abrupt transition of 𝜌𝑠 𝑖between the previously mentioned extreme cases once an attack is detected, but smoother update rules are also possible. Remark 3. The scenarios in  𝛬𝑖[𝑘∶𝑘+𝑁p−1] are based on trustworthy historical data obtained in similar situations. For this reason, the set  𝛬𝑖[𝑘]may change with time. How to retrieve the most appropriate  𝛬𝑖[𝑘] for the current state of the system is a matter of future study. At this moment, the topic goes beyond the scope of this work. Remark 4. The probability for each scenario can be modified depending on how much the local agent suspects that it is under attack. Each agent can learn based on previous behaviors and when it has been attacked. In this manner, the confidence of the information from its neighbors is reduced in the event of an attack. Otherwise, it is increased. For example, 𝜌0 𝑖= 0 if there is no confidence in the coordination process. Remark 5. The defense mechanism is based on prices, which capture the interlocking influence from the rest of the system. From this viewpoint, prices may be abnormal due to the influence of more than one agent. The only issue here comes if one agent modifies its constraints, but even this can be mitigated by switching to a decentralized operation based on historical prices (e.g., if the probability of the current scenario becomes zero). 5. Theoretical and algorithmic properties of the defense mechanism The proposed method has been designed as a defense mechanism to be used when attacks are detected. In this section, we briefly discuss the consequences that follow from its utilization from a theoretical viewpoint. In the first place, we recall that dual decomposition is a coordination mechanism that allows solving a centralized control problem in a distributed fashion. In particular, it provides a way of decoupling shared constraints so that the computation of the solution of the control problem can be performed by different entities in the control network. Hence, any theoretical property such as stability or robustness that needs to be guaranteed must be derived from the optimization problem being distributed. Nevertheless, any modification of the prices with respect to their optimal values makes the distributed solution different from the centralized one. For this reason, the impact of the defense mechanism is as follows: •Feasibility of the local optimization problems: prices enter as additional linear terms on the cost of the optimization problem. Hence, they do not affect the domain of the value function of 6 J.M. Maestre, P. Velarde, H. Ishii et al. Control Engineering Practice 114 (2021) 104879 the local control law. For this reason, the feasibility of local quadratic programming problems is not endangered by the use of the defense mechanism. •Convergence of the DMPC algorithm: this property depends on the update of the prices, which is performed by means of a gradient search. However, to ensure convergence, the probability 𝜌0 𝑖assigned to 𝜆0 𝑖[𝑘∶𝑘+𝑁p− 1] has to be greater than zero. Note that if 𝜌0 𝑖= 0, the agent using the defense mechanism becomes disconnected from the updates of the prices. This will steer the coupled variables towards the values set by the defender. Hence, if the attacker is using a fake constraint set that excludes the values of the shared variables used by the defender, convergence will be lost and the algorithm will terminate when the maximum number of iterations is reached. In order to avoid issues in this regard, we will assume that all coupled variables are subject to the same set of constraints and 𝜌0 𝑖>0. •Coupled constraints satisfaction: in dual decomposition, whenever the iterations are stopped before convergence has been attained, the value of the local variables related with the coupling constraints may differ. In other words, iterates need not be feasible regarding the satisfaction of coupling constraints. In this situation, a projection onto the feasible set to fulfill these constraints is necessary. Given that dual decomposition typically converges slowly, it is not rare that iterations have to be stopped prematurely due to the timing constraints imposed by the sampling rate. Here, the same situation may occur and the same solutions used in standard dual decomposition have to be employed, e.g., a coordinator layer makes the projection, agents agree to implement a mean value of their shared variables, and so on. Nevertheless, note that there is no problem regarding the satisfaction of coupled constraints when convergence is attained. Hence, from a practical viewpoint, there is no impact of the defense mechanism in this regard. •Local constraints satisfaction: these constraints are satisfied because they depend only on local optimization variables. Given that the local problems are feasible, they will be satisfied. •Stability: this property is inherited from the centralized control problem being distributed. Depending on how it is achieved, the attacks and the defense mechanism may have an impact on it. In particular, there is a high risk of losing the stability guarantees of the DMPC scheme if the property depends on the fulfillment of coupled constraints, as was shown before, e.g., a terminal region/terminal cost approach. In the case where the satisfaction of this property is critical, it is advisable to use robust formulations of the local controllers with respect to the value of the shared variables. Also, the use of a coordination layer or a supervisor may help to guarantee stability. •Optimality: given that the distributed solution is not computed with optimal price values, there is a loss of optimality with the defense mechanism. Nevertheless, note that the attack generates a loss of optimality as well, so this property was lost in any case. The loss of optimality can be quantified. To this end, let us define 𝑔𝑖(𝜆𝑖) = min 𝑢𝑖[𝑘∶𝑘+𝑁p−1] 𝑘+𝑁p−1 ∑ 𝑗=𝑘 (𝓁𝑖(𝑥𝑖[𝑗+ 1], 𝑢𝑖[𝑗]) + 𝜆𝑖[𝑗]𝑇𝐶𝑖𝑢𝑖[𝑗]) (25) subject to (11)(b)–(11)(d), where 𝜆𝑖[𝑗]is the price used by agent 𝑖at time 𝑗. Here, we take 𝜆𝑖[𝑗] = 𝑁𝑠 ∑ 𝑙=0 𝜌𝑠 𝑖𝜆𝑙 𝑖[𝑗](26) if agent 𝑖is implementing the defense mechanism. Note that the attacker may be using fake prices. It can be easily checked that the following inequalities must hold: 𝑁 ∑ 𝑖=0 𝑔𝑖(𝜆𝑖[𝑗]) ≥ 𝑁 ∑ 𝑖=0 𝑔𝑖( 𝜆𝑖[𝑗]) ≥ 𝑁 ∑ 𝑖=0 𝑔𝑖(𝜆∗ 𝑖[𝑗]) ≥ 𝑁 ∑ 𝑖=0 𝑔𝑖(𝟎),(27) where 𝜆∗ 𝑖[𝑘]is the optimal price obtained by standard dual decomposition when no attack is performed and  𝜆𝑖[𝑘]are nominal values according to (18). As can be seen, the global cost in case of attack is greater than the nominal cost (according to the trigger for the activation of the defense mechanism) and the optimal global cost, which in turn is greater than the sum of local costs when they are optimized selfishly. Note that the first, second, and fourth expressions in (27) can be easily calculated during the iterations of the DMPC scheme. Hence, (27) provides us with a means to calculate a suboptimality bound of the dual decomposition scheme when the defense mechanism is implemented. 6. Case study and results In this section, we present a case study based on experiments with the four tanks system (Johansson,2000), which can be partitioned into two subsystems. The case study is used to test the effects of attacks with standard dual decomposition DMPC and the aforementioned scenario based approaches to robustify the overall system. We must remark that the attacks and defense mechanisms applied are only given for illustration purposes. The attacks are relatively simple and have not been optimized by any means. The same holds for the defense mechanism proposed. Instead, we have used a reduced set of the scenarios to show the effectiveness of the defense mechanism even if it is applied empirically. 6.1. Description of the four tank plant The four tank plant used consists of a modification to the four interconnected water tanks presented by Johansson (2000) to test control techniques using industrial-type instrumentation and control systems (Alvarado et al.,2011). A picture of the plant used in our experiment is shown in Fig. 3a. A plant diagram can be seen in Fig. 3b. It is composed of four tanks: two top tanks (3 and 4), which discharge into two bottom ones (1 and 2). Each tank is filled with the flow from a storage tank situated at the bottom of the plant by two pumps (𝑞𝐴 and 𝑞𝐵). The input flows are regulated by three-way valves. The system is modeled as follows: 𝑑ℎ1 𝑑𝑡 = − 𝑎1 𝑆1√2𝑔ℎ1+𝑎3 𝑆3√2𝑔ℎ3+𝛾𝑎 𝑆1 𝑞𝐴,(28) 𝑑ℎ2 𝑑𝑡 = − 𝑎2 𝑆2√2𝑔ℎ2+𝑎4 𝑆4√2𝑔ℎ4+𝛾𝑏 𝑆2 𝑞𝐵,(29) 𝑑ℎ3 𝑑𝑡 = − 𝑎3 𝑆3√2𝑔ℎ3+(1 − 𝛾𝑏) 𝑆3 𝑞𝐵,(30) 𝑑ℎ4 𝑑𝑡 = − 𝑎4 𝑆4√2𝑔ℎ4+(1 − 𝛾𝑎) 𝑆4 𝑞𝐴,(31) where 𝑥𝑖is the flow level of each 𝑖th tank, 𝑆𝑖is the cross section, 𝑎𝑖 is the discharge constant, 𝑔is the value of the gravity, and 𝛾𝑗is the ratio of the three-way 𝑗th valve, with 𝑗∈ {𝑎, 𝑏}. The values of these parameters are given in Table 1. The centralized discrete-time linear time-invariant model was obtained at operating point (𝑞0 𝐴,𝑞0 𝐵, and ℎ0 𝑖), with a sample time of 5s. The system is defined as 𝑥[𝑘+ 1] = ⎡⎢⎢⎢⎢⎣ 0.9325 0 0.0419 0 0 0.9297 0 0.0390 0 0 0.9566 0 0 0 0 0.9595 ⎤⎥⎥⎥⎥⎦ 𝑥[𝑘] +⎡⎢⎢⎢⎢⎣ 0.0134 0.0006 0.0006 0.0179 0 0.0272 0.0317 0 ⎤⎥⎥⎥⎥⎦ 𝑢[𝑘],(32) 7 J.M. Maestre, P. Velarde, H. Ishii et al. Control Engineering Practice 114 (2021) 104879 Fig. 3. The four tank system. Fig. 4. Water levels and cumulative costs by using standard DMPC approach. where 𝑥𝑖[𝑘] = ℎ𝑖[𝑘] − ℎ0 𝑖for 𝑖∈ {1,2,3,4} and 𝑢𝑗[𝑘] = 𝑞𝑗[𝑘] − 𝑞0 𝑗for 𝑗∈ {𝐴, 𝐵}, for each time instant 𝑘∈Z+. Also, Table 1 shows the parameters employed in the water-tank process. In order to ensure the correct performance of the plant and its equipment, the system is subject to constraints given by 0.2 m ≤ℎ1[𝑘], ℎ3[𝑘]≤1.36 m,(33a) 0.2 m ≤ℎ2[𝑘], ℎ4[𝑘]≤1.36 m,(33b) 0 m3∕h ≤𝑞𝐴[𝑘]≤3.26 m3∕h,(33c) 0 m3∕h ≤𝑞𝐵[𝑘]≤4 m3∕h.(33d) The optimization problem for this system is given by 𝐽(𝑥[𝑘], 𝑢[𝑘]) = (34) min 𝑢[𝑘∶𝑘+𝑁p−1] 𝑘+𝑁p−1 ∑ 𝑙=𝑘(𝑥[𝑙+ 1] − 𝑥ref )𝑇𝑄(𝑥[𝑙+ 1] − 𝑥ref )+𝑢𝑇[𝑙]𝑅𝑢[𝑙], subject to (33). Here, 𝑥ref is the given reference level for the state, and 𝑄∈R𝑛𝑥×𝑛𝑥and 𝑅∈R𝑛𝑢×𝑛𝑢are the weights for the states and the control inputs, respectively. 8 J.M. Maestre, P. Velarde, H. Ishii et al. Control Engineering Practice 114 (2021) 104879 Fig. 5. Water levels and cumulative costs by using a false reference attack. 6.2. Standard dual decomposition DMPC The overall system can be decomposed into two input-coupled subsystems following the partitioning proposed in Alvarado et al. (2011), which accounts for its structure and the coupling between subsystems when pumps are activated. In particular, the first subsystem is composed of tanks 1 and 3, and the second one is composed of tanks 2 and 4. The optimization problem for each subsystem 𝑖is expressed as 𝑆𝑖∶𝐽𝑖(𝑥𝑖, 𝑢𝑖) = (35a) min 𝑢𝑖[𝑘∶𝑘+𝑁p−1] 𝑘+𝑁p−1 ∑ 𝑙=𝑘(𝑥𝑖[𝑙+ 1] − 𝑥𝑖ref )𝑇𝑄𝑖(𝑥𝑖[𝑙+ 1] − 𝑥𝑖ref )+𝑢𝑇 𝑖[𝑙]𝑅𝑖𝑢𝑖[𝑙], subject to 𝑥𝑖[𝑙+ 1] = 𝐴𝑖𝑥[𝑙] + 𝐵𝑖𝑢𝑖[𝑙],∀𝑙∈ [𝑘∶𝑘+𝑁p− 1],(35b) 𝑥𝑖[𝑙] ∈ 𝑖,∀𝑙∈ [𝑘∶𝑘+𝑁p− 1],(35c) 𝑢𝑖[𝑙] ∈ 𝑖,∀𝑙∈ [𝑘∶𝑘+𝑁p− 1].(35d) Also, both subsystems are subject to the coupling constraint 𝑢1[𝑙] = 𝑢2[𝑙],∀𝑙∈ [𝑘∶𝑘+𝑁p− 1].(36) Here 𝑥1[𝑘] = [ℎ𝑖[𝑘] − ℎ0 𝑖]𝑖∈{1,3},𝑥2[𝑘] = [ℎ𝑖[𝑘] − ℎ0 𝑖]𝑖∈{2,4}, and 𝑢1[𝑘]and 𝑢2[𝑘]correspond to 𝑞𝐴[𝑘] − 𝑞0 𝐴and 𝑞𝐵[𝑘] − 𝑞0 𝐵, respectively. Moreover, 𝐴1,𝐴2,𝐵1, and 𝐵2are time invariant matrices of the system. The experiments were carried out by using a prediction horizon of 𝑁p= 5 along a test time of 20 min, i.e., 240 time steps. The reference levels were established as ℎ1ref = 0.5m, ℎ2ref = 0.6m, ℎ3ref = 0.7m, and ℎ4ref = 0.8m. Moreover, 𝑄and 𝑅were respectively set as 𝐼2×2 and 0.01𝐼2×2 for both subsystems to prioritize the reference tracking over the control effort, with 𝐼2×2 being the unit matrix of corresponding dimension. In our experiments, the setpoint has been intentionally set to make it difficult for the agents to control the system. Even if no attacks are performed, the control system is unable to reach its target, forcing local controllers to negotiate in order to attain a trade-off between their corresponding goals. In this condition, it is easier to see how one agent can take advantage of the other one by manipulating the negotiation process. Likewise, the four-tank plant is very versatile and the problem setup has been slightly modified so as to increase the coupling between the subsystems. In particular, we have included a reference value for each tank, whereas it is usual to focus only on the two bottom tanks, which are considered as the outputs of the plant (Alvarado et al.,2011; Johansson,2000). Remark 6. A reachable setpoint might seem a better choice for the experiments, but it brings several complications. To begin with, what is the selfish incentive that the attacker has to modify its local problem if it is at equilibrium? At equilibrium it reaches the lowest possible cost according to its true preferences, and therefore there is no possibility to be better off by altering its local problem. Clearly, it might attempt to modify the transient so as to attain a better trajectory and accumulated cost as it reaches its target, but this requires dynamically optimizing the attack in a way that is beyond the scope of our work. Another possibility is to consider that the attacker has a purely malicious behavior and desires to disrupt the normal operation of the system, maintaining its attack at all times. While this will surely modify the steady-state values (as it is illustrated for example in Figs. 1 and 2), it deviates from the assumption of rationality in the attacks that is considered in our work. All things considered, and given that the focus of the article is on the effects of the attacks and the defense mechanism, the setpoint was intentionally set unreachable to avoid these issues and make it difficult for the agents to control the system. Nevertheless, if this would not be the case, our experiments show that the system variables of the agents using the defense mechanism will tend to their nominal values. Fig. 4 shows the results for both agents obtained by using the standard dual decomposition based DMPC. Note that once convergence 9