Coalitional Control: Cooperative Game Theory and Control

1 Coalitional Control Cooperative game theory and control Filiberto Fele, Jos´ e M. Maestre, and Eduardo F. Camacho The evolution of information and communication technologies (ICT) has yielded means of sharing measures and other information in an efficient and flexible way [2], increasing the size and complexity of control applications [3]. At the same time, the improvements in the computational and communicational capabilities of control devices have fostered the development of non-centralized control architectures, already motivated by the inherent structural constraints of large-scale systems. Computer-based control approaches such as model predictive control (MPC) are visible beneficiaries of these advances and have registered a significant growth regarding both theoretical and applied fields [4], [5]. Whether or not the system arises from the interaction of different entities, it is generally possible to identify a set of coupled local control problems, often defining a clear structure, that jointly configure the global one. Hence, the variables of a system can be grouped to highlight weakly coupled blocks: within each block (usually designated as neighborhood) dynamic interactions propagate quickly, affecting the rest of the system on a longer time scale [6]. In most cases centralized strategies do not exploit such structure, sometimes leading to unviable computational or communicational requirements. As a natural way of avoiding the dependence on unavailable information—as well as to keep computational requirements at a minimum—it is desirable to formulate control laws based exclusively on local information [7], [8]. Non-centralized control strategies seek a tradeoff between performance loss and a scalable and flexible implementation [2]. In general, though, the stronger the interaction among different Financial support by the FP7-ICT project DYMASOS (ref. 611281), Spanish MINECO project COOPERA (DPI2013-46912) and Junta de Andaluc´ ıa project “Gesti´ on ´ Optima de Edificios de Energ´ ıa Cero” (P11-TEP-8129) is gratefully acknowledged. The authors are with the Department of Systems Engineering and Automation, University of Seville, 41092 Seville, Spain. Corresponding author: F. Fele ([email protected]). The final version of the article is available at https://doi.org/10.1109/MCS.2016.2621465 (please cite as [1]). ©2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. 2 parts of a system, the denser the communication required between the control agents—the extreme case corresponding to a distributed solution of the centralized control problem [9]. An extra degree of flexibility in the optimization of the computational and communicational requirements can be derived by the online identification of subsystems’ interactions and the consequent real-time adjustment of the control law structure. This is the idea behind coalitional control, a novel theory inspired by cooperative games, where the control strategy adapts to the varying coupling conditions between the controllers, promoting the formation of coalitions—clusters of controllers that cooperate in order to benefit from a jointly optimized control action. The adaptation of the controller topology may be the result of a top-down architecture, that is, imposed by a supervisor [10]–[12], or of a bottom-up approach [13], an autonomous coalition formation process occurring between the control agents. In all cases, the outcome will be a dynamically evolving coalitional structure of the overall controller. Coalitional control focuses on the local interests that motivate the controllers to assemble, an aspect so far rarely contemplated in the distributed control literature. Indeed, although a well-defined global organizational objective may be present in large-scale infrastructures, it is not uncommon for the individual components to show interests that do not align with the global one [14]. The smart grid and the intelligent transportation system are clear examples: a consistent research effort is being devoted to the issues associated with their management, typically involving different game-theoretic models in order to grasp the complex interaction phenomena produced by their heterogeneous user population [15], [16]. This article presents the main concepts and challenges in coalitional control, and the links with cooperative network game theory. In the remainder, a path is traced from the issues related with model partitioning— a fundamental step in the design of distributed controllers—to the solutions proposed in the literature whose principles delineate the idea of coalitional control. I. CONTROLLING LARGE-SCALE SYSTEMS:THE PATH TO COALITIONAL CONTROL Consider a large-scale system described by the following discrete-time model: x+=f(x, u), y=g(x), (1) where x∈Rnand u∈Rqare the global state and control input vectors, respectively, constrained in the sets Xand U, and y∈Rpis the vector gathering the outputs of the system. 3 A standard practice is to implement controllers on top of an already defined plant’s structure—usually following structural constraints and fault tolerance requirements. This typically involves a prior partition of (1) into a set N={1, . . . , N}of submodels, such that the essential dynamics characterizing the system are retained. Dynamic interactions typically occur between adjacent subsystems: coupling effects sensed beyond neighboring subsystems are seen through intermediate subsystems. Hence, the system’s topology appears as a fundamental guideline for the reduction of the overall information exchange, which represents a critical aspect of distributed cooperative MPC schemes [9]. It is clear how closely related the problems of system partitioning and distributed control are. Coalitional control encompasses both of them, allowing the dynamic partitioning of the system into cooperative components. A. Model partitioning Model partitioning consists in the assignment of subsets of the global state and input variables to each of the agents involved in the control of a system, such that x={xi}i∈N ∈Rn,u={ui}i∈N ∈Rq. According to the classification provided by [17], the system decomposition can be either horizontal or hierarchical. The first type relates to the physical structure of the system, while the second is based on the nature of the process, its characteristic time scales, and the control objectives. Hierarchical decompositions provide larger flexibility in shaping the controller to the heterogeneity of many large-scale systems—different sampling times, asynchronous operation of the different parts [6], [18]. A general systematic methodology is hard to define due to the manifold nature (subsystems’ interactions, timescale, communication constraints, privacy concerns) of controlled systems, so that ad hoc approaches are frequently implemented. Once the global model has been partitioned, each resulting subsystem i∈ N is assigned to a local control agent that has partial knowledge of the overall system, such that its behavior can be described by x+ i=fi(xi, ui) + wi, yi=gi(xi) + ϖi, (2) where xi∈Rniand ui∈Rqiare respectively the local state and input vectors, constrained in the sets Xiand Uirespectively, and yi∈Rpiis the local output vector. The vector wi∈Rnirepresents the measurable state disturbances resulting from the coupling with other subsystems, wi=X j∈Mi fij(xj, uj),(3) 4 where Mi≜{j∈ N \ {i}:there exists (xj, uj)∈ Xj× Uj|fij(xj, uj)=0}is referred to as the neighborhood of subsystem i. Similarly, ϖi∈Rpiexpresses the measurable disturbances on the local output. Notice that neither the external state and inputs xjand ujnor their relation fij with the local ones are known a priori by the local agent, eventually inducing some modeling error due to possibly neglected state–state, input–state, or input–output interactions. In the search for the optimal structure of the control system, an important limiting factor comes from the available computation and communication resources. For instance, the excessive partitioning of the system may reduce the model size and hence the computational requirements at each node, yet at the expense of an increased communication load—which in turn limits the possibilities of a parallel implementation of the control algorithm. Similarly, while a small sampling time allows to obtain a model that reflects the actual structure of the system, it will result in high communication rates between nodes, as well as in a shorter time available for computation [2]. Besides, more often than not, large-scale systems are characterized by multi-scale dynamics—typically slower overall dynamics arising out of a group of subsystems with fast dynamics. With specific attention towards the strong relationship between model decomposition and sampling time, a structural analysis for decomposition is presented in [2], oriented to the implementation of a hierarchical multi-rate estimation and control architecture. In [19], the coupling structure of the plant is analyzed prior to the design of feedback laws in order to decompose the model into hierarchically coupled clusters. A two-layer multirate control for such hierarchical decomposition is proposed in [20], where information is exchanged at each time step within clusters of strongly coupled subsystems, while a slower communication rate is required between different clusters. The interaction among subsystems can be as well viewed as multiplicative output uncertainty. In [21], the selection of best model partition within a given set of candidates is based on their associated (open loop) uncertainty bounds. The implications of system decomposition on robustness are investigated in [22]. The robustness of distributed MPC schemes characterized by different levels of knowledge of the non-local dynamics of the system is evaluated by means of an H∞index. The formulation of a multi-objective MINLP based on this index allows to seek a tradeoff between global robust performance and the degree of connectivity among agents (translated in terms of local interaction model coverage). Given Nsubsystems, the MINLP considers N(N−1) binary variables to choose the structure of the controller over all possible connections among the subsystems. Simulation results show how controllers characterized by 5 dense connectivity are more sensitive to errors in the interaction models. When these are affected by significant uncertainty, a fully decentralized structure can provide higher robustness. Several other methods have been proposed in the literature. In [23], the most relevant variables of the system are controlled by a central coordinator MPC, and a set of decentralized controllers complete the control action by responding to the inputs of the coordinator. The work of [24] addresses the assignment of actuators to a given set of controllers on the basis of two criteria, in order to achieve submodels of manageable size for MPC control. The first is an open-loop criterion based on maximizing the connectivity of the weighted graph representing the system, as an expression of the Hankel norm resulting by the controllability and observability Grammians. The second is a closed-loop criterion that aims at minimizing the performance degradation due to model partitioning, measured through the MPC’s cost function. A method for the derivation of a distributed model is suggested in [25], based on the Kalman canonical form of the linear state-space model for each input-output pair [26]. An exhaustive study of the issues related to model partitioning for the decentralized control of large-scale systems can be found in [7]. One contribution of [7] is that of showing how graph theory methods are well suited to gain an insight about the structural controllability and observability of large-scale systems, and employ these as a basis for model decomposition. Graph theory methods have been as well employed for the analysis of the impact of system topology on controllability and observability in [27], [28]. Based on graph partitioning is the work of [29], where a decomposition method providing a set of nonoverlapping subgraphs, with balanced number of vertices and minimal number of interconnecting edges, was developed. More recently, a threshold on the Shapley value [30] of the potential communication links between the controllers was proposed in [31] as a criterion for partitioning. B. Cooperative control We assume in the remainder that the objective of each local controller is to drive the subsystem’s state towards the origin of the state space. The cost of subsystem iat any given time step kis expressed by ℓi(xi, ui), referred to as the stage cost. At time k, a control sequence is derived by the solution of the 6 MPC problem min ui Ji= Np−1 X t=0 ℓi(xi(t|k), ui(t|k)) + Vi(xi(Np|k)) (4a) s.t. xi(t+ 1|k) = fi(xi(t|k), ui(t|k)) + ˆwi(t|k),(4b) hin i(xi(t|k), ui(t|k)) ≤0, t = 0, . . . , Np,(4c) heq i(xi(t|k), ui(t|k)) = 0, t = 0, . . . , Np−1,(4d) xi(0|k) = xi(k).(4e) The first element of the minimizer u∗ i≜[ui(0|k)∗, ui(1|k)∗, . . . , ui(Np−1|k)∗]is applied as input to the subsystem, and the problem solved again at subsequent time steps in a receding horizon fashion. Notice that, in absence of measures from the rest of the system, an estimate of the disturbance term (3) is employed in the solution of (4). For this reason, decentralized schemes generally require either assuming a priori bounds on the coupling between subsystems, or considering worst-case interactions, resulting in a loss of performance [2]. Communication between local controllers allows to achieve a visible enhancement in the overall control performance; however, it is difficult to synthesize a general result, due to the dependence on the particular scheme used [32], [33]. It is essential that the communication serves as a means to reach a shared consensus—an improvement over the decision that each agent would be able to make by relying merely on locally available information. Once the model partitioning is given, the control by means of a cooperative scheme requires to answer fundamental questions. We analyze some of them in the following. a) How communication improves performance?: According to the degree of selfishness of the objective function employed, two categories of controllers emerge in the distributed MPC literature: noncooperative—where the agents pursue a local objective—and cooperative—where the influence on the rest of the system, including how non-local objectives are affected, is accounted for in the choice of the inputs locally applied by each agent. More specifically, a global objective—consisting of a given (weighted) combination of individual objectives—is optimized over local inputs ui,i∈ N, min ui J= Np−1 X t=0 ℓ(xi(t|k), ui(t|k), x−i(t|k), u−i(t|k)) + V(xi(Np|k), x−i(Np|k)),(5) 7 subject to (4b)–(4e) (subscript −idesignates all subsystems j∈ N \ {i}). Several techniques can be applied to obtain feedback over the values of nonlocal variables in (5): typically, controllers update these values over intermediate iterations, by direct exchange or through Lagrangian multipliers. The importance of cooperation among MPC control agents is underlined in [9]: the exchange of information among subsystems provided with mutual interaction models does not constitute a sufficient guarantee for closed-loop stability, due to the competition arising from the pursuit of conflicting objectives. Similar conclusions are given by [34], showing that increased cooperation not always translates to a gain in performance; indeed, in some cases it may even lead to a performance loss. This emphasizes the importance of the criteria upon which the cooperation is based. In case that the interests of local controllers diverge from the global benefit, a reward scheme may be implemented at the supervisory layer in order to bring these interests as much in line as possible [14]. b) Who should communicate with whom?: The control agents communicate through a data network whose topology can be described by means of the undirected graph G= (N,E), where to each subsystem in N is assigned a node. Dependence of the optimal topology E ⊆ N × N of the communication graph on the coupling between the local control problems is expected. The answer can be derived from a case-oriented evaluation of the performance deterioration due to the absence of communication between a given pair of local controllers. Control architectures for large-scale systems typically include a supervisory level, which may act as a coordinator to set the necessary information flows [14]. The sharing of information in a networked system can be represented through a knowledge graph, where each node stands for the model of a given subsystem, and the edges indicate that the information about the models is available to the agents controlling the pointed nodes. The connectivity and the degree distribution of such graph can be interpreted in quantitative terms [35]. In [34], a graph representing the coupling between subsystems is updated at each time step by identifying the constraints that were active at the previous time step. Then, the cooperating sets are formed by those subsystems that are connected by a path in the graph. Further control-related studies, such as [36], [37], focus on the PageRank index, a variant of the eigenvector centrality measure, used to quantify the relevance of nodes from a coalition formation viewpoint. c) What information should be available to each local agent?: The answer to this problem depends on the coupling source and the type of distributed control strategy that is implemented. In general, the need for information exchange is inversely proportional to the coverage of the locally available information. In [38], the control is performed by means of a (global) linear feedback law. Continuous exchange of information among agents is avoided by providing dynamic models of the coupled subsystems to each 8 agent, used to predict an estimate of the evolution of its neighbors’ state when no new measure is communicated. In [11] the effect of input coupling is viewed—on the grounds of the slow dynamics of the plant under study—as a constant disturbance along the prediction horizon, which is then estimated with a Kalman filter. Usual choices are the expected state and/or input sequences [39], [40], or auxiliary coordination variables such as prices [41] or sensitivities [42]. See also [43] for further details regarding the information exchange in different schemes. However, regardless of the knowledge of the global dynamics of the system that may be available to each agent, the closed-loop behavior can range from stable and almost optimal, to unstable Nash equilibria if conflicting objectives are pursued. This sensible difference in performance is likely to be detected when dealing with strongly coupled subsystems [9]. d) How to deal with constrained communications?: The system dynamics impose limits on the time available to make agreements over cooperative decisions. Limitations in bandwidth or in energy consumption, as well as communication delays or packet losses concerning the data link infrastructure, need to be taken into account. Some algorithms are more flexible than others in this regard, providing superior robustness to delays or failures in communication. For example, the strategy presented in [25] admits the injection of suboptimal control actions in order to relax the heavy communicational requirement typical of iterative distributed schemes. Through a hierarchical design, this same strategy has been later extended to allow the asynchronous update among different neighborhoods [6]. Another example of algorithms showing such properties can be found in [40], where the robust tube feedback formulation admits steps with no updates (by employing shifted input sequences). C. Dynamic neighborhoods and coalitional control The coupling among subsystems can be categorized over three classes. In presence of dynamical coupling, a subsystem’s behavior may be influenced by the value of the state and/or input of some other subsystems. Output coupling is a particular case of state coupling. This class of coupling can be described as in (2)– (3). When the system is characterized by constraints involving multiple subsystems, the coupling is seen through the constraints. This wide class includes constraints on input and/or state variables (such as physical limits of the plant or common-pool resources), and constraints not involving physical limits but related with the objective functions of the subsystems (for example, limited benefit within a free market with limited demand). Such problems may be analyzed under a zero-sum games perspective. Notice that constraint-coupled subsystems are not necessarily dynamically coupled. In this case, constraints (4c)–(4e) 9 take the form: hin(x, u)≤0, heq(x, u) = 0,(6) where hin and heq cannot be fully decomposed into independent equations hin i(xi, ui),heq i(xi, ui),i∈ N. Lastly, coupling can appear through the objective function: subsystems are part of a larger interacting environment and, therefore, it is likely that external variables exert some influence on their performance. These variables are usually related with the state of other subsystems, and often express economic indices. Notice that this category may overlap with the previous ones. Broadly speaking, a neighborhood designates a group of agents whose control problems show appreciable coupling that can be sensed within a limited time delay. Although the coupling structure may be fixed for a given system, in most cases the effect of coupling fluctuates. At this point, a natural question is what to do when coupling varies with time? Is it reasonable to consider time-varying neighborhoods? Some distributed control schemes in the literature have already moved in this direction. The notion of cooperating sets is employed in [34], [40]. Within any given cooperating set, one subsystem at each time step locally computes optimal control actions for all the subsystems involved in the set, even if only the results relative to that subsystem are broadcast. The rationale behind computing such “hypothetical” nonlocal control inputs is to optimize the individual strategy considering what others may be able to achieve, so that cooperation is indirectly promoted among the agents by favoring the best plans for everyone within the cooperating set. The composition of such sets is updated according to a graph representing the active coupling constraints. Interestingly, the results of [34] show that the optimal cooperating set is not necessarily restricted to directly coupled subsystems. In the distributed MPC scheme of [44], the cost incurred by each local controller is dynamically adjusted to fulfill minimum local requirements, on the basis of situational altruism criteria. On a similar line is the work of [45], where a flexible hierarchical MPC scheme is proposed for a hydro-power valley, where the priority of the agents in optimizing their control actions can be rearranged according to the different operational conditions. The work of [11] investigates the design of a hierarchical control scheme characterized by some flexibility over the employ of the data network through which the local control agents exchange information. The application on an irrigation canal is considered as a case study. The optimized design of a communication topology and its associated set of decentralized feedback control laws can be posed as a mixed-integer 16 take into account the most unfavorable externalities produced by any coalitional setup of the rest of agents. Particularly interesting when global objectives do not take over local ones, the third and final step consists in the (iii) distribution of the value of a coalition among its members. The payoff ϕiis the utility received by each agent i∈ C by the division of v(C); the vector of payoffs assigned to all the agents is referred to as the allocation. A variety of payoff rules have been proposed in the cooperative game theory, such as the core or the Shapley and Banzhaf values [56]. Of course, this third step is only possible if the real value v(C)associated with coalition C ⊆ N can be divided and transferred among its members (for instance, in the form of side-payments used to attract players). Let us define J(j) ias the quota relative to agent j∈ Ci in the coalition cost (9a). Notice that the solution of (9) does not imply any relation between the cost J∗ jachievable independently by any j∈ Ci, and the cost J∗(j) iincurred through its participation in the coalition. Provided a surplus is available, there exists a payoff assignment function such that J∗(j) i≤J∗ jis fulfilled for all j∈ Ci. As first pointed out in [57], the payoff allocation concepts developed for canonical games do not admit a straightforward implementation in presence of a coalitional structure different from the grand coalition. In the same work, the definitions of the core, the Shapley value and the nucleolus were extended to static coalition formation games, by redefining the group rationality concept with that analogous of relative efficiency. However, the results clearly showed that the complexity of the problem grows noticeably when dynamic coalition formation is considered, especially when the solution has to be computed in a distributed manner [49]. This fact motivates the application-oriented solutions found in the recent literature, such as [52], [58]. The essential steps of a coalitional control algorithm are summarized in Table I. To handle the combinatorial explosion problem, Steps 3a and 3b can be executed at a lower rate as in [10]. Prior to the application of any change in the topology, theoretical properties such as stability or robustness can be checked [59], [60]. Once the structure of the coalitions is defined, the local controllers exchange information with the members of their corresponding coalitions to calculate the control actions using a distributed control scheme [43]. III. CHALLENGES FOR COALITIONAL CONTROL As the architecture of multiagent systems becomes more complex—featuring reconfigurable topologies [10], [11], [61], [62], misaligned individual interests [13], [16], [63], [64], human-in-the-loop control [65], plug and play capabilities [59]—the relationship between possible restrictions on the global 17 TABLE I STEPS OF A COALITIONAL CONTROL ALGORITHM. Step Level Function 1 Local Measure the state xi(k). 2 Local Evaluate performance Ji. 3a (Top-down)Global Supervisory layer gathers information of the controllers’ performance and evaluates alternative network topologies. Changes in the topology are allowed only if the theoretical properties of interest (for example, stability) are retained after the switching. 3a (Bottom-up) Coalition + neighbors Collect information of the controllers involved in the coalition and its neighbors and decide whether to enable or disable links. Changes are allowed only if properties of interest are retained. 3b Coalition Update the information about the structure and members of the coalition. 4 Coalition Exchange information with the rest of the members of the coalition and calculate the control actions νi(k). No communication takes place with other coalitions. 5 Local Implement ui(k). Go to step 1. availability of information and the system performance is increasingly unclear [66]. Some of the challenges and open issues for coalitional control are summarized in the remainder. a) Criteria for coalition formation: How to determine the most appropriate coalitional structure for the overall system is a problem that shows fundamental analogies with model partitioning. Two different perspectives on coalition formation can be considered. The first is a top-down approach, where the global coalitional structure is optimized at a supervisory layer. Of fundamental importance is the criterion used to break the overall system into coalitions. Some possibilities have been already explored, such as the minimization of an index that combines optimal control performance and communication costs [11], or an H∞ robustness index—reflecting model-plant mismatches due to nonlinearities or inaccurate identification—as proposed in [22]. To address individual rationality as well, the second consists of a bottom-up approach: here the formation of coalitions is produced as the outcome of an autonomous bargaining procedure. While the first approach may offer some advantages in view of global system stability guarantees, the second appeals as a more appropriate scheme for real-world (plug and play) applications. b) Information exchange: Information plays a fundamental role in the coordination of multiagent systems. Either attained through direct sensing or communication, information is the basis for the local decisions of agents and, as such, decisive to the emergent global behavior [66]. Despite the huge effort dedicated to the development of distributed controllers for large-scale systems, communication has been mainly 18 studied from a transmission perspective, covering issues such as bandwith limitations, data loss, or the effects of noisy channels [67]. The very nature of the information exchange has received little attention. Indeed, in order to allow fundamental properties of centralized control, such as system-wide optimality and stability, the majority of the literature about distributed control overlooked privacy-related issues in order to focus on the overall system performance. Does providing agents with additional information always lead to improvements in the performance? On the other hand, can the excess of information be detrimental under some given system-wide perspectives? These questions have been partially addressed in works such as [35], [66]. In [66], a graph-coloring problem is used as a simple platform for studying the effect of information on multiagent collaboration. Results demonstrate that an increased amount of information is able to improve the efficiency of the Nash equilibria; on the other hand, it degrades the convergence rate of the distributed algorithm, where the agents seek to maximize their utility. c) Constraints on coalition formation: Bigger coalitions provide better control performance at the cost of increasing cooperation requirements. The enforcement of limitations on the composition of coalitions may be necessary. Some limitations can be of direct application (as for example size constraints), whereas others can implicitly derive by the penalization of specific metrics (use of the network infrastructure, identity of the participants, optimization variables, time expected to solve the control problem). In either case, this is a problem that requires attention. See [68] for a detailed analysis about constraints in the coalition formation process. d) Partial cooperation and information: When dealing with systems characterized by a strong heterogeneity, selfish interests may hinder the sharing of knowledge relevant for control purposes and cooperation as well. It is possible that only a subset of the control agents is willing to exchange information about their subsystems. It is therefore necessary to explore the performance bounds of the control loop possibly achievable with partial system information. Similar issues have been already studied in the field of Economics, using tools provided by the game theory. The authors of [35] extend a performance metric introduced by [69]—the competitive ratio—for the purpose of quantifying the distance from the optimum of the distributed solution of an LP problem when the information is locally segmented. From the coalitional control standpoint, it is arguably critical to characterize the improvement provided by a broader knowledge of the system, and promote the formation of coalitions accordingly. Notice that the previous question can be reversed: what is the minimal partition of the system model information necessary to guarantee some performance goal? In order to address this question, the authors of [35] point out that additional metrics need to be formulated in order to allow the characterization of the 19 different partitioning possibilities and the relative minimality notion. Again, possible candidate for such task are already available in the game theory literature about multi-agent decision making under partial information. e) Performance metrics: The development of the coalitional control field requires the definition of key performance indicators, especially those capable of capturing the trade-off between the information exchanged by the control agents and the performance of the system. For example, in [35] the connection between closed-loop performance and the amount of exchanged information is characterized for distributed linear quadratic controllers. Bounds are provided on the minimal information exchange needed to achieve an improvement over the performance of the best decentralized (communication-less) scheme. By addressing a networked resource allocation problem, the work of [70] identifies how a measure of locality in the individual control laws can be translated into a bound on the overall achievable efficiency. The relationship between the redundancy of information in the control laws implemented by agents and the achievable efficiency of the overall behavior is then characterized, providing bounds on the efficiency of the stable solutions. When full information regarding the mission space is available to the agents, the efficiency of the resulting stable solutions is guaranteed to lie within 50% of the optimal. However, as the reach of the information becomes more limited, the efficiency of the stable solutions may be as low as 1/N of the optimal, where Ndesignates the number of agents. f) Commitment between cooperating agents: Depending on the type of information exchanged and its purpose, it is possible to classify the cooperation in decreasing degrees of subsystems’ integration, ranging from full information (the coalition is formally equivalent to a unique subsystem) to binding to a prearranged output interval, or sharing planned trajectories, interaction models, objectives. Data for different degrees of cooperation can be computed offline and retrieved during system operation, in order to implement control schemes with varying requirements (from completely decentralized to centralized) on top of the same control infrastructure [9], [11]. This includes the possibility of reserving some backup strategies for the case of communication failure [61]. g) Theoretical guarantees: Classical theoretical properties such as stability or robustness for the closedloop system are more challenging in this context, specially in the case of bottom-up approaches, where little global information may be available at the individual subsystem level. h) Coalitional guarantees: Research on the novel properties specific of this type of control systems is needed. An example would be the degree of robustness of the coalitions with respect to the external incentives that their members may have for leaving. 20 i) Combinatorial explosion: Another issue is linked with the number of possible structures of the controller under a given data network. Approaching the problem of finding the optimal coalitional structure with exhaustive search can become unviable even for relatively small number of agents, due to the exponential growth of the number of possible coalitions. Constraints on the communication topology and the number of agents in a coalition might help to relieve this issue, since the realization of some configurations would be denied. Alternatively, limitations on the number of links that can be switched during a certain period would reduce the search space and ease the on-line implementation of coalitional control schemes. IV. AN ILLUSTRATIVE CASE Smart grids constitute an exhaustive example of complex large-scale system. They encompass advanced power, communication, control and computing technologies. In such heterogeneous domain the game theory finds natural applications, such as energy markets and dynamic pricing. The potential of the integration of game theoretical tools in the control of smart grids is discussed in [15]. In existing power systems, consumers are serviced by a main electricity grid that delivers the power over the transmission lines to a substation which, in turn, delivers the power over the low-voltage distribution network. One of the building blocks of the smart grid is the microgrid, a network of distributed energy sources located at the distribution side of the grid that can provide energy to a restricted geographical area. It may operate either together with the main grid or in an autonomous fashion. Power supply within microgrids is mainly based on small production from renewable sources or combined heat and electricity generation. Microgrids can count storage facilities (including electric vehicles’ batteries) and flexible demand among their resources as well. The service capacity of microgrids can be exploited to relieve the demand on the main grid. However, the intermittent generation coupled with the unpredictable nature of the demand implies that customers serviced by a microgrid may come to need extra energy from other sources. Of course, this extra energy can be provided by the main power grid. Nevertheless, the future smart grid is envisioned to encompass a large number of microgrid elements. Whenever some microgrids have an excess of production while others incur in a power shortage, a mutual exchange of energy can be beneficial for both parties, instead of relying on the main grid. The advantage of a local exchange is not limited to this: indeed, the energy transfer between nearby microgrids can significantly reduce the amount of power wasted for long-distance transmission over the distribution lines. 21 The aggregation of renewable-energy plants for their participation in the electricity market is analyzed as a canonical coalitional game in [50]. The main objective is the reduction of the variability in the production due to the unpredictability of renewable energy sources, exploiting the decorrelation of wind speeds at separated geographic locations. The focus is then on finding a fair mechanism for the allocation of the benefit among the producers, which can alleviate the risk associated with market participation. Intuitively, those individuals who contribute to a larger reduction in the production’s variability should receive a greater share of the benefit achieved through the cooperation. The problem is solved in [50] through an approximated computation of the nucleolus of the associated game, carried out as a single LP optimization that minimizes the worst-case dissatisfaction for all possible coalitions. In [71], a cooperative energy exchange game among micro-grids, with the objective of minimizing line power losses, is formulated. The formulation involves the solution of two subproblems: an auction matching game to find the pairs of suppliers and consumers minimizing power losses, and a coalition formation game to establish the coalitions. The rules proposed in [58], based on the Pareto order, are used to decide whether to form or break coalitions. The results show that such cooperative energy exchange mechanism has the advantage of improving the autonomy of microgrids with respect to the main grid, as well as reducing the losses over the distribution lines by promoting local energy trade among neighboring microgrids. The coalition structure can be adjusted to meet variations in the demand/supply. The coordination of demand and supply among communicating prosumers is addressed in [72] with the objective of alleviating the load on substations’ transformers. The focus is on the routing of information about individual demand/supply availability used for the estimation of the aggregate imbalance. The minimization of the energy imbalance of the cooperating nodes—which translates in less energy required from the main grid—is the objective of the control problem, whose distributed solution is coordinated by means of Lagrangian multipliers. Here we consider a set N={1, . . . , N}of consumer nodes equipped with domestic production systems such as solar panels and storage devices (see Figure 1). These nodes, situated within a small area as shown in Figure 2, pertain to the distribution layer of the grid, and are connected to the main grid through a substation. According to the coalitional control algorithm described in the remainder, the nodes can aggregate into coalitions. Inside each coalition, the power transfer is the result of an optimal control problem, formulated such as to minimize the cost of buying energy from the main grid for the entire coalition. Power losses over the distribution lines due to the energy transfer between members of the 22 Fig. 1. We consider a set of nodes connected to the main grid, equipped as well with local generation and storage devices. These prosumers can establish a local energy market by aggregating into coalitions, so as to minimize the cost of buying energy from the main grid. Power losses over the distribution lines are added to the incurred costs. As a result of the application of the coalitional control algorithm, the associations of nodes are reorganized according to variations in the local demand and supply, as well as to changes in the energy prices. The figure shows the energy balance at each node. coalition are added to the incurred costs (costs of power losses from the main grid are not considered separately from the spot price). It is interesting to see how, as a result of the application of the coalitional control algorithm, the associations of nodes are reorganized according to the variations in the local demand and production, as well as to the changes in the energy prices. Let us consider first the case in which the consumer nodes only rely on the main grid. For any node i∈ N, the energy exchange with the main grid is quantified by the variable xgrid i, representing the difference ¯ di−¯pibetween local demand and generation. Whenever xgrid i>0, node ibuys energy dgrid i=xgrid i from the grid—to fulfill the residual demand that cannot be matched by its local production—at the spot price cgrid i. In case of local energy surplus instead, that is xgrid i<0, node ican sell energy pgrid ito the grid at the feed-in tariff vgrid i. We assume that both generation and demand profiles are deterministic, known beforehand within a 5-hour horizon. We also assume that the tariff scheme implemented by utility companies is always such that cgrid i(k)> vgrid i(k). The local availability of a storage device allows the agent some flexibility when facing the spot prices imposed by the grid: the purchase and the sale of energy can be shifted in order to benefit of more convenient prices. At this point, the control problem individually addressed by each node would be (symbols are defined in Table II) 23 Fig. 2. Reciprocal location of the 8 prosumers considered in the example: the distance between rows of houses is 0.5 km. This distance is taken into account for the losses in the distribution lines resulting from the local energy transfer. TABLE II LIST OF SYMBOLS EMPLOYED IN THE DESCRIPTION OF THE EXAMPLE. Symbol Description Unit ¯ dDemand [kWh] ¯pGeneration output [kWh] dgrid Energy bought from the grid [kWh] pgrid Energy sold to the grid [kWh] cgrid Grid spot price [CU] vgrid Feed-in tariff [CU] dcoal Energy bought from the coalition [kWh] pcoal Energy sold to the coalition [kWh] ccoal Coalition spot price [CU] vcoal Coalition feed-in tariff [CU] sState of charge of the storage [kWh] ∆sIncrement in the storage level [kWh] min ui Np−1 X t=0 cgrid i(t|k)dgrid i(t|k)−vgrid i(t|k)pgrid i(t|k),(13a) s.t. si(t+ 1|k) = si(t|k)+∆si(t|k),(13b) ¯ di(t|k)+∆si(t|k) + pgrid i(t|k) = ¯pi(t|k) + dgrid i(t|k),(13c) dgrid i(t|k), pgrid i(t|k)≥0,(13d) si(t|k)∈[0, si,max], t = 0, . . . , Np,(13e) si(0|k) = si(k).(13f) 24 The manipulable inputs are ui≜[∆s⊺ i,d⊺ i,p⊺ i]⊺, where the bold notation indicates a vector containing the values along the prediction horizon. The monetary amount expressed by (13a) is the local balance of the energy exchange with the grid. Since the problem is formulated as a minimization, a positive balance indicates the expense required to buy energy from the grid, whereas a negative balance is the revenue from feeding energy to the grid. Constraints (13b)–(13f) represent, respectively, the dynamics of the storage, the energy balance at the node, the admitted values of inputs and storage level, and the initial state of charge. Problem (13) is now modified taking into account the possibility for two or more nodes to agree over mutual exchange of energy, establishing a common pool of resources. The local variable xcoal i≜ dcoal i−pcoal i, whose components are the demand of energy to the coalition and the energy transferred to its members, describes such exchange. The objective function of the coalitional problem involves a component for the minimization of the costs of energy exchange with the main grid, and an additional component concerning the minimization of power losses due to transfers between members of the coalition. min ν,Cρcoal X i,j∈C Np−1 X t=0 rij xcoal ij (t|k) 2!2 +X i∈C Np−1 X t=0 cgrid i(t|k)dgrid i(t|k)−vgrid i(t|k)pgrid i(t|k),(14a) s.t. si(t+ 1|k) = si(t|k)+∆si(t|k), i ∈ C (14b) ¯ di(t|k)+∆si(t|k) + pgrid i(t|k) + pcoal i(t|k) = ¯pi(t|k) + dgrid i(t|k) + dcoal i(t|k), i ∈ C,(14c) X i∈C pcoal i(t|k) = X i∈C dcoal i(t|k),(14d) dgrid i(t|k), pgrid i(t|k)≥0, i ∈ C,(14e) si(t|k)∈[0, si,max], t = 0, . . . , Np, i ∈ C (14f) si(0|k) = si(k), i ∈ C,(14g) C ⊆ N,(14h) where xcoal ij is the interchange of energy among the pair of agents i, j ∈ C, and rij is their relative Euclidean distance. The coalitional energy transfer dcoal i−pcoal iis taken into account in the balance constraints (14c). Constraint (14d) express the zero sum energy balance of the coalition. Given its complexity, problem (14) is approximated here by splitting it in two subproblems, each one 25 concerning one of the two components of the objective function. So, for any C ⊆ N, the first subproblem problem is formulated as: min ν J=X i∈C Np−1 X t=0 cgrid i(t|k)dgrid i(t|k)−vgrid i(t|k)pgrid i(t|k),(15) subject to (14b)–(14g). The resulting energy transfers are accompanied by an energy loss over the distribution lines: an approximation of these is added to Jas Jχ=ρcoalˆrX i∈C (dcoal i)2,(16) where ˆris a representative mean distance between every pair of coalition members, and dcoal iis the energy demanded to the coalition by agent i. Finally, the value v(C)of any given coalition C ⊆ N (recall that v: 2N7→ Ris the function mapping any possible coalition of agents into a real value) is defined as v(C)≜J+Jχ, where (16) represents the cost of forming the coalition. Now, the second subproblem consists in finding the partition P={C1,...,CNc}of Nsuch that the preference of every agent is satisfied according to the Pareto order. In order to do this, individual payoffs within each possible subset of agents S ⊆ N are first calculated as those defined by the Shapley value ϕS:R2|S| 7→ R|S|. Thus, for i∈ S: ϕS i(v) = X C⊆S\{i} |C|!(|S| − |C| − 1)! |S|![v(C ∪ {i})−v(C)] .(17) Note how the marginal contribution v(C ∪ {i})−v(C)is weighted by the probability for any agent iof joining the coalition C ⊆ S \ {i}, in case the agents form the coalition Sin a random order: in this way, the value assigned by Shapley’s criterion corresponds to the individual expected marginal contribution. Once this step is accomplished, a mapping Φ : N × 2N7→ Rof the individual payoff for each possible coalition the agent i∈ N can participate in is available. At time step k,Φ(i, C, k)defines the cost (or the benefit) incurred by agent iby participating in the coalition C ⊆ N. Note that such payoff will coincide with the cost of energy exchange with the main grid (Φ(i, {i}, k)≡cgrid i(k)dgrid i(k)−vgrid i(k)pgrid i(k)) in case the agent does not participate in any coalition. On the other hand, if the agent is member of a coalition, the equivalent price payed/earned on the coalition’s internal market can be derived as: ceq i(k)≜Φ(i, C, k) xgrid i(k) + xcoal i(k)CU kWh,(18) where CU stands for currency unit. Such mapping provides each agent with a preference order over the 32 [20] ——, “Hierarchical Distributed Control for Interconnected Systems,” in 13th IFAC Symposium on Large Scale Complex Systems: Theory and Applications, Shanghai, China, Jul. 2013, pp. 419–425. [21] Y. Samyudia, P. Lee, and I. Cameron, “A methodology for multi-unit control design,” Chemical Engineering Science, vol. 49, no. 23, pp. 3871 – 3882, 1994. [22] W. Al-Gherwi, H. Budman, and A. Elkamel, “Selection of control structure for distributed model predictive control in the presence of model errors,” Journal of Process Control, vol. 20, no. 3, pp. 270 – 284, 2010. [23] E. M. B. Aske, S. Strand, and S. Skogestad, “Coordinator MPC for maximizing plant throughput,” Computers & Chemical Engineering, vol. 32, pp. 195 – 204, Jan. 2008. [24] N. Motee and B. Sayyar-Rodsari, “Optimal partitioning in distributed model predictive control,” in Proceedings of the 2003 American Control Conference, vol. 6, 2003, pp. 5300–5305. [25] B. T. Stewart, A. N. Venkat, J. B. Rawlings, S. J. Wright, and G. Pannocchia, “Cooperative distributed model predictive control,” Systems and Control Letters, vol. 59, no. 8, pp. 460 – 469, 2010. [26] P. J. Antsaklis and A. N. Michel, Linear systems. New York: McGraw-Hill, 1997. [27] Y.-Y. Liu, J.-J. Slotine, and A.-L. Barabasi, “Controllability of complex networks,” Nature, vol. 473, pp. 167–173, 2011. [28] T. H. Summers and J. Lygeros, “Optimal sensor and actuator placement in complex dynamical networks,” in Proceedings of the 19th IFAC World Congress, Cape Town, South Africa, Aug. 2014, pp. 3784–3789. [29] C. Ocampo-Mart´ ınez, S. Bovo, and V. Puig, “Partitioning approach oriented to the decentralised predictive control of large-scale systems,” Journal of Process Control, vol. 21, no. 5, pp. 775 – 786, 2011. [Online]. Available: www.sciencedirect.com/science/article/pii/S0959152410002398 [30] L. Shapley, “A value for n-person games,” Annals of Math. Studies, vol. 28, pp. 307–317, 1953. [31] F. J. Muros, J. M. Maestre, E. Algaba, C. Ocampo-Mart´ ınez, and E. F. Camacho, “An application of the Shapley value to perform system partitioning,” in Proceedings of the 2015 American Control Conference, 2015, pp. 2143–2148. [32] I. Alvarado, D. Limon, D. Mu˜ noz de la Pe˜ na, J. M. Maestre, F. Valencia, H. Scheu, R. R. Negenborn, M. A. Ridao, B. De Schutter, J. Espinosa, and W. Marquardt, “A comparative analysis of distributed MPC techniques applied to the HD-MPC four tank benchmark,” Journal of Process Control, vol. 21, no. 5, pp. 800–815, 2011. [33] J. M. Maestre, M. A. Ridao, A. Kozma, C. Savorgnan, M. Diehl, M. D. Doan, A. Sadowska, T. Keviczky, B. De Schutter, H. Scheu, W. Marquardt, F. Valencia, and J. Espinosa, “A comparison of distributed mpc schemes on a hydro-power plant benchmark,” Optimal Control Applications and Methods, vol. 36, no. 3, pp. 306–332, 2015, oca.2154. [Online]. Available: http://dx.doi.org/10.1002/oca.2154 [34] P. Trodden and A. Richards, “Adaptive cooperation in robust distributed model predictive control,” in Control Applications, (CCA) Intelligent Control, (ISIC), 2009 IEEE, July 2009, pp. 896–901. 33 [35] C. Langbort and J. Delvenne, “Distributed design methods for linear quadratic control and their limitations,” Automatic Control, IEEE Transactions on, vol. 55, no. 9, pp. 2085–2093, Sep. 2010. [36] H. Ishii and R. Tempo, “The PageRank problem, multiagent consensus, and web aggregation: A systems and control viewpoint,” Control Systems, IEEE, vol. 34, no. 3, pp. 34–53, June 2014. [37] J. M. Maestre and H. Ishii, “A cooperative game theory approach to the pagerank problem,” in Proceedings of the 2016 American Control Conference, Boston, MA, USA, Jul. 2016. [38] Y. Sun and N. H. El-Farra, “Quasi-decentralized model-based networked control of process systems,” Computers and Chemical Engineering, vol. 32, pp. 2016–2029, 2008. [39] J. M. Maestre, D. Mu˜ noz de la Pe˜ na, E. F. Camacho, and T. Alamo, “Distributed model predictive control based on agent negotiation,” Journal of Process Control, vol. 21, no. 5, 2011. [40] P. Trodden and A. Richards, “Cooperative distributed MPC of linear systems with coupled constraints,” Automatica, vol. 49, no. 2, pp. 479 – 487, 2013. [Online]. Available: www.sciencedirect.com/science/article/pii/S0005109812005286 [41] R. R. Negenborn, P. J. van Overloop, T. Keviczky, and B. De Schutter, “Distributed model predictive control of irrigation canals,” Networks and Heterogeneus Media, vol. 4, pp. 359–380, 2009. [42] H. Scheu, J. Busch, and W. Marquardt, “Nonlinear distributed dynamic optimization based on first order sensitivities,” in Proceedings of the American Control Conference, Baltimore, Maryland, USA, July 2010. [43] J. M. Maestre and R. R. Negenborn, Distributed Model Predictive Control Made Easy, ser. Intelligent Systems, Control and Automation: Science and Engineering. Springer, 2014, vol. 69. [44] M. Lopes de Lima, E. Camponogara, D. Limon Marruedo, and D. Mu˜ noz de la Pena, “Distributed satisficing mpc,” Control Systems Technology, IEEE Transactions on, vol. 23, no. 1, pp. 305–312, Jan 2015. [45] A. N´ u˜ nez, C. Ocampo-Mart´ ınez, B. De Schutter, F. Valencia, J. L´ opez, and J. Espinosa, “A multiobjective-based switching topology for hierarchical model predictive control applied to a hydro-power valley,” in 3rd IFAC International Conference on Intelligent Control and Automation Science, Chengdu, China, Sep. 2013, pp. 529–534. [46] F. Valencia, “Game theory based distributed model predictive control: An approach to large-scale systems control,” Ph.D. dissertation, GAUNAL, Universidad Nacional de Colombia, September 2012. [Online]. Available: www.bdigital.unal.edu.co/8401/4/71367264.2012.pdf [47] M. Slikker and A. van den Nouweland, Social and Economics Networks in Cooperative Game Theory. Kluwer Academic Publishers, 2001. [48] T. Rahwan, T. Michalak, M. Wooldridge, and N. R. Jennings, “Anytime coalition structure generation in multi-agent systems with positive or negative externalities,” Artificial Intelligence, vol. 186, no. 0, pp. 95 – 122, 2012. [Online]. Available: www.sciencedirect.com/science/article/pii/S0004370212000288 34 [49] W. Saad, Z. Han, M. Debbah, A. Hjorungnes, and T. Basar, “Coalitional game theory for communication networks,” IEEE Signal Processing Magazine, Special Issue on Game Theory, vol. 26, no. 5, pp. 77 – 97, Sep. 2009. [50] E. Baeyens, E. Y. Bitar, P. P. Khargonekar, and K. Poolla, “Coalitional aggregation of wind power,” Power Systems, IEEE Transactions on, vol. 28, no. 4, pp. 3774–3784, Nov 2013. [51] F. Malandrino, C. Casetti, C.-F. Chiasserini, and M. Reineri, “A game-theory analysis of charging stations selection by EV drivers,” Performance Evaluation, vol. 83 - 84, pp. 16 – 31, Jan. 2015. [Online]. Available: www.sciencedirect.com/science/article/pii/S0166531614001163 [52] D. Ray, A game-theoretic perspective on coalition formation. Oxford University Press, 2007. [53] T. Sandholm, K. Larson, M. Andersson, O. Shehory, and F. Tohm´ e, “Coalition structure generation with worst case guarantees,” Artificial Intelligence, vol. 111, no. 1–2, pp. 209 – 238, 1999. [Online]. Available: www.sciencedirect.com/science/article/pii/S0004370299000363 [54] G. de O. Ramos, J. Rial, and A. Bazzan, “Self-adapting coalition formation among electric vehicles in smart grids,” in Self-Adaptive and Self-Organizing Systems (SASO), 2013 IEEE 7th International Conference on, Sept 2013, pp. 11–20. [55] G. Chalkiadakis, E. Elkind, and M. Wooldridge, Computational aspects of cooperative game theory. USA: Morgan & Claypool, 2012. [56] R. B. Myerson, Game Theory. Analysis of Conflict. Harvard University Press, 1997. [57] R. Aumann and J. Dreze, “Cooperative games with coalition structures,” International Journal of Game Theory, vol. 3, no. 4, pp. 217–237, 1975. [Online]. Available: dx.doi.org/10.1007/BF01766876 [58] K. R. Apt and A. Witzel, “A generic approach to coalition formation,” International Game Theory Review (IGTR), vol. 11, no. 3, pp. 347–367, 2009. [59] S. Riverso, M. Farina, and G. Ferrari-Trecate, “Plug-and-play model predictive control based on robust control invariant sets,” Automatica, vol. 50, no. 8, pp. 2179–2186, 2014. [60] P. A. Trodden, P. R. Baldivieso Monasterios, and J. M. Maestre, “Distributed MPC with minimization of mutual disturbance sets,” in Proceedings of the 2016 American Control Conference, Boston, MA, USA, Jul. 2016. [61] D. Gross, M. Jilg, and O. Stursberg, “Design of Distributed Controllers and Communication Topologies Considering Link Failures,” in 2013 European Control Conference, Zurich, Switzerland, Jul. 2013, pp. 3288 – 3294. [62] A. N´ u˜ nez, C. Ocampo-Mart´ ınez, J. M. Maestre, and B. De Schutter, “Time-varying scheme for noncentralized model predictive control of large-scale systems,” Mathematical Problems in Engineering, vol. 2015, 2015. [63] J. Barreiro-Gomez, C. Ocampo-Mart´ ınez, J. M. Maestre, and N. Quijano, “Multi-objective model-free control based on population dynamics and cooperative games,” in 54th Conference on Decision and Control, Osaka, Japan, Dec. 2015. 35 [64] A. Nedic and D. Bauso, “Dynamic coalitional TU games: Distributed bargaining among players’ neighbors,” Automatic Control, IEEE Transactions on, vol. 58, no. 6, pp. 1363–1376, June 2013. [65] J. M. Maestre, P. J. Van Overloop, M. Hashemy, A. Sadowska, and E. F. Camacho, “Human in the Loop Model Predictive Control: an Irrigation Canal Case Study,” in Proceedings of the 2014 Control and Decision Conference, Los Angeles, CA, USA, Dec. 2014. [66] J. R. Marden, B. Touri, R. Gopalakrishnan, and J. P. O’Brien, “Impact of information in a simple multiagent collaborative task,” in 2015 54th IEEE Conference on Decision and Control (CDC), Dec 2015, pp. 4543–4548. [67] G. N. Nair, F. Fagnani, S. Zampieri, and R. J. Evans, “Feedback control under data rate constraints: An overview,” Proceedings of the IEEE, vol. 95, no. 1, pp. 108–137, Jan 2007. [68] T. Rahwan, T. Michalak, E. Elkind, P. Faliszewski, J. Sroka, M. Wooldridge, and N. Jennings, “Constrained coalition formation,” in Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence. AAAI, August 2011, pp. 719–725. [Online]. Available: eprints.soton.ac.uk/272269/ [69] C. H. Papadimitriou and M. Yannakis, “Linear programming without the matrix,” in 25th ACM Symp. Theory Comput., 1993, pp. 121–127. [70] J. R. Marden, “The role of information in multiagent coordination,” in 2014 53rd IEEE Annual Conference on Decision and Control (CDC), Dec 2014, pp. 445–450. [71] W. Saad, Z. Han, and H. Poor, “Coalitional game theory for cooperative micro-grid distribution networks,” in Communications Workshops (ICC), 2011 IEEE International Conference on, June 2011, pp. 1–5. [72] G. K. Larsen, N. D. van Foreest, and J. M. Scherpen, “Power supply–demand balance in a smart grid: An information sharing model for a market mechanism,” Applied Mathematical Modelling, vol. 38, no. 13, pp. 3350 – 3360, 2014. Depósito de Investigación de la Universidad de Sevilla https://idus.us.es/ This is an Accepted Manuscript of an article published by IEEE in IEEE Control Systems Magazine, Vol. 37, Issue 1 on January 2017, available at: https://doi.org/10.1109/MCS.2016.2621465 © 2017 IEEE.