Informational structures: A dynamical system approach for integrated information

RESEARCH ARTICLE Informational structures: A dynamical system approach for integrated information Francisco J. EstebanID 1☯ , Javier A. Galadı ´ID 2☯ , Jose ´A. LangaID 2☯ *, Jose ´R. PortilloID 3☯ , Fernando Soler-ToscanoID 4☯ 1Department of Experimental Biology, University of Jae ´n, Jae ´n, Spain, 2Department of Differential Equations and Numerical Analysis, University of Seville, Seville, Spain, 3Department of Applied Mathematic I, University of Seville, Seville, Spain, 4Department of Philosophy, Logic and Philosophy of Science, University of Seville, Seville, Spain ☯These authors contributed equally to this work. *[email protected] Abstract Integrated Information Theory (IIT) has become nowadays the most sensible general theory of consciousness. In addition to very important statements, it opens the door for an abstract (mathematical) formulation of the theory. Given a mechanism in a particular state, IIT identifies a conscious experience with a conceptual structure, an informational object which exists, is composed of identified parts, is informative, integrated and maximally irreducible. This paper introduces a space-time continuous version of the concept of integrated information. To this aim, a graph and a dynamical systems treatment is used to define, for a given mechanism in a state for which a dynamics is settled, an Informational Structure, which is associated to the global attractor at each time of the system. By definition, the informational structure determines all the past and future behavior of the system, possesses an informational nature and, moreover, enriches all the points of the phase space with cause-effect power by means of its associated Informational Field. A detailed description of its inner structure by invariants and connections between them allows to associate a transition probability matrix to each informational structure and to develop a measure for the level of integrated information of the system. Author summary In this paper we introduce a space-time continuous version for the level of integrated information of a network on which a dynamics is defined. The concept of integrated information comes from the IIT of consciousness. By a strict mathematical formulation, we complement the existing IIT theoretical framework from a dynamical systems perspective. In other words, we develop the bases for a continuous mathematical approach to IIT introducing a dynamical system as the driving rule of a given mechanism. We also introduce and define the concepts of Informational Structure and Informational Field as the complex network with the power to ascertain the dynamics (past and future scenarios) of the studied phenomena. The detailed description of an informational structure is showing PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 1 / 33 a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 OPEN ACCESS Citation: Esteban FJ, Galadı ´JA, Langa JA, Portillo JR, Soler-Toscano F (2018) Informational structures: A dynamical system approach for integrated information. PLoS Comput Biol 14(9): e1006154. https://doi.org/10.1371/journal. pcbi.1006154 Editor: Viktor K Jirsa, Institut de Neurosciences des Systèmes, FRANCE Received: October 19, 2017 Accepted: April 25, 2018 Published: September 13, 2018 Copyright: ©2018 Esteban et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: All code developed for the implementation of the results and the generation of the technical figures can be found from the Open Science Framework (https://osf.io/ 5tajz/). Funding: All the authors were supported by Ministerio de Economı ´a, Industria y Competitividad Proyecto Explora Ciencia MTM2014-61312-EXP. JAL was also partially supported by Junta de Andalucı ´a under Proyecto de Excelencia FQM-1492 and FEDER Ministerio de Economı ´a, Industria y the cause-effect power of a mechanism in a state and thus, a characterization of the quantity and quality of information, and the way this is integrated. We firstly introduce how network patterns arise from dynamic phenomena on networks, leading to the concept of informational structure. Then, we formally introduce the mathematical objects supporting the theory, from graphs to informational structures, throughout the integration of dynamics on graphs with a global model of differential equations. After this, we formally present some of the IIT’s postulates associated to a given mechanism. Finally, we provide the quantitative and qualitative characterization of the integrated information, and how it depends on the geometry of the mechanism. Introduction Dynamical Systems and Graph Theory are naturally coupled since any real phenomenon is usually described as a complex graph in which the evolution of time produces changes in specific measures on nodes or links among them [1,2]. In this work, the starting point is any structural network, including a parcelling of the brain, possessing an intrinsic dynamics. For brain dynamics, the collective behavior of a group of neurons can be represented as a node with a particular dynamics along time [3–6]. In general, the mathematical way to describe and characterize dynamics is by (ordinary or partial) differential equations (continuous time) [7] or difference equations (discrete time) [8]. Global models on brain dynamics are grounded on anatomical structural networks built under parcelling of the brain surface [9–11]. Indeed, they are based on systems of differential equations described on complex networks, which may include noise, delays, and time-dependent coefficients. Thus, the designed dynamical system models the activity of nodes connected to each other by a given adjacency matrix. A global dynamics emerges through simulated dynamics at each node, which is coupled to others as detailed in the anatomical structural network (see, for instance, [12] for the structural networks on primate connectivity). Then, an empirical functional network and a simulated functional network emerge by correlation or synchronization of data on the structural network [4, 13,14], showing a similar behavior and topology after a proper fitting of the parameters in the differential equations associated to the dynamics. We take advantage of this approach to apply some of the main results on the modern theory of dynamical systems showing that, given a dynamics on a network, there exists an object, the global attractor [15–18], determining all the asymptotic behaviour of each state of the network. The attractor exists and its nature is essentially informational, as it possesses the power to produce a curvature of the phase space enriching every point with the information on its possible past and future dynamics. The structure of the global attractors (or attracting complex networks [19,20]), described as composed structures by invariants and connections, naturally shows that its information is structured, composed by different parts, and can be unreachable from the study of the information of its parts, so allowing for a definition of integrated information. Integrated Information Theory (IIT, [21]), created by G. Tononi [22–24] starts with a phenomenological approach to the theory of consciousness. It assumes that consciousness exists and tries to describe it by defining the axioms that it satisfies. Having the axioms on hand, they serve to introduce the postulates that every physical mechanism has to obey in order to produce a conscious experience. This fact opens the door to the possibility of the mathematization of the theory by defining and describing postulates on concrete networks where a dynamics can be settled. It is then possible to define the appropriate structured Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 2 / 33 Competitividad grant MTM2015-63723-P. FJE was also supported by ACCIO ´N 1 PAIUJA 2017 2018: EI_CTS02_2017 (ref: 06.26.06.20.7A). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing interests: The authors have declared that no competing interests exist. dynamics which is supposed to explain a conscious experience by preserving its axioms. The IIT approach allows to represent a conscious experience and even to measure it quantitatively and qualitatively by the so called integrated information F max , which, at the same time indicates that, at the base of consciousness, there are essentially phenomena of causal processes of integrated information nature [25]. This fact links IIT to Information Theory and the Theory of Causality [26]. On the mathematical level, IIT approach is based on graphs consisting of logic gates and transition probabilities describing causality of consecutive discrete states on those graphs [21]. In this paper we present a continuous-time version (see Section Results for a formal description) related to IIT based on the theory of dynamical systems. IIT bases any particular experience on a mechanism, defined in a particular state, which possesses a well defined causeeffect power. Our starting point is given by a graph describing a mechanism and, thus, a graph is first defined. However, we focus our study on the network patterns arising from dynamical phenomena. So, a dynamical system and the associated mathematical objects (global attractor, equilibrium points, unstable invariant sets) have to be also defined. As a novelty in dynamical systems theory, the global attractor (which, for the gradient case consists of the equilibria and the heteroclinic connections between them [15,17,27]) is redefined as an object of informational nature, an Informational Structure (IS). An IS is a flow-invariant object of the phase space described by a set of selected invariant global solutions of the associated dynamical system, such as stationary points (equilibria) and connecting orbits among them (Fig 1). This set of invariants inside the IS creates a new structure, a new complex network with the power to ascertain the dynamics (past and future scenarios) of natural phenomena. Every IS posseses an associated Informational Field (IF), globally described from the attraction and repulsion rates on the nodes of the IS. We are able to translate the energy landscape caused by the IS and the IF into a transition probability matrix (TPM) to pass from one state to another within the system (see Section Results). Thus, the level of information of a mechanism in a state is going to be given by the global amount of deformation of the phase space caused by the intrinsic power of the IS and IF. The geometrical characterization of ISs can provide both the quality of the related information and, in particular, the shape in which it is integrated in the whole system, allowing to measure the level of integrated information it contains. Thus, the quality of the information comes from the detailed study of informational structure, which now possesses an intrinsic dynamics and enjoys a continuous change. This structure depends on the parameters of the underlying equations and has the ability to possibly rapidly change in the response to the change of those parameters (see Section Materials and methods). From this continuous approach, we are able to introduce first definitions for postulates of existence, composition, information and integration for a mechanism in a state. There is still a gap to the more elaborated formal definitions from IIT 3.0 [21] (see Section Discussion), including the composition and exclusion postulates. However, our framework naturally leads to a study on the continuous dependence between the topology of the network and the level of integrated information for a given mechanism (see Section Results). Materials and methods Dynamics on graphs Many real phenomena can be described by a set of key nodes and their associated connections, building a (generically) complex network. In this way, we can always construct an application between a real situation and an abstract graph describing its essential skeleton. An undirected graph is an ordered pair G¼ ðV;EÞcomprising a non-empty set Vof vertices (or nodes) together with a set Eof edges joining 2-element subsets of V. The order of a graph is given by Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 3 / 33 the number of nodes, and its size by the number of edges. A directed graph or digraph is a graph in which edges (named arcs) have orientations. We want to study the behaviour on networks of systems of evolutionary differential equations as du dt ¼Fðt;uÞ;ð1Þ where Fis a nonlinear map from ðt;uÞ 2 RRNto RN. For modeling purposes, we could also add, for instance, delays, stochastic terms, or to make solution u(t) also depend on a subset O of the three dimensional space, i.e. u(t,x), for x2OR3:Given an initial condition, suppose existence and uniqueness of solutions. If not, a multivalued approach could also be adapted. The phase space X(in our case X¼RN) represents the framework in which the dynamics described by a group of transformations S(t): X!Xis developed. Given a phase space X we define a dynamical system on Xas a family of non-linear operators fSðtÞgt2Rþ, SðtÞ:X!X u2X;SðtÞu2X which describes the dynamics of each element u2X. In particular, S(t)u 0 =u(t;u 0 ) is the solution of the differential Eq (1) at time twith initial condition u 0 . Fig 1. A. A mechanism (graph) of two nodes where a system of two differential equations given by (7) is defined, one for each node, using the given values for the αand γparameters. B. The Informational Structure (IS) is a new complex network made by four stationary points (equilibria) and directed links defined by global solutions. Each stationary point of the IS is a state associated to a subgraph of the original mechanism (non-null existing nodes are shown in black). The actual state of the mechanism corresponds to a state of the IS, highlighted in pink at the figure (the state where both u 1 and u 2 have a value greater than 0). C. (background) The associated directional field describing the tangent directions of trajectories inside the IS. The two straight lines (in orange and yellow) are the nullclines associated to the system and they intersect in three stationary points (except (0, 0)): one is a semitrivial stationary point in the Xaxis; the second is a semitrivial stationary point in the Yaxis. The last one is the stationary point with two strictly positive values. All of the stationary points constitute the nodes of the IS. Each stationary point is hyperbolic and locally creates a field of directions towards (stability) or from them (unstabilities). The informational field can be globally described by the sum of the stability and unstability influences of each continuous stationary solution passing when going to one node to other in the IS. D. The measurement of the amount of information to link any pair of nodes allows to define a Transition Probability Matrix (TPM) with the probability for each state of going to any other. States of the IS are denoted by the list of nodes having a value greater than 0. State (0) represents the node of the IS with both u 1 and u 2 equal to 0. https://doi.org/10.1371/journal.pcbi.1006154.g001 Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 4 / 33 The global attractor is the central concept in dynamical system theory, since it describes all the future scenarios of a dynamical system. It is defined as follows [15–18,28,29]: A set AXis a global attractor for {S(t): t0} if it is (i) compact, (ii) invariant under {S(t): t0}, i.e. SðtÞA¼Afor all t0, and (iii) attracts bounded subsets of Xunder {S(t): t0} for the Hausdorff semidistance; that is, for all BXbounded lim t!þ1 distHðSðtÞB;AÞ:¼lim t!þ1 sup b2Binf a2AðSðtÞb;aÞ ¼ 0: Observe that (ii) is showing a crucial property of an attractor, as supposes a set with a proper intrinsic dynamics. Moreover, (iii) points that this set is determining all the future dynamics on the phase space X. We say that u2Xis an equilibrium point (or stationary solution) for the semigroup S(t) if S(t)u=u, for all t0. A stationary point is a trivial case for a global solution associated to S(t), i.e., x:R!Xsuch that ξ(t+s) = S(t)ξ(s) for all s2R,t2Rþ. Stationary points are the minimal invariant objects inside a global attractor. Every invariant set is a subset of the global attractor [15]. Generically, connections among invariant sets in the attractor describe its structure [27,30]. To this aim we need the following definitions, which also allow us to define the behaviour towards the past in a global attractor. The unstable set of an invariant set Xis defined by WuðXÞ ¼ fz2X:there is a global solution x:R!XforSðtÞ satisfyingxð0Þ ¼ zand such that limt! 1distðxðtÞ;XÞ ¼ 0g: The stable set of an invariant set Xis defined by WsðXÞ ¼ fz2X:such that limt!þ1distðSðtÞz;XÞ ¼ 0g: We have to think in a global attractor as a set which does not depend on initial conditions, with an intrinsic proper dynamics, composed by a set of special solutions (global solutions), which are connecting particular invariants, so generating a complex directed graph. Moreover, the global attractor has the following properties [17,18]: 1. It is the maximal invariant set in the phase space. 2. It is the smallest closed attracting set. 3. It is made of bounded complete solutions, i.e., solutions that exists for all time t2R;and so giving information for the asymptotic past of the system. 4. Generically, its structure is described by invariant subsets and connecting global solutions among them [31,32]. The Fundamental Theorem of DynamicalSystems The Fundamental Theorem of Dynamical Systems [33] states that every dynamical system on a compact metric space X(the one defined on a global attractor, for instance) has a geometrical structure described by a (finite or countable) number (indexed by I) of sets {E i } i2I with an intrinsic recurrent dynamics and a gradient-like dynamics outside them. In other words, when we define a dynamical system on a graph, the attractor can be always described by a (finite or countable) number of invariants and connections between them. Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 5 / 33 Gradient attractors. We say [15,17,31] that a semigroup {S(t): t0} with a global attractor Aand a disjoint family of isolated invariant sets E= {E 1 ,,E n } is a gradient semigroup with respect to Eif there exists a continuous function V:X!Rsuch that (i) ½0;1Þ∍t7!VðSðtÞuÞ 2 Ris non-increasing for each u2X (ii) Vis constant in E i , for each 1 in; and (iii) V(S(t)u) = V(u) for all t0 if and only if u2[ n j¼1 Ei: In this case we call Va Lyapunov functional related to E. For gradient semigroups, the structure of the global attractor can be described as follows [15,27]: Let {S(t): t0} be a gradient semigroup with respect to the finite set E≔{E 1 ,E 2 ,,E n }. If {S(t): t0} has a global attractor A, then Acan be written as the union of the unstable manifolds related to each set in E, i.e, A¼[ n j¼1 WuðEjÞ:ð2Þ When E j are equilibria u j, the attractor is described as the union of the unstable manifolds associated to them A¼[ n j¼1 Wuðu jÞ: This description of a gradient system shows a geometrical picture of the global attractor, in which all the stationary points or isolated invariant sets (also defined as Morse sets, [31,32]) are ordered by connections related to its level of attraction [34] or stability. They conform a Morse decomposition of the global attractor [32,33,35–37]. When we refer to the global attractor for the dynamics on a graph, observe that each node given by a partially feasible equilibrium point in the attractor represents an attracting complex subgraph of the original one. Thus, the attractor can be understood as a new complex dynamical network describing all the possible feasible future networks [19,20]. In particular, it contains all the information related to future scenarios of the model. Energy levels. Any Morse decomposition E= {E 1 ,,E n } of a compact invariant set A leads to a partial order among the isolated invariant sets E i ; that is, we can define an order between two isolated invariant sets E i and E j if there is a chain of global solutions fx‘;1‘rg ð3Þ with lim t! 1distðx‘ðtÞ;E‘Þ ¼ 0 and lim t!1 distðx‘ðtÞ;E‘þ1Þ ¼ 0 1ℓr−1, with E 1 =E i and E r =E j . This implies that, given any dynamically gradient semigroup with respect to the disjoint family of isolated invariant sets E= {E 1 ,,E n }, there exists a partial order in E. In [34] (see also [38]) it is shown that there exists a Morse decomposition given by the so-called energy Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 6 / 33 levels N¼ fN1;N2;;Npg,pn. Each of the levels Ni, 1 ipis made of a finite union of the isolated invariant sets in Eand Nis totally ordered by the dynamics defined by (3). Indeed, the associated Lyapunov function has strictly decreasing values in any two different level-sets of Nand any two elements of Ewhich are contained in the same element of N(same energy level) are not connected. Attracting invariant networks. The existence of a global attractor implies that when we look at the evolution in time of each u i (t) we can see the “picture” in asymptotic time drawn by all ðu1ðtÞ;u2ðtÞ;. . . ;uNðtÞÞ 2 RN. Thus, this evolution is engaged with an invariant attracting network in which i) Each node represents a stationary solution (or, generically, a minimal invariant subset of the global attractor), which can be described as a binary vector representing the activation or inactivation of nodes, i.e., its points constitute a subgraph of the former graph representing the modelled system. ii) There exist oriented connections between the above subgraphs, so leading to a directed graph. iii) Each subgraph has totally determined stability properties; i.e., it is known not only which other nodes are connected to it or to which it is connected, but also, for instance, the velocity or the tendency for the attraction. iv) The whole invariant structure is determining the long time behaviour of all solutions of the system. In summary, behind a graph with dynamics (a mechanism in our conception) there exists a new complex network of connected subgraphs, governing all possible scenarios of the system. Informational structures: A formal definition The starting point in our approach is a system of connected elements where a dynamics is defined. This system is called a mechanism. Composition and exclusion postulates in IIT 3.0 allow to consider mechanisms given by any subset of the system. Here, we are only focusing in the mechanism given by the whole system. A global attracting network can be characterized by the amount of information it provides to the mechanism, since the nature of a global attractor is essentially informational. Indeed, the informational nature of the attractor is based on the following assertions: 1. They live in the phase space X, an abstract formulation for the description of the flow. 2. Their existence is not experimentally established, i.e., they are not necessarily related to physical experiments. They exist, associated and grounded to a particular mechanism, as a (small) compendium of “selected solutions”, which forms a complex structure and, moreover, determines the behaviour of all other solutions. The state of a mechanism is given by the state of its nodes. If those nodes take real values, the state is given by a vector of real numbers. When a dynamical system is defined for a mechanism in a state, it has cause-effect power, meaning that it conditions the possible past and future states of the mechanism. We can find the cause-effect power of a mechanism in a state by looking at its (local) attracting invariant sets. Typically, these are stationary points or periodic orbits [32,39–41], but it could also contain invariant sets with chaotic dynamics [42–44]. Following IIT, we will measure the amount of information on the structure made by these invariants, in the sense of its power to restrict the past and the future states. In IIT language, Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 7 / 33 this is intrinsic information: differences (state of a mechanism) that make a difference (restricted set of possible past and future states) within a system (see the IIT glossary in [21]). Now we can define an Informational Structure (IS). Suppose we have a complex graph G given by Nnodes and links among them. We denote by G i every subset of G:An informational structure is a complex graph I¼ fG1;;GMgwhich nodes are subgraphs G i of Gand links among them, with the following properties 1. Links between nodes are directed by their dynamics. 2. Each link has a defined weight. 3. Each link has a dynamical behaviour given by a model of differential equations. 4. Dynamics are defined by stability of nodes. An IS (see Fig 2) becomes a natural emerging object (of informational nature) from a given mechanism in a state with intrinsic cause-effect power to the past and the future. Moreover, some properties of ISs can be given: 1. ISs exist and are composed by a set of simple elements. They have cause-effect power, producing intrinsic information within the mechanism. 2. At any instant, an IS determines all the subset of past and futures states. 3. The quantity of information can be measured, for any given time, by the size of the IS at time t. Quality of the information is related to the shape of the IS. 4. Integration can be measured by partitions of the global graph. A global model. Lotka-Volterra models have been used to generate reproducible transient sequences in neural circuits [45–49]. In our case, for a general model for Nnodes, we define a system of Ndifferential equations given by: dui dt ¼uiaiþX N j¼1 gijuj !;i¼1; :::; N;ð4Þ where the matrix Γ= (γ ij )2R N×N is referred to the interaction-matrix. In matrix formulation, (4) reads as du dt ¼uaþGuð Þ;ð5Þ with A2RN2and a2RN. Given an initial data for (5), sufficient conditions for the existence and uniqueness of global solutions are well-known (see, for instance, [50,51]) The phase space for (5) is the positive orthant RN þ¼ fu¼ ðu1;;uNÞ 2 RN;ui0;i¼1;;Ng: This set of equations will define a dynamics on a structural graph with Nnodes, taking one equation for the description of the dynamics on each of the nodes. We model the dynamics on a given mechanism from Lotka-Volterra systems since they lead to a non-linear and nontrivial class of examples where the characterization of ISs and its dependence on parameters is, to some extent, very well understood [50,52]. Indeed, we can find, under some conditions on parameters, that there exits a finite number of equilibria (then, with trivial recurrent behaviour) and directed connections between them, generating an hierarchical organization by level Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 8 / 33 sets of equilibria ordered by connections in a gradient-like fashion [19,20]. With respect to IIT, each IS is associated to a mechanism in a state and gives it intrinsic information. As noted, we also want each Informational Structure to flow in time. Thus, we can consider the system of differential equations driven by time dependent sources α(t) (and/or γ ij (t)), given by: _ ui¼uiaiðtÞþX N j¼1 gijuj !;i¼1; :::; N:ð6Þ Informational fields. Note that an IS is not only a complex network of possible configurations of a graph. Moreover, it is well organized by their connections, showing the intrinsic Fig 2. A. Mechanism with five nodes u 1 ,,u 5 representing five interacting components represented as a graph G.B.αvalues and γvalues (position (i,j) in the matrix represents the influence of node u i over u j ) are the parameters defining the dynamics of the system. C. Nonlinear system of five equations to calculate the stationary points of the dynamical system. D. Informational Structure (IS) associated to the mechanism. Observe that the IS is a new complex network made by directed links, related to dynamics, of subgraphs G i of the original one G:Each node in the IS contains a stationary point of the dynamical system. Nodes of each G i with a value greater than 0 are shown in black. Grey nodes have value 0. Arrows in the IS relates to the cause-effect power of each state, going from one state to another. The relation induced by the arcs is transitive, but only the minimal arcs to understand the dynamics of the IS are represented. https://doi.org/10.1371/journal.pcbi.1006154.g002 Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 9 / 33 with j>i, we pass through the local dynamical action of all intermediate IS-nodes u j1;...uþ iþ1 between them. In this way, to measure the strength connection (to the past) from u jto u i; once we have changed signs, we sum the (exponential) attraction of u j, the (exponential) attraction from all intermediate nodes and finally the (exponential) repulsion dynamics from u i. The normalization of these values so that their sum is 1 allows a natural translation into probability distributions associated to each state, one for the past and another for the future. Thus, to build the TPM for the future, noted as TPM fut with entries TPM fut (i,j) we make pfðu i;u jÞ ¼ Sfutðu i;u jÞ PkiSfutðu i;u kÞ; with pfðu i;u jÞ ¼ TPMfutði;jÞ. Figs 7and 8contain the transition probability matrices for the informational structure of Fig 5 (right) by looking, respectively, at the future and the past. States (nodes) in the informational structure are represented (with parentheses as notation) by the list of elements (nodes of the mechanism, Fig 5 left) with a value greater than 0. The state with no elements greater than 0 is represented by (0). Recall that a stationary solution u jcan be reached from u iif and only if u iu j, in the sense that solutions are represented by the set of nodes with values greater than 0. Information. Inspired by IIT, the level of information of a mechanism in a state is compared both for the past (cause information) and the future (effect information). Fig 7. Transition probability matrix for the future of the IS in Fig 5. https://doi.org/10.1371/journal.pcbi.1006154.g007 Fig 8. Transition probability matrix for the past of the IS in Fig 5. https://doi.org/10.1371/journal.pcbi.1006154.g008 Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 16 / 33 In order to introduce the formal definitions, consider an informational structure IS with N nodes fu 1;...;u Ng. Also, # u i¼ jfu jj1jN;u iu jgj ^ # u i¼ jfu jj1jN;u ju igj # u iis the number of points in the informational structure accessible from u ito the future (supersets of u i) and ^ # u ithe number of accessible points from u ito the past (subsets of u i). Note that each node in the IS determines a subset of it. Cause information. Cause information measures the different probability distributions to the past obtained by considering the knowledge of just the structure (unconstrained past, p up ) and the structure in the current state (cause repertoire, p cr ). The probability distribution for unconstrained past, p up , is obtained by considering that any node u jcan be the actual one with the same probability. Formally, pupðu iÞ ¼ 1 NX N j¼1 ppðu iju jÞð12Þ where cause repertoire is directly defined by p p in the TPM (past) pcrðu iju jÞ ¼ ppðu iju jÞ;ð13Þ and ppðu i;u jÞ ¼ TPMpastði;jÞ: Cause information is the distance between both probability distributions. Earth mover’s distance EMD (or Wasserstein metric) [58] is used, so that ciðISÞ¼ EMDðpup;pcrÞ ð14Þ Effect information. Effect information is computed analogously to those presented for cause information. Formally, puf ðu iÞ ¼ 1 NX N j¼1 pfðu iju jÞð15Þ perðu iju jÞ ¼ pfðu iju jÞð16Þ eiðISÞ ¼ EMDðpuf ;perÞð17Þ Cause-effect information. Finally, the cause-effect information of the informational structure IS is the minimum of cause information and effect information, ceiðISÞ ¼ minfciðISÞ;eiðISÞg:ð18Þ Fig 9 shows the distributions involved in the calculation of cause-effect information in state (u 1 ,u 3 ) of the informational structure in Fig 5 (that is, the state in which only u 1 and u 3 have a value greater than 0). Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 17 / 33 Integration. Information is said to be integrated when it cannot be obtained by looking at the parts of the system but only at the system as a whole. We can measure the integration of an informational structure by trying to reconstruct it from all possible partitions. As we are considering mechanisms with a dynamical behaviour, when a partition P= Pof a mechanism with nodes {u 1 ,. . .,u N } is considered, the IS is computed by taking apart from the dynamics of each u i the nodes outside the same partition. Partitions are computed for a mechanism at a given state u¼ ðu 1;. . . ;u NÞ, so in the equation for u i , any node u j in the other partition is given the constant value u j. So, the IS for the partition P= Pis not computed by using (7) but u0 iðtÞ ¼ aiuiðtÞ uiðtÞ2þX uj2P uj6¼ui gijuiðtÞujðtÞþX uj2 P gijuiðtÞu jif ui2P aiuiðtÞ uiðtÞ2þX uj2 P uj6¼ui gijuiðtÞujðtÞþX uj2P gijuiðtÞu jif ui2 P 8 > > > > > < > > > > > : ð19Þ The informational structure of the partition contains the set of stable points of (19) together with the transition probability matrices for them, as defined above. To measure integration, all partitions with non empty Pand  Pare considered. Cause and effect repertoires are calculated for all of them, following (12) and (15). The partition with the cause repertoire closer to the cause repertoire of the IS is MIP cause , the minimum information partition with respect to the cause. The partition with the effect repertoire closer to that of IS is MIP effect .Fig 10 shows the minimum information partitions for the informational structure of Fig 5. The MIP cause is {u 1 ,u 3 }/{u 2 ,u 4 } (top) and MIP effect is {u 1 ,u 2 ,u 4 }/{u 3 } (bottom). Fig 9. Information in state (u 1 ,u 3 ). Cause and effect probabilities of both the structure (unconstrained repertoire given by (12)) and dynamics (cause repertoire from (13)) are compared by using EMD. Cause-effect information (cei) is the minimum of cause information (ci) and effect information (ei). https://doi.org/10.1371/journal.pcbi.1006154.g009 Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 18 / 33 Fig 10. Minimum information partition (MIP) for state (u 1 ,u 3 ). (A-C): Partition {u 1 ,u 3 }/{u 2 ,u 4 } of the system in Fig 5 with the cause repertoire closer to that of the complete system for state (u 1 ,u 3 ) (Fig 9). The ISs in Aand B correspond to the submechanisms {u 1 ,u 3 } and {u 2 ,u 4 }, respectively. Nodes in the IS Care those in the Cartesian product of Aand B. States of the ISs are highlighted in pink, observe that only u 1 and u 3 have values greater than 0 in the state of each IS, as the partition is for state (u 1 ,u 3 ). (D-F): Partition {u 1 ,u 2 ,u 4 }/{u 3 } with the effect repertoire closer to that of the whole system for state (u 1 ,u 3 ). https://doi.org/10.1371/journal.pcbi.1006154.g010 Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 19 / 33 Finally, integration ϕis given by the minimum of ϕ cause and ϕ effect , where ϕ cause is the distance between the cause repertoires of IS and MIP cause , and ϕ effect is the distance between the effect repertoires of IS and MIP effect . Both distances are calculated using EMD. Fig 11 shows the calculation of ϕfor the system of Fig 5 in state (u 1 ,u 3 ). Exclusion. Consciousness flows at a particular spatio-temporal grain, and this is the base for the exclusion postulate in IIT 3.0 [21], which we do not develop here (see Section Discussion). First, in a mechanism with many nodes, it may happen that not all of them are simultaneously contributing to the conscious experience. While ϕ= 0 for the whole mechanism, it may happen that for some submechanism ϕ>0. Indeed, it may be the case that several submechanisms of the same given mechanism are simultaneously integrating information at a given state. Moreover, consciousness flows in time. But the way it evolves is slower than neuronal firing. In our conception, integrated information is related to the values given by αand γ parameters in (7). Those parameters may be associated with intensity on connectivity (γ ij ) and neuromodulators (α i ) which change at a slower flow than neuronal firing [21]. In our approach, small changes in the parameters do not always imply a change in the informational structure, if parameters move within the same cone (see [59,60]). This fact might explain how a conscious experience may persist while neural activity is (chaotically) changing. However, when change moves the parameters to a different cone the IS suffers a bifurcation on its structure, so changing the level of integrated information. Topology of a mechanism and integrated information Although ISs and mechanisms possess quantitative and qualitative major differences on the structure and topology of both networks, it is clear that ISs possess an strongly dependence of their mechanisms’ topology. To show this dependence (but no determination) between mechanisms and associated ISs, we have tried to model the continuous evolution of integrated information for simplified mechanisms. This is probably one of the virtues of our continuous approach to integrated information. In particular, we consider the cases of totally disconnected mechanisms, lattice ones, the presence of a hub, and totally connected mechanisms, showing the key functions of the topology and strength of connections with respect to integrated information. To allow the comparison of the different topologies, the reference value for α i is 1.6 Fig 11. Calculation of ϕfor state (u 1 ,u 3 ). Cause and effect repertoires of the MIP (partitions for past and future) are compared to the repertoires of the informational structure of the complete system. The integration of the system is given by ϕ, the minimum of ϕ cause and ϕ effect .Left: The cause repertoire of the MIP cause (Fig 10, top) at state (u 1 ,u 3 ). It is compared by using EMD with the cause repertoire of the whole system (Fig 9, top left) resulting ϕ cause = 0.000208394. Right:ϕ effect is calculated in an analogous way. https://doi.org/10.1371/journal.pcbi.1006154.g011 Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 20 / 33 and for γ ij is 0.1. The behaviour of ϕ-cause and ϕ-effect when some of these parameters change is shown for the different mechanisms. Totally disconnected mechanisms. The way we define the cause-effect power of a disconnected mechanism always leads to null integration for the level of information. We highlight that integrated information is positive only if we connect the parts in the mechanism, also showing the dependence on the value of integrated information related to the continuous change on the strenght of the connecting parameter. Fig 12 shows an example of a disconnected mechanism. There are two groups of nodes {u 1 ,u 2 } and {u 3 ,u 4 ,u 5 }. Connections are only inside each group but not from one group to the other. The consequence is that the IS of the mechanism behave like the Cartesian product of the ISs of the partition (left and right ISs in the figure). We can set the connections between u 2 and u 3 , looking at the integration in different states of the mechanism (Fig 13). As expected, when γ 23 =γ 32 = 0 there is no integration. Cyclic mechanisms. Now we consider a mechanism of 5 nodes {u 1 ,u 2 ,u 3 ,u 4 ,u 5 } with all α i values equal to 1.6. Connections γ ij create a cycle so that all of them are 0 except {γ 12 ,γ 23 , γ 34 ,γ 45 ,γ 51 } that have the same value. Fig 14 shows the changing level of integration of the mechanism in states (u 1 ,u 2 ,u 3 ) and (u 1 ,u 3 ,u 5 ) as the strength of the connections grows up. If we look at min(ϕcause, ϕeffect), it is 0 when connections are 0, then it grows up to some maximum and then return to 0. Fig 12. A totally disconnected mechanism. Top: Disconnected mechanism. Left and right: Each of the partitions in the disconnected mechanism has an associated IS. The nodes of the IS of the whole mechanism (not represented in the figure) correspond to the Cartesian product of both smaller ISs. For example, the values for nodes u 2 and u 3 in the state (u 2 ,u 3 ) of the whole IS are the same that their values at states (u 2 ) and (u 3 ) of the left and right ISs, respectively (states highlighted in pink). Bottom:α i and γ ij values for the mechanism. https://doi.org/10.1371/journal.pcbi.1006154.g012 Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 21 / 33 Small world mechanisms: Presence of hubs. Fig 15 shows a mechanism where node u 3 has the role of a hub connecting all other nodes. Fig 16 shows the integration of the system in two different states as the value of α 3 changes. Observe that in this case ϕ-effect is more sensible than ϕ-cause to small changes on the strength of the hub connection. Totally connected mechanisms. We consider now a totally connected mechanism of 5 nodes with all γ ij values equal to 0.1. Fig 17 shows how integration changes as the α i values change. It is observed that, even for a totally connected mechanism, integrated information is a delicate measure which generically does not hold with positive values. In both states, when α i values are greater to 2, integration (ϕ-effect) is 0. Fig 13. Integration of the mechanism in Fig 12 as we increase the value of the connections between u 2 and u 3 .Values of ϕ-cause and ϕ-effect are shown. The resulting ϕ=min(ϕ cause ,ϕ effect ) at each point is the minimum of both values. Top: Integration in state (u 1 ,u 3 ). Bottom: Integration in state (u 2 ,u 3 ,u 4 ). https://doi.org/10.1371/journal.pcbi.1006154.g013 Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 22 / 33 Discussion Dynamics on complex networks characterized by informational structures The concept of informational structure is not only related to the understanding of brain processes and their functionality but, more broadly, as an inescapable tool when analyzing dynamics in complex networks [19,20]. For instance, in Theoretical Ecology and Economy related to the modeling of mutualistic systems [61,62], a very important subject is the study of the dependence between the topology of complex networks (lattice systems, unconnected systems, Fig 14. Integration in a cyclic mechanism. Top: Integration in state (u 1 ,u 2 ,u 3 ). Bottom: Integration in state (u 1 ,u 3 ,u 5 ). Note that the level of integrated information is not only a consequence of the topology of the mechanism (a lattice one in this case), but also on the strength of the connecting parameters and the particular state. In case of state (u 1 ,u 2 ,u 3 ) the three nodes with a positive value are consecutive and in (u 1 ,u 3 ,u 5 ) while u 1 and u 5 are linked in the cycle, u 3 is separated. https://doi.org/10.1371/journal.pcbi.1006154.g014 Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 23 / 33 Fig 15. A mechanism with a hub node u 3 .Values of parameters α i and γ ij are shown at the right, with α 3 =xacting as a variable. https://doi.org/10.1371/journal.pcbi.1006154.g015 Fig 16. Integration in different states of the hub mechanism of Fig 15 as α 3 increases. Top: Integration in state (u 1 ,u 3 ). Bottom: Integration in state (u 1 ,u 2 ,u 3 ). https://doi.org/10.1371/journal.pcbi.1006154.g016 Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 24 / 33 totally connected ones, random ones, “small world” networks) and the observed dynamics on their sets of solutions. The architecture of biodiversity [63,64] is thus referred as the way in which cooperative systems of plants-animals structure their connections in order to get a mechanism (complex graph) achieving optimal levels in robustness and life abundance. The nested organization of this kind of complex networks seems to play a key role for higher biodiversity. However, it is clear that the topology of these networks is not determining all the future dynamics [65], which seems also to be coupled to other inputs as modularity [66] or the strength of parameters [67]. This is also a very important task in Neuroscience [54,68]. Informational structures associated to phenomena described by dynamics on complex networks contain all the future options for the evolution of the phenomena, as they possess all the Fig 17. Integration in a totally connected mechanism. Parameters γ ij are all equal to 0.1. The evolution of ϕ-cause and ϕ-effect when α i values go from 0 to 3 is shown for two different states. https://doi.org/10.1371/journal.pcbi.1006154.g017 Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 25 / 33 46. Afraimovich VP, Zhigulin MI, Rabinovich MI. On the origin of reproducible sequential activity in neural circuits. Chaos. 2004; 14 (4): 1123–1129. https://doi.org/10.1063/1.1819625 PMID: 15568926 47. Afraimovich VS, Tristan I, Varona P, Rabinovich MI. Transient dynamics in complex systems: heteroclinic sequences with multidimensional unstable manifolds. Discontinuity, Nonlinearity and Complexity. 2013; 2(1): 21–41. 48. Muezzinoglu ME, Tristan I, Huerta R, Afraimovich VS, Rabinovich MI. Transient versus attractors in complex networks. International J. of Bifurcation and Chaos. 2010; 20 (6): 1653–1675. https://doi.org/ 10.1142/S0218127410026745 49. Rabinovich MI, Varona P, Tristan I, Afraimovich VS. (2014) Chunking dynamics: heteroclinics in mind. Frontiers in Computational Neuroscience. 2014; 8: 1–10. 50. Murray JD, Mathematical Biology. New York: Springer; 1993. 51. Takeuchi Y. Global Dynamical Properties of Lotka_Volterra Systems. Singapore: World Scientific Publishing Co. Pte. Ltd.; 1996. 52. Takeuchi Y, Adachi N. The existence of globally stable equilibria of ecosystems of a generalized Volterra type. J. Math. Biol. 1980; 10: 401–415. https://doi.org/10.1007/BF00276098 53. Deco G, Jirsa VK. Ongoing cortical activity at rest: criticality, multistability, and ghost attractors. J Neurosci. 2012; 32 (10): 3366–3375. https://doi.org/10.1523/JNEUROSCI.2523-11.2012 PMID: 22399758 54. Golos M, Jirsa VK, Dauce ´E. Multistability in Large Scale Models of Brain Activity. PLOS Computational Biology. 2015; 11. https://doi.org/10.1371/journal.pcbi.1004644 PMID: 26709852 55. Ma T, Wang S. Dynamic bifurcation and stability in the Rayleigh–Be ´nard convection. Commun. Math. Sci. 2004; 2: 159–193. https://doi.org/10.4310/CMS.2004.v2.n2.a2 56. Ma T, Wang S. Bifurcation Theory and Applications. World Scientific: Singapore; 2005. 57. Glendinning P. Stability, instability and chaos: an introduction to the theory of nonlinear differential equations. New York: Cambridge University Press; 1994. 58. Rubner Y, Tomasi C, Guibas L. The earth movers distance as a metric for image retrieval. Int J Comput Vis. 2000; 40(2): 99–121. https://doi.org/10.1023/A:1026543900054 59. Cottle RW, Pang J, Stone RE. The Linear Complementarity Problem. Boston: Academic Press, Inc.; 1992. 60. Murty KG, Yu F. Linear Complementarity, Linear and Nonlinear Programming. Internet Edition, 1997. Available from: http://www-personal.umich.edu/murty/books/linear-complementarity-webbook/lcpcomplete.pdf 61. Bascompte J, Jordano P. Mutualistic networks. Monographs in Population Biology 53. Princeton and Oxford: Princeton University Press; 2014. 62. Bastolla U, Fortuna MA, Pascual-Garcı ´a A, Ferrera A, Luque B, Bascompte J. The architecture of mutualistic networks minimizes competition and increases biodiversity. Nature. 2009; 458: 1018–1020. https://doi.org/10.1038/nature07950 PMID: 19396144 63. Bascompte J, Jordano P, Melia ´n CJ, Olesen JM. The nested assembly of plant-animal mutualistic networks. Proc. Natl Acad. Sci. USA. 2003; 100: 9383–9387. https://doi.org/10.1073/pnas.1633576100 PMID: 12881488 64. Bascompte J, Jordano P.The structure of plant-animal mutualistic networks: the architecture of biodiversity. Annu. Rev. Ecol. Evol. Syst. 2007; 38: 567–593. https://doi.org/10.1146/annurev.ecolsys.38. 091206.095818 65. Barrat A, Barthe ´lemy M, Vespignani A. Dynamical processes on complex networks. New York: Cambridge University Press; 2008. 66. Gilarranz LJ, Bronwyn R, Liña ´n-Cembrano G, Bascompte J, Gonzalez A. Effects of network modularity on the spread of perturbation impact in experimental metapopulation. Science. 2017; 357, issue 6347: 199–201. https://doi.org/10.1126/science.aal4122 PMID: 28706071 67. Rohr RP, Saavedra S, Bascompte J. On the structural stability of mutualistic systems. Science. 2014; 345: 416–425. https://doi.org/10.1126/science.1253497 68. Deco G, Sendem M, Jirsa VK. How anatomy shapes dynamics: a semi-analytical study of the brain at rest by a simple spin model. Frontiers in Computational Neuroscience. 2012; 6: 1–7. https://doi.org/10. 3389/fncom.2012.00068 69. Puu T. Attractors, Bifurcations & Chaos. Nonlinear phenomena in Economics. Berlin: Springer-Verlag, 2nd edition; 2003. 70. Arnold VI. Catastrophe Theory. Berlin: Springer-Verlag, 3rd edition; 1992. 71. Bassett DS, Bullmore E. Small-world brain networks. Neuroscientist. 2006; 12(6): 512–523. https://doi. org/10.1177/1073858406293182 PMID: 17079517 Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 32 / 33 72. Tognoli E, Kelso JA. The Metastable Brain. Neuron. 2014; 81(1): 35–48. https://doi.org/10.1016/j. neuron.2013.12.022 PMID: 24411730 73. Werner AG, Jirsa VK. Metastability, criticality and phase transitions in brain and its models. Biosystems. 2007; 90 (2): 496–508. https://doi.org/10.1016/j.biosystems.2006.12.001 PMID: 17316974 74. Hansen ECA, Battaglia D, Spiegler A, Deco G, Jirsa VK. Functional Connectivity Dynamics: Modeling the switching behaviour of the resting state. Neuroimage. 2014; 105: 525–535. https://doi.org/10.1016/ j.neuroimage.2014.11.001 PMID: 25462790 75. Friston K. The free-energy principle: A unified brain theory? Nat Rev Neurosci. 2010. 11:127–38. https://doi.org/10.1038/nrn2787 PMID: 20068583 76. Sengupta B, Tozzi A, Cooray GK, Douglas PK, Friston KJ (2016) Towards a neuronal Gauge Theory. PLoS Biology 14(3). https://doi.org/10.1371/journal.pbio.1002400 PMID: 26953636 77. Sengupta B, Friston KJ (2017) Approximate Bayesian inference as a gauge theory. arXiv:1705.06614v2 78. Pearl J. Causality: Models, Reasoning, and Inference. Cambridge: Cambridge University Press, 2nd edition; 2009. 79. Koch C, Massimini M, Boly M, Tononi G. Neural correlates of consciousness: progress and problems. Nat Rev Neurosci. 2016; 17(5): 307–321. https://doi.org/10.1038/nrn.2016.22 PMID: 27094080 80. Jirsa VK. The Virtual Brain. Available from: http://www.thevirtualbrain.org/tvb/zwei/home Informational structures: A dynamical system approach for integrated information PLOS Computational Biology | https://doi.org/10.1371/journal.pcbi.1006154 September 13, 2018 33 / 33