Spiking Neural P Systems and Modularization of Complex Networks from Cortical Neural Network to Social Networks

Spiking Neural P Systems and Modularization of Complex Networks from Cortical Neural Network to Social Networks Adam ObtuÃlowicz Institute of Mathematics, Polish Academy of Sciences ´ Sniadeckich 8, P.O.B. 21, 00-956 Warsaw, Poland [email protected] Summary. An idea of modularization of complex networks (from cortial neural net, Internet computer network, to market and social networks) is explained and some its topic motivations are presented. Then some known modularization algorithms and modular architectures (constructions) of complex networks are discussed in the context of possible applications of spiking neural P systems in order to improve these modularization algorithms and to analyze massively parallel processes in networks of modular architecture. 1 Introduction The aim of this paper is to discuss certain interconnections between spiking neural P systems [16], [28], and an idea of modularization of complex networks from cortial neural net, Internet computer network, to market and social networks, where the idea of modularization comprises modular architectures (structures or constructions) of those networks and modularization algorithms for retrieving modular structure of networks. The interconnections are understood here as proposals of application of spiking neural P systems to improve the modularization algorithms and to analyze massively parallel processes in networks of modular architecture or construction (emergence of new modules, etc., [9], [23]). In Section 2 we explain the idea of modularization of complex networks and some its topic motivations. In Section 3 we outline open problems concerning improvement of some modularization algorithms by application of spiking neural P systems and investigations of massively parallel processes in networks of modular construction by applying these P systems. 110 A. ObtuÃlowicz 2 Modularization of Complex Networks and Its Topic Motivations Amodularization1of a complex network or a graph is understood to be a decomposition or a partition of the underlying set of nodes of the network or the graph, respectively, into subsets called modules, often identified with subnetworks determined by these subsets and treated as autonomous processing units in cooperation with other units (on a higher level if abstraction). A collection of modules of a given network or a graph can be also a subject of modularization, i.e. a decomposition into subcollections of modules, etc., where the resulting subcollections of modules are called higher level modules. There are many reasons, motivations, and practical applications of modularization and we outline here some topics: 1) cortial neural network is modularized from anatomical, physiological, and scale (or magnitude) reasons, see, e.g., [24] or [27] for more references, into •cortial minicolumns which are modules consisting of neurons, •cortial hypercolumns which are some sets of minicolumns, •cortial areas which are some sets of hypercolumns, where cortial hypercolumns and areas are higher level modules, 2) natural self-modularization of cortial neural network into neuronal groups during evolution process described by M. G. Edelman’s Theory of Neuronal Group Selection (Neuronal Darwinism), see [18], [17] for a spiking neural network version, 3) modularization of cortial neural network into assemblies of neurons appears useful for neuronal representation of cognitive functions and processes because: •a single neuron behavior is less certain or more noisy than a behavior of an assembly of neurons, •the number of synaptic connections of a single neuron with other neurons is smaller than that of an assembly of neurons with other assemblies of neurons, •according to M. Kaiser [20] hierarchical cluster (higher level module) architecture “may provide the structural basis for the stable and diverse functional patterns observed in cortial networks”, 4) emergence (or extraction) of community structures in social networks, biological networks, and Internet computer network is a modularization of these networks discussed by M. J. E. Newman, [6], [7], [26], [29], see also applications of similar modularization in city planning discussed by Ch. Alexander [2], 5) modularization of artificial cortial-like networks for image processing, e.g., regularization for improving segmentation, see J. A. Anderson and P. Sutton, cf. [21], applications of an idea of a Network of Networks (NoN), 6) modularization which gives rise to hierarchical and fractal graphs and networks, [25], [29], [30], [33], to compress the information contained in large complex networks. 1The term ‘modularization’ is used e.g., in [19]. Spiking Neural P Systems 111 The higher level modules and their motivation are also discussed in [10], [11], [12], [22], [32], [31]. 3 Modularization Algorithms and Modular Architectures In the cases 2)–5) algorithms of modularization are considered, i.e. algorithms of distinguishing or extraction of modules, see e.g., [3], [18]. Thus one asks for those spiking neural P systems which could realize these algorithms through massive parallelism of computations providing •efficiency of computation, •those computation processes which could be close (from simulation reason) to real distributed processes of emergence of neuronal groups (see [18]) or community structures in social networks, where distributed processes of emergence of neuronal groups are massively parallel processes of simultaneous emergence of many those groups which are autonomous understood that, e.g., each group has at least one neuron which does not belong to other groups. These spiking neural P systems could give rise to constructing new brainbased devices (robots) similar to those which belong to the family Darwin due to M.G. Edelman [8]. The new brain-based devices could simulate maturing processes, where emergence of neuronal groups and groups of groups give rise to new cognitive functions. We show now an example of a link between modularization algorithms and spiking neural P systems which suggests the proposed above applications of these P systems. Namely, basing on the algorithm for identification of neuronal groups described in [18] we outline a method of extraction of a process of simultaneous emergence of many neuronal groups from a process generated by a spiking neural P system. Let S[C>S0be the next state relation determined by simultaneous application (in maximal parallelism mode) of the rules of a spiking neural P system, where S,S0are spike contents of neurons of the system and Cis the set of those synapses of the system which are activated to transform Sinto S0according to some maximal consistent set of the rules of the system during a unit of time. For a finite process generated by a spiking neural P system and represented by S0[C1>S1[C2>S2. . . Sn−1[Cn>Sn(n > 2) (1) we extract from it a process of simultaneous emergence of many neuronal groups which is represented by a sequence G1. . . Gi∗of sets of synapses of the system such that • G1is the set of maximal (with respect to inclusion relation of sets) subsets x of C1such that the synapses in xhave a common source neuron which is a counterpart of an anchor neuron, see the first step of the algorithm in [18], 112 A. ObtuÃlowicz •for i > 1 we define Gito be the set of maximal sets in the collection Ki=ny¯ ¯ ¯x(y=x∪ {s∈ Ci|the source neuron of synapse s is the target neuron of some synapse in x}for some x∈ Gi−1o until this collection is non-empty or, equivalently, until i=i∗, where i∗is the greatest number for which Ki∗is non-empty. The elements of Gi∗represent neural circuits which correspond to neuronal groups emerging simultaneously in the process represented by (1). Since the networks and their modularization discussed in 2)–4) are also approached by using probabilistic and statistical methods of clustering (see [1]) and by using random graphs (understood as in the B. Bolobas book [4]), it is worth to discuss a concept of a stochastic (or random) spiking neural P system whose synaptic connections form a random graph. Besides the new applications of spiking neural P systems suggested above one could ask for an application of M.-A. Gutti´erez-Naranjo and M. P´erez-Jim´enez models for Hebbian learning with spiking neural P systems [15] to explain in a new way •temporal correlation hypothesis of visual feature integration [14], also dealing with modularization, where modules are neural assemblies emerging in distributed processes like the processes of emergence of neuronal groups described above, for the connections of neuronal groups and binding (some generalization of feature integration), see [17]. •emergence of brain cognitive functions, where some modularizations of cortial neural network are considered, see [5], [13]. We propose to use higher-level networks with neighbourhood graphs, introduced in [27], as a precise description of results of some modularizations of networks and modular architectures including higher level modules. References 1. Albert, R., Barab´asi, A.-L., Statistical mechanics of complex networks, Reviews of Modern Physics 74 (2002), pp. 47–97. 2. Alexander, Ch., Notes on the Synthesis of Form, second ed., Harvard Univ. Press, Cambridge, MA, 1971. 3. Boccaletti, S., et al., Complex networks:Structure and dynamics, Physics Reports 424 (2006), pp. 175–308. 4. Bolobas, B., Random Graphs, Academic Press, 1985. 5. Changeux, J.-P., Dehaene, S., The neuronal workspace model:Conscious processing and learning in Learning Theory and Behaviour, in: Learning Memory. A Comprehensive Reference, ed. R. Menzel, vol. 1, 2008, pp. 729–758. Spiking Neural P Systems 113 6. Clauset, A., Moore, Ch., Newman, M. E. J., Structural inference of hierarchies in networks, in: Proceedings of 23rd International Conference on Machine Learning, Pittsburgh, PA, 2006. 7. Clauset, A., Moore, Ch., Newman, M. E. J., Hierarchical structure and prediction of missing links in networks, Nature 453 (2008), pp. 98–101. 8. Edelman, G. M., Learning in and from brain-based devices, Science 318 (16 November 2007), pp. 1103–1105. 9. Fernando, Ch., Karishma, K. K., Szathmary, E., Copying and evolution of neuronal topology, PLOS One 3 (November 2008), Issue 11, e3775. 10. Fingelkurts, An. A., Fingelkurts, Al. A., Operational architectonics of perception and cognition (a principle of self-organized metastable brain states), presented at the VI Parmenides Workshop of Institute of Medical Psychology of Munich University, Elba, Italy, April 5 to 10, 2003. 11. Fingelkurts, An. A., Fingelkurts, Al. A., Mapping of brain operational architectonics, in: Focus on Brain Mapping Research, ed. F. J. Chen, 2005, pp. 59–98. 12. Fingelkurts, An. A., Fingelkurts, Al. A., Kahkonen, S., New perspectives in pharmaco-electroencephalography, Progress in Neuro-Psychopharmacology & Biological Psychiatry 29 (2005), pp. 193–199. 13. Goldman-Rakic, P. S., Topography of cognition:parallel distributed networks in primate association cortex, Annual Rev. Neurosc. 11 (1988), pp. 137–156. 14. Gray, C. M., The temporal correlation hypothesis of visual feature integration still alive and well, Neuron 24 (1999), pp. 31–47. 15. Gutierez-Naranjo, M. A., Perez-Jimenez, M. J., A spiking neural P systems based model for Hebbian learning, in: Proceedings of 9th Workshop on Membrane Computing, Edinburgh, July 28 – July 31, 2008, ed. P. Frisco et al., Technical Report HW-MASC-TR-0061, School of Mathematical and Computer Sciences, Heriot–Watt University, Edinburgh, UK, 2008, pp. 189–207. 16. Ionescu, M., P˘aun, Gh., Yokomori, Y., Spiking neural P systems, Fund. Inform. 71 (2006), pp. 279–308. 17. Izhikevich, E. M., Polychronization:computation with spikes, Neuronal Computation 18 (2006), pp. 245–282. 18. Izhikevich, E. M., Gally, J. A., Edelman, G. M., Spike-timing dynamics of neuronal groups, Cerebral Cortex 14 (2004), pp. 933–944. 19. Johansson, Ch., Lasner, A., A hierarchical brain inspired computing systems, in: Proceedings NOLTA 2006, 11–14 September 2006, Bologna, Italy, pp. 599–602. 20. Kaiser, M., Brain architecture:a design for natural computation, Phil. Trans. R. Soc. A 365 (2007), pp. 3033–3045. 21. Ling Guan, Anderson, J. A., Sutton, J. P., A network of networks processing model for image regularization, IEEE Transactions on Neural Networks 8 (1997), No. 1, pp. 169–174. 22. Mason, R. D., Robertson, W., Mapping hierarchical neural networks to VLSI hardware, Neural Networks 8 (1995), pp. 905–913. 23. Moore, Ch., Ghoshal, G., Newman, M. E. J., Exact solution for models of evolving networks with addition and deletion of nodes, arXiv:cond-mat/0604069v1 [condmath.stat-mech], 4 April 2006. 24. Mountcastle, V. B., The columnar organization of the neocortex, Brain 120 (1997), pp. 701–722. 25. M¨uller-Linow, M., Hilgetag, C. C., H¨utt, M.-T., Organization of excitable dynamics in hierarchical biological networks, PLOS Computational Biology 4 (2008), Issue 9, e100190. 114 A. ObtuÃlowicz 26. Newman, M. E. J., The structure and function of complex networks, SIAM Review 45 (2003), pp. 167–256. 27. ObtuÃlowicz, A., On mathematical modeling of anatomical assembly, spatial features, and functional organization of cortex by application of hereditarily finite sets, in: Proceedings of 9th Workshop on Membrane Computing, Edinburgh, July 28 – July 31, 2008, ed. P. Frisco et al., Technical Report HW-MASC-TR-0061, School of Mathematical and Computer Sciences, Heriot–Watt University, Edinburgh, UK, 2008, pp. 371–382. 28. P˘aun, Gh., Perez-Jimenez, M. J., Spiking neural P systems. Recent results, research topics, in: 6th Brainstorming Week on Membrane Computing, Sevilla 2008, web page. 29. Ravasz, E., Barab´asi, A.-L., Hierarchical organization in complex networks, Physical Review E 67 (2003), 026112. 30. Sporns, O., Small–world connectivity, motif composition, and complexity of fractal neuronal connections, BioSystems 85 (2006), pp. 55–64. 31. Sporns, O., Chialvo, D. R., Kaiser, M., Hilgetag, C. C., Organization, development and function of complex brain networks, Trends in Cognitive Sciences 8 (2004), pp. 418–425. 32. Sporns, O., Tononi, G., Edelman, G. M., Theoretical neuroanatomy:relating anatomical and functional connectivity in graphs and cortial connection matrices, Cerebral Cortex 10 (2000), pp. 127–141. 33. Wuchty, S., Ravasz, E., Barab´asi, A.-L., The Architecture of Biological Networks.