Exploration of spatial-temporal dynamic phenomena in a 32×32-cell stored program two-layer CNN universal machine chip prototype

1 Exploration of spatial-temporal dynamic phenomena in a 32×32-cells stored program 2-layer CNN Universal Machine Chip Prototype István Petrás1, Csaba Rekeczky1, Tamás Roska1, Ricardo Carmona2, Francisco Jiménez-Garrido2, Angel Rodríguez-Vázquez2 1 Analogical and Neural Computing Laboratory, Computer and Automation Research Institute, Hungarian Academy of Sciences, Kende u.11, Budapest, 1111 - Hungary 2Instituto de Microelectrónica de Sevilla-CNM-CSIC Avda. Reina Mercedes s/n, 41012 Sevilla (SPAIN) Tel. +34955056666 Fax. +34955056686 Abstract. This paper describes a full-custom mixed-signal chip that embeds digitally programmable analog parallel processing and distributed image memory on a common silicon substrate. The chip was designed and fabricated in a standard 0.5µm CMOS technology and contains approximately 500,000 transistors. It consists of 1024 processing units arranged into a 32×32 grid. Each processing element contains two coupled CNN cores, thus, constituting two parallel layers of 32×32 nodes. The functional features of the chip are in accordance with the 2nd Order Complex Cell CNN-UM architecture. It is composed of two CNN layers with programmable inter- and intra-layer connections between cells. Other features are: cellular, spatial-invariant array architecture; randomly selectable memory of instructions; random storage and retrieval of intermediate images. The chip is capable of completing algorithmic image processing tasks controlled by the user-selected stored instructions. The internal analog circuitry is designed to operate with 7-bits equivalent accuracy. The physical implementation of a CNN containing second order cells allows real time experiments of complex dynamics and active wave phenomena. Such well-known phenomena from the reaction-diffusion equations are traveling waves, autowaves, and spiral-waves. All of these active waves are demonstrated on-chip. Moreover this chip was specifically designed to be suitable for the computation of biologically inspired retina models. These computational experiments have been carried out in a development environment designed for testing and programming the analogic (analog-and-logic) programmable array processors. 1 Introduction Various phenomena of the world around us are the result of local interactions of particles. This is true for both micro and macro world, for molecules, cells and even for members of a population. Two large classes of such infinitely many phenomena are the pattern formation in biology (e.g. spots of animals or plants) or in chemistry and active wave propagation (e.g. action potential in the nervous system, propagation of cardiac muscle excitation, blood coagulation process). Fig. 1 shows examples of pattern formation and active wave phenomena. The expression “active wave” means that the wave propagation occurs in an “energetically active” medium, i.e. the energy conservation law does not hold because the media can inject energy into the system, therefore the amplitude and the waveform are preserved during the propagation. These are the most important differences when compared with classical waves. To make any reasonable complex pattern formation, we need the local activity of the cells [34]. Up to now the typical way to carry out experiments related to these phenomena was either laboratory work (e.g. with chemicals) or through time-consuming computer simulation. Other option is to build dedicated hardware circuitry. Although it is fast, its disadvantage is the lack of easy programmability. The Complex Cell CNN programmable array computer [5][11] is an extension of the CNN Universal Machine [1]-[4]. The multilayer or higher order elementary cells are or- 2 ganized into a 32×32 square grid. Each cell has second order dynamics and local interconnections to its neighbors. This structure is especially suitable for computing a certain set of ordinal differential equations. Simple PDEs can be transformed so that they can be easily programmed on the array computer. With the programmable Complex Cell CNN Universal Machine scientists are provided with a unique opportunity to study active-wave propagation and other reaction-diffusion differential equation based phenomena in a programmable manner. Exploiting the inherent programmability of the CNN architecture, complex wave-computing analogic algorithms can be designed. In this paper we show how to transform the mathematical model of the phenomena outlined above into a form suitable for parallel processing on the Complex Cell CNN chip and we present the first measurement results. Fig. 1: spiral- and autowaves and patterns measured on the CACE1k chip (image size: 32×32) The following section describes the mathematical model of the core of the complex cell chip. In Section 3 the prototype chip is introduced. Section 4 describes the relation between the reaction-diffusion equations and the structure of the complex cell chip. Finally, Section 5 contains the real-time chip measurements. We should emphasize that no simulation results are presented here. This chip was designed also for computing biologically inspired retina modeling. The implementation of the main effect of a retina model will be published in the near future elsewhere. 2 Computational architecture The architecture of the chip follows the design of the first order CNN-UM chip [10], but its first order cell core is replaced by a second order one [5]. With this change we are able to reproduce basic active wave phenomena (See Fig. 1). Fig. 2 shows the functional architecture of the 2nd order, two layer CACE1k core. in p ut la y er 1 la y er 2 a12 a21y2A1x1 A2x2x2 x1 τ ττ τ1 τ ττ τ2 z1 b2u2 z2 b1u1 τ ττ τ1 ≥ ≥≥ ≥ τ ττ τ2 out p ut Fig. 2: Functional diagram of the CACE1k chip The evolution law of the complex cell CNN is the following differential equation system: 3 () () 10,10,1,1 )()()( )( )()()( )( ,22,121,2,2,2 ,2 2 ,11,212,1,1,1 ,1 1 1 1 ≤≤≤≤≤≤≤≤ ++++−= ++++−= ∑ ∑ ∈ ∈ ijij ijijij Nkl klklij ij ijijij Nkl klklij ij xuMjMi zubtxatxAtxg dt tdx zubtxatxAtxg dt tdx τ τ (2.1) Where:      −<− ≤ > =∞→ 1 1 1 lim)( ,, ,, ,, , ijnijn ijnijn ijnijn m ijn xifmx xifx xifmx xg (2.2) Function g(.) regulates the state so that it stays within the +1..-1 interval. Variables u1, u2 are the independent inputs, b1, b2 are their weight factors respectively; z1,ij, z2,ij are space variant bias maps.. Variables x1, x2 denotes the state variables of the layers. Each xij corresponds to one cell; it is one pixel of a M by N image if we consider the system’s output as a picture. A1, A2 are the weights of the intra-layer couplings, a1, a2 are the inter-layer weights. Equation (2.1) utilizes the so-called full-range model (FSR)[7], where the voltage of the state variable is always the same as the output. 3 Chip description 3.1 Architecture of the chip and basic processor structure The prototype chip consists of an analog programmable array processor of 32×32 identical cells (Fig. 3), surrounded by the boundary conditions for the CNN dynamics. There is also an I/O interface, a timing and control unit and a program memory. The interface consists of a serializing-deserializing analog multiplexor. The program memory is composed of the analog program memory, and the logic program memory and the switch configuration registers. The analog instructions and reference signals need to be transmitted to every cell in the network in the form of analog voltages. Thus, a bank of D/A converters interfaces the analog program memory with the processing array. Distributing analog references across large distances within a chip is not a trivial task. Apart from the problems caused by electromagnetic interference, voltage drops in long metal lines carrying currents can be quite noticeable. Thus, in order to avoid this signal buffering and low-resistance paths must be provided. Finally, the timing unit is made of an internal clock/counter and a set of FSMs that generate the internal signals which control the processes of images up/downloading and program memory accesses. 4 Fig. 3: Architecture of the chip Every elementary processor of the Complex Cell CNN programmable array chip includes two coupled continuous-time CNN cores (Fig. 4(a)) belonging to each of the two different layers of the network. The synaptic connections between processing elements of the same or different layer are represented by arrows in the diagram. The basic processor also contains a programmable local logic unit (LLU) and local analog and logic memories (LAMs and LLMs) to store intermediate results. All the blocks in the cell communicate via an intra-cell data bus, which is multiplexed to the array I/O interface. Control bits and switch configuration are passed to the cell directly from the global programming unit. (a) (b) Fig. 4: Conceptual diagram of the (a) basic cell and (b) CNN layer node The internal structure of each CNN core is depicted in Fig. 4(b). Each core receives contributions from the rest of the processing nodes in the neighborhood which are summed and integrated in the state capacitor. The two layers differ in that the first layer has a scalable time constant, controlled by the appropriate binary code, while the second layer has a fixed time constant. The evolution of the state variable is also driven by self-feedback and by the feed forward action of the stored input and bias patterns. There is a voltage 5 limiter for implementing the FSR CNN model [7]. The state variable is transmitted in voltage form to the synaptic blocks, in the periphery of the cell, where weighted contributions to the neighbors’ are generated. There is also a current memory that will be employed for cancellation of the offset of the synaptic blocks. Initialization of the state, input and/or bias voltages is done through a mesh of multiplexing analog switches that connect to the cell’s internal data bus. 3.2 Prototype chip data The prototype chip has been designed and fabricated using 0.5µm single-poly triple-metal CMOS technology. Its dimensions are 9.27 x 8.45 mm2 (Fig. 5). The cell density achieved is 29.24cells/mm2. The time constant of the layers is around 100ns (unscaled). The programmable dynamics of the chip permit the observation of different phenomena of the type of propagation of waves, pattern generation, etc (See Section 5 for examples). Table 1 summarizes the most relevant data of the prototype chip. Technology 0.5 µ m CMOS 1-P 3-M Number of cells 32 x 32 Die area 9.27 x 8.45 mm2 Die area (w/o pads) 8.77 x 7.94 mm2 Array area 5.98 x 5.93 mm2 Package ceramic PGA – 100 Power supply voltage 3.3V Logic "0" / Logic "1" 0V / 3.3V Accuracy on the weights 8b Image samples resolution 7-8b I/O rates 10Ms/s CNN time constant below 100ns Table 1: Characteristic parameters of the CACE1k prototype chip Fig. 5: Microphotograph of the prototype chip 6 4 Complex cell CNN-UM chip as a “PDE solver” – typical waves 4.1 General PDE formulation Let us consider the following PDE formulation of a coupled vector valued nonlinear reaction-diffusion system (examples for nonlinear PDEs could be found in [26]- [32]): 11 11111012 22 22222021 10 10 1 1 (,) [ (,) ( (,))] ( (,)) ( (, )) ( (,)) (,) [ (,) ( (,))] ( (,)) ( (, )) ( (,)) (, ) (); (,) (| ( (, dxt divc xt grad xt xt xt xt dt dxt divc xt grad xt xt xt xt dt xt x c xt g grad xt φφαφβφγφ φφαφβφγφ φφ φ −=++ −=++ == && & & & & && & & & & &&& * 20 20 2 2 )) | ) (, ) (); (,) (| ( (,))|)yt x c yt g grad xt φφ φ == &&& * (4.1) where 12 [(,) (,)]xt xt φφ  can be interpreted as the time evolution of a vector valued image intensity ( 10 20 [() ()]xx φφ  are the initial images), the vector  x represents the spatial coordinates, the time variable t can also be interpreted as the scaling parameter and c is the conductance parameter in the diffusion term. The right hand side of the equations consists of three reaction terms: (i) α (.) is the “self-reacting term”; (ii) β (.) is a “spatial constraint” (calculated from the initial data); and (iii) γ (.) is the “cross-coupling” term. Some proposed functions for g(.) are: () () ggradIxtKK ggradIxtK K 1 22 2 11 0 100 =− > =+     >> +− exp ( ( , )) / , ((,)) / , ,  αα (4.2) Decoupling the two equations (γ1 = γ2 = 0) and choosing α (ξ) = - ξ the PDE formulation is given in the form of the so-called “biased” nonlinear anisotropic diffusion equation (Nordström [27]) an extended version of the Perona-Malik formulation [26]. Gerig et. al [29] first proposed the application of coupled nonlinear diffusion systems for vector-valued image processing that has been further studied by many others [32]. For our purposes – motivated by silicon implementation – we focus on a simplified version of (4.1) fixing the diffusion parameter to a constant value. 4.2 Spatial discretisation In two spatial dimensions and assuming c = const. (4.1) and (4.2) reduces to: 11 1 11 11012 22 2 22 22021 ( , , ) [ ( ( , , ))] ( ( , , )) ( ( , , )) ( ( , , )) ( , , ) [ ( ( , , ))] ( ( , , )) ( ( , , )) ( ( , , )) dxyt cdivgrad xyt xyt xyt xyt dt dxyt c divgrad xyt xyt xyt xyt dt φφαφβφγφ φφαφβφγφ −=++ −=++ (4.3) After spatial discretisation using the finite difference approach one obtains: 1, 1 1 2, 1 1, 1 1, 1, 1 1, 1 1, 1 1, 1 1 10, 2, 2 2 1, 2 2, 2 2, 2, 1 2, 1 2, 1 2, () ( ( )) ( ( )) ( ) ( ( ) ( ) ( ) ( )) ( ) 4 () (()) (()) () ( () () () 4 ij ij ij ij i j i j ij ij ij ij ij ij ij i j i j ij ij dt c t tct t ttt dt dt c ttct ttt dt φ γφαφ φ φφφφ βφ φγφαφ φ φφφφ −+ −+ −+ −+ =+−+ ++++ = + −+ +++ 1220, ()) ( ) ij t βφ + (4.4) Finally, with α (.)= α 0 and β (.) = β 0 a simplified form (with spatial symmetry and isotropy) of the CNN complex cell equation is derived: 7 1, 1 1 2, 1 1 1, 1, 1 1, 1 1, 1 1, 1 1, 2, 2 2 1, 2 2 2, 2, 1 2, 1 2, 1 2, 1 2, 1, 1 10, () () ( ) () ( () () () ()) 4 () () ( ) () ( () () () ()) 4 ij ij ij i j i j ij ij ij ij ij ij i j i j ij ij ij ij ij dt c tct t tttz dt dt c tct t tttz dt z φ γφ α φ φ φ φ φ φγφ α φ φ φ φ φ βφ −+ −+ −+ −+ =+− + ++++ =+− + ++++ =12,220,2 ;ij ij zz z βφ +=+ (4.5) Making φ ij explicitly depend on a state variable ξ ij such as φ ij= f( ξ ij) = sigm( ξ ij) leads a good approximation to the full-range CNN circuit model ([7], sigm(.) could be a piecewise linear or monotonic continuous smooth function playing the role of a signallimiter). The form of the corresponding 2nd order CNN template is as follows: 1 11 1 1 1 1 12 1 11 1 1 1 1 2 22 2 2 2 2 21 2 22 2 2 2 2 0/40 /4 /4 ; ; ; 1; 0 0/40 0/40 /4 /4 ; ; ; 1; 0 0/40 c Ac cc A B z c c Ac cc A B z c αγβτ αγβτ   =− = = = =      =− = = = =    (4. 6) Analysis: It should be noted that the diagonal terms could also be added resulting in a much better spatial isotropy during the diffusion process. Symmetry and spatial isotropy of these templates are due to the fact that there are no convection terms in the original formulation (4.1)–(4.2). With these modifications the sign and magnitude of the template entries in (4.6) could be changed in any spatial directions. The time constant is also fixed and equal for the two layers. Having two different time variables (say t'' ≠ t') in (4.1) leads to a "double time-scale" in the descritized system t'' / t' = τ 2 / τ 1 ≠ 1 which has practical relevance when looking at various second order models. 4.3 Typical wave classes There are a number of parameter settings of special interest that lead to very different qualitative behaviors in this symmetric second order system. We have examined and explored primarily the following simple cases (see also Sections 5.2): (i) trigger-wave generation (diffusion process in "saturation" at various speeds) c1 > 0 ; c2 > 0 ; α 1-(c1+1)>0; α 2-(c2+1)>0; γ 1 = γ 2 = 0; β 1 = β 2 = 0; τ 2 / τ 1 ≠ 1 (ii) traveling-wave, spiral-wave and auto-wave generation (spatially interacting trigger-waves) c1 > 0 ; c2 > 0 ; α 1-(c1+1)>0; α 2-(c2+1)>0; γ 1 ≠ 0; γ 2 ≠ 0; β 1 ≠ 0; β 2 = 0; τ 2 / τ 1 ≠ 1 (iii) retina effects (spatially interacting receptive fields) α 1 ≈ c1 >0; α 2 ≈ c2 > 0; γ 1 > 0; γ 2 < 0; β 1 ≠ 0; β 2 ≠ 0; τ 2 / τ 1 ≠ 1 5 Measurement and computation examples In the following we present chip measurement of the different active wave phenomena: trigger wave, traveling wave, autowave, and spatio-temporal edge detection. The movie files of the measurements are available at [33] 5.1 Stored programmability The CACE1k chip can be programmed using the so-called AMC (ANALOGIC MACHINE CODE) language. This is similar to the assembler language but it contains additional highlevel instructions, such as: image loading, frame grabber instructions, etc. and built in im- 8 age processing functions. In the Appendix is a simple AMC program which computes the snapshots of the autowave example (Subsection 5.2.4). The actual computation is performed in line 40. Before this line is the preparation for the computation: setting up the parameters. After the computation is done, the results are read out and displayed from the chip’s local analog memories. The elementary program of the chip is the template. It contains the weight factors of the coupling between the cells and weights for the input and the bias map. 16..1,1:τ 2121 1221 2 1,1 2 0,1 2 1,1 2 1,0 2 0,0 2 1,0 2 1,1 2 0,1 2 1,1 2 1 1,1 1 0,1 1 1,1 1 1,0 1 0,0 1 1,0 1 1,1 1 0,1 1 1,1 1 ==           =           = − − −−−− − − −−−− τ 21 τ:τ AA zzbb aa aaa aaa aaa aaa aaa aaa (5.1) The operation of the array computer is completely determined by the 25 template values, the initial states and boundary conditions. In (5.1) A1 and A2 matrices denotes the weights inter layer connections of the slower and the faster layer respectively. The strength of the influence of the second layer on the first is controlled by a21 and a12 stands for the reverse case. Symbols b1, b2, z1 and z2 are the weights of the independent inputs and the space variant bias maps. The ratio of the time constants of the two CNN layers are controlled by τ1, τ2 is fixed. An analogic algorithm is made up of combination of template executions, logic instructions and spatial arithmetic operations. The example shown in the Appendix is a very simple one, it contains only template execution operation. 5.2 Wave phenomenon This subsection contains the on-chip measurements of the active wave phenomena. 5.2.1 Trigger wave This example shows a very simple effect: an active (nonlinear) wave in the faster layer initiates a second wave in the slower layer through the positive interlayer coupling. The input is fed to the faster layer. Fig. 6 shows snapshots of the wave evolution. 1:165.16.030 03.3 6.08.06.0 8.06.08.0 6.08.06.0 111 15.11 111 2121 122121 =−==== ==           −=           = 21 τ:τ AA zzbb aa (5.2) Fig. 6: Trigger wave propagation in the faster and in the slower layer. The first film shows the output of the faster layer. 9 5.2.2 Erasure effect This example is an “enhancement” of the trigger wave example: a negative coupling is introduced from the slower layer back to the faster one. As a result, after a while, the trigger wave on the faster layer is erased by the trigger wave from the slower layer. As in the case of the trigger wave the input is given to the faster layer. Fig. 7 shows the snapshots of the wave evolution on the faster layer and Fig. 8 shows the evolution on the slower layer. 1:167.26.030 4.53.3 6.08.06.0 8.06.08.0 6.08.06.0 111 15.11 111 2121 122121 =−==== −==           −=           = 21 τ:τ AA zzbb aa (5.3) Fig. 7: Retina like wide-field erasure effect, faster layer. Fig. 8: Retina like wide-field erasure effect, slower layer. 5.2.3 Traveling wave The characteristic of this wave is that a single wave front travels across the active medium. The autowave is simply formed from the trigger wave template by adding a negative intra-layer coupling from the slower to the faster layer. In Fig. 9 observe the annihilation property of this wave type.