A stored program 2/sup nd/ order/3-layer complex cell CNN-UM

2000 SM IEEE International Workshop on Cellular Neural Networks and Their Applications Proceedings A STORED PROGRAM 2ND ORDERb-LAYER COMPLEX CELL CNN-ui"l Cs. Rekeczky', T. Serrano-Gotarredona., T. Roska', and A. Rodriguez-Vbzquez' 'Analogical and Neural Computing Laboratorj, Computer and Automation Institute, Hungarian Academy of Sciences, H-1 1 1 1 Budapest, Kende U. 13-17, Hungary 'Instituto de Microelectrhica de Sevilla, Centro de Nacional de Microeletrhica, Ed. Cica, Avda. Reina Mercedes s/n. Sevilla41012, Spain ABSTRACT: A stored program Fd order/3-layer complex cell Cellular Neural Network Universal Machine (CNN-UM) architecture is introduced. We discuss a number of phenomena that can be generated in this system by a single CNN transient. In particular, it is pointed out that by a proper combination of two dynamic layers some operations can be easily implemented that would require an approximation or iterative algorithmic solution relying on aJirst order CNN system. Thus, during these experiments. in a 2*d orderN-layer CNN new multi-layer fempfates have been identfied as ejficienr basic building blocks for various applicotion motivated analogic algorithms. Introduction In recent years, design and theoretical analysis of CNNs based on second order cells (or two-layer first order systems) have been discussed and motivated by number of researchers (e.g. [5]-[9]). In particular, auto-wave and spiral-wave phenomena to generate and control artificial locomotion [7]-[8], implementation of 2D spatio-temporal Gabor-type filters for motion analysis [5]-[6], and log-domain synthesis of reaction-diffusion CNNs for wave generation and pattern formation [9] have been thoroughly discussed. In this paper, a 2"d orded3-layer Cellular Neural Network ([I]-[4]) Universal Machine ([3]) architecture will be introduced and briefly discussed along with a broad set of phenomena that can be generated relying on this system. The architecture Following a successful line of design and fabrication of CNN-UM chips [ 161-[ 181 we propose an extension of these prototypes to a 2"d orded3-layer CNN-UM architecture as given in Fig. 1. We have replaced the first order base cell in [18] by a second order complex cell (mutually coupled first order cells) and introduced a built-in arithmetics. This makes it possible that besides the output of both first order cells (creating two dynamic layers) the spatio-temporal combination of these outputs (a static third layer) is also directly accessible for further processing. The number of programmable synapses is in the order of the previous design [18] and the complexity of the on-chip global programming unit is unchanged. Local analog memories (4) and logical memories (4) are shared and directly accessible for the two layers. Input c2 +,- +,- i Layer 3 v v g - 0 Figure 1 The core architecture of the 2ad order/ 3-layer CNN-UM Specification: - L1 and L2 consist of first order RC cells /full-range model/ - L1 and LZ have adjustable time constants: double time-scale property (T,/z* = O.l+10), [20] - the output characteristic for L1 and LZ is piecewise linear sigmoid-type - all templates contain linear synapses - neighborhood radii: r = 0: BI, A12, and A>,; FI: All and A22. 0-7803-6344-2/ool$10.00 02000 IEEE 213 - max. number of synapses is 13 or 21 on a 4-connected or 8-connected layers, respectively - LI and L2 can be initialized separately - LI and L2 have a switchable input through B, - the boundary condition can be constant or zero flux - L3 is directly connected to LI and L2 (the signal sign is reversible) (1+5+2+5=13; 1+9+2+9=21). System equations (fis sigmoid-type pwl function;f. is the built-in arithmetics): cI-G,ij =-xl.i/ /RI + C~ii.M~i,u +a21~2jj +bo~ij +zI, c2x2jj =-x2,ij lR2 + ~1.v =f(xl,ij) Y2,ij = f(X2,j ) M€Nl A22.kly2,kl +a12Yljj +=2 , kk NI x3,, =.fAxl,~42,v) z, = RICl, r2 = R2C2,1 Si < M, Chua - YangO SI xv (0) I < 1, Template format (with all tunable parameters): E, =bo; 21; 22; TI; z2 Relying on the above described architecture we have reproduced the basic wave phenomena (travellingwaves, auto-waves and spiral-waves) as a spatio-temporal interaction of trigger-waves, the simplest wave that can be generated in a first order system. Similarly, new pattem formation effects have been derived as the result of the interaction of pattems generated by two first order systems. It has also been shown, that combining trigger-waves, diffusion and pattem formation effects a number of useful image processing operations can be defined based on this system including edge enhancement, pattem formation, active contour detection, wave metric, edge and skeleton detection. Combination of these operations within the framework of the CNN prototyping system (CCPS, [19]) makes it possible to build and test novel analogic algorithms designed for specific applications. The Universe of Phenomena in a 2nd Order CNN Figure 2 The universe of phenomena that have been studied In the Znd order CNN system. In the sequel, these phenomena will be briefly described and illustrated showing output snapshots of our 2"d orded3-layer experimental CNN-UM system. All these phenomena are generated with a single CNN transient (thus called a spatio-temporal flow, see Fig. 2) by a proper combination of single-layer spatio-temporal dynamics. Remark: in the following experiments we focus on the 2"d order sub-system, thus the 3rd layer will be omitted. 214 Wave phenomena These experiments combine two trigger-waves travelling at different speeds (snapshots are shown from the output of the first layer). Trigger-waves - binary waves chan colliding fronts are merging Travelling-waves - a single travellin im ulse; colliding fronts annihilate. Bn Auto-waves - travelling impulses that are continuously generated from the source location; colliding fronts annihilate Spiral-waves - spiral-shaped travel1 conti colliding fronts annihilate Pattern Formation These experiments combine high-pass, low-pass and band-pass type instability to generate a stable pattem [lo]- [I I] (the stable output of the second layer is shown for two different parameter settings). 215 Image Processing trigger-waves, diffusion and various filtering effects. [12] (Observe that both layer outputs are halftoned images). The 2"' order sub-system makes it also possible to synthesize useful image processing operations by combining Harftoning - a gray-scale to binary image transformation that preserves the main features of the original image Original 1'' layer 2"' layer Smoothing and edge enhancement - a gray-scale to gray-scale image transformation that enhances the edges (see a comprehensive discussion on similar CNN based image processing methods in [13]). The architecture is neuromorphic, i.e. resembles the structure of the cone-horizontal system in the outer retina of the vertebrates where the horizontal layer is driven by the cones (Observe that the oitput of the first layer is a band-pass filtered while the output of the second layer is a low-pass filtered image). Original 1 layer 2"' layer Wave metric - a trigger-wave based method to measure the difference between objects [15] (Post processing: maximum detection on the output of the Znd layer). I II 1" object 2"' object ? L 1 * layer Active Contour Detection - a combined diffusion and trigger-wave based method for identifying the contour of a labeled region [I41 (Post processing: edge detection on the output of the 2"d layer). Edge and Skeleton Detection - morphological detection based on trigger-waves combined with high-pass effects (Remark the edge and the skeleton is obtained for different parameter settings). 216 I Original H Edge (1 ' layer) 2 keIeton(1' layer) Summary We have introduced and briefly discussed a 2"d order/ 3-layer CNN-UM architecture. It has been shown that a broad class of phenomena can be reproduced within the framework of this system. This motivates the synthesis of novel analogic CNN algorithms that can fully exploit the capabilities of this multi-layer CNN with stored programmibility. Details and chip design issues will be reported in forthcoming papers. References [I] L. 0. Chua and L. Yang, "Cellular Neural Networks: Theory and Applications", IEEE Trans. on Circuits and Systems, Vol. 35, pp. 1257-1290, Oct. 1988. [2] L. 0. Chua, and T. Roska, "The CNN Paradigm", IEEE Tray. on Circuils and Systems, Vol. 40, pp. 147-156, March 1993. [3] T. Roska and L. 0. Chua, "The CNN Universal Machine", IEEE Trans. on Circuifs and Systems, Vol. 40, pp. 163-173, March 1993. [4] F. S. Werblin, T. Roska, and L. 0. Chua, "The Analogic CNN Universal Machine as a Bionic Eye", Infernational Journal of Circuif TheoryandApplications, Vol. 23, pp. 541-569, 1995. [5] B. E. Shi, "2D Steerable and Scalable Cortical Filters", in Proc. of fhe Infernafional Conference on Microelectronics for Neural, Fuzzy and Bio-inspired Sysfems, pp. 232-239, Granada, Spain, 19999. [6] B. E. Shi, "Gabor-type Filtering in Space and Time with Cellular Neural Networks", IEEE Trans. on Circuits and Systems, Vol. 45, pp. 121-132, 1998. [7] P. Arena, S. Baglio, L. Foma, and G. Manganaro, "Self-Organization in a Two Layer CNN", IEEE Trans. on Circuits and Sysfems, Vol. 45, No. 2,pp. 157-162, 1998. [8] P. Arena, L. Fortuna, M. Branciforte, "Reaction-diffusion CNN Algorithms to Generate Artificial Locomotion", IEEE Trans. on Circuits and Sysfems, Vol. 46, No. 2, pp. 253-260, 1999. [9] T. Serrano-Gotarredona and A. Rodriguez-Vkquez, "Implementation of Complex Dynamics Reaction-diffusion CNNs", to appear in Journal of VLSI Signal Processing, 1999. [IO] P. Thiran, K. R. Crounse, L. 0. Chua, and M. Hasler, "Pattern Formation Properties of Autonomous Cellular Neural Networks", IEEE Trans. on Circuits and Systems, Vol. 42, pp. 757-774, 1995. [I I] K. R. Crounse, L. 0. Chua, P. Thiran, and G. Setti, "Characterization and Dynamics of Pattern Formation in Cellular Neural Networks", International Journal of Bi/urcafion and Chaos, Vol. 6, pp. 1703-1724, 1996. 1121 K. R. Crounse, T. Roska and L. 0. Chua, "Image Halftoning with Cellular Neural Networks", IEEE Trans. on Circuifs and Sysfems, Vol. 40, pp. 267-283, 1993. [I31 K. R. Crounse and L. 0. Chua, "Methods for Image Processing and Pattern Formation in Cellular Neural Networks", IEEE Trans. on CircuifsandSystems, Vol. 42, No. 10, pp. 583-601, 1995. [I41 9. Rekeczky and L. 0. Chua, "Computing with Front Propagation: Active Contour and Skeleton Models in Continuoustime CNN", Journal of VLSI Signal Processing Systems, Vol. 23, No. 213, pp. 373-402, November-December 1999. [I51 I. Szatm&ri, Cs. Rekeczky, and T. Roska, "A Nonlinear Wave Metric and its CNN Implementation for Object Classification", Journal of VLSI Signal Processing Sysfems, Vol. 23, No. 213, pp. 437-448, November-December 1999. [ 161 S. Espejo, R. Carmona, R. Domin&ez-Castro, and A. Rodrigliez-Vtkquez, "CNN Universal Chip in CMOS Technology", InfernafionalJoumal of Circuif TheoryandApplications, Vol. 24, pp. 93-1 11, 1996. [I71 R. Dominguez-Castro, S. Espejo, A. Rodriguez-Vazquez, R. Carmona, P. FBldesy, A. Zarhdy, P. Szolgay, T. Szidnyi, and T. Roska, "A 0.8 pm CMOS 2-D Programmable Mixed-Signal Focal-Plane Array Processor with On-Chip Binary Imaging and Instructions Storage", IEEEJ. SolidStafe Circuifs, pp. 103-1026, July, 1997. 1181 S. Espejo, R. Dominguez-Castro, G. Liiiln, A. Rodriguez-VBzquez, "A 64x64 CNN Universal Chip with Analog and Digital 110, in Proc. of Sfh Inf. ConJ on Elecfronics, Circuits and Sysfems ICECS'98, pp. 203-206, Lisbon, September 1998. [I91 T. Roska, P. Szolgay, A. Zdndy, P. Venetianer, A. Radvhyi, and T. Szildnyi, "On a CNN Chip-Prototyping System", in Proceedings of 3rdInternafional Workshop on Cellular Neural Nehvorks and Their Applications CNNA'94, pp. 375-381, Rome, 1994. [20] T. Kozek and T. Roska, "A double time-scale CNN for solving 2-D Navier-Stokes equations", International Journal of Circuit Theory and Applications, Special Issue on CNN 11: Part I, Vol. 24, pp. 49-56, 1996. 217