Exploration of spatial-temporal dynamic phenomena in a 32×32-cell stored program two-layer CNN universal machine chip prototype

1 Explo a ion o spa ial- empo al dynamic phenomena in a 32×32-cells s o ed p og am 2-laye CNN Uni e sal Machine Chip P o o ype Is án Pe ás1, Csaba Rekeczky1, Tamás Roska1, Rica do Ca mona2, F ancisco Jiménez-Ga ido2, Angel Rod íguez-Vázquez2 1 Analogical and Neu al Compu ing Labo a o y, Compu e and Au oma ion Resea ch Ins i u e, Hunga ian Academy o Sciences, Kende u.11, Budapes , 1111 - Hunga y 2Ins i u o de Mic oelec ónica de Se illa-CNM-CSIC A da. Reina Me cedes s/n, 41012 Se illa (SPAIN) Tel. +34955056666 Fax. +34955056686 Abs ac . This pape desc ibes a ull-cus om mixed-signal chip ha embeds digi ally p og ammable analog pa allel p ocessing and dis ibu ed image memo y on a common silicon subs a e. The chip was designed and ab ica ed in a s anda d 0.5µm CMOS echnology and con ains app oxima ely 500,000 ansis o s. I consis s o 1024 p ocessing uni s a anged in o a 32×32 g id. Each p ocessing elemen con ains wo cou- pled CNN co es, hus, cons i u ing wo pa allel laye s o 32×32 nodes. The unc ional ea u es o he chip a e in acco dance wi h he 2nd O de Complex Cell CNN-UM a chi ec u e. I is composed o wo CNN laye s wi h p og ammable in e - and in a-laye connec ions be ween cells. O he ea u es a e: cellula , spa- ial-in a ian a ay a chi ec u e; andomly selec able memo y o ins uc ions; andom s o age and e ie al o in e media e images. The chip is capable o comple ing algo i hmic image p ocessing asks con olled by he use -selec ed s o ed ins uc ions. The in e nal analog ci cui y is designed o ope a e wi h 7-bi s equi a- len accu acy. The physical implemen a ion o a CNN con aining second o de cells allows eal ime ex- pe imen s o complex dynamics and ac i e wa e phenomena. Such well-known phenomena om he eac- ion-di usion equa ions a e a eling wa es, au owa es, and spi al-wa es. All o hese ac i e wa es a e demons a ed on-chip. Mo eo e his chip was speci ically designed o be sui able o he compu a ion o biologically inspi ed e ina models. These compu a ional expe imen s ha e been ca ied ou in a de elop- men en i onmen designed o es ing and p og amming he analogic (analog-and-logic) p og ammable a ay p ocesso s. 1 In oduc ion Va ious phenomena o he wo ld a ound us a e he esul o local in e ac ions o pa i- cles. This is ue o bo h mic o and mac o wo ld, o molecules, cells and e en o membe s o a popula ion. Two la ge classes o such in ini ely many phenomena a e he pa e n o ma ion in biology (e.g. spo s o animals o plan s) o in chemis y and ac i e wa e p opaga ion (e.g. ac ion po en ial in he ne ous sys em, p opaga ion o ca diac muscle exci a ion, blood coagula ion p ocess). Fig. 1 shows examples o pa e n o ma- ion and ac i e wa e phenomena. The exp ession “ac i e wa e” means ha he wa e p opaga ion occu s in an “ene ge i- cally ac i e” medium, i.e. he ene gy conse a ion law does no hold because he media can injec ene gy in o he sys em, he e o e he ampli ude and he wa e o m a e p e- se ed du ing he p opaga ion. These a e he mos impo an di e ences when compa ed wi h classical wa es. To make any easonable complex pa e n o ma ion, we need he local ac i i y o he cells [34]. Up o now he ypical way o ca y ou expe imen s ela ed o hese phenomena was ei he labo a o y wo k (e.g. wi h chemicals) o h ough ime-consuming compu e simu- la ion. O he op ion is o build dedica ed ha dwa e ci cui y. Al hough i is as , i s disad- an age is he lack o easy p og ammabili y. The Complex Cell CNN p og ammable a ay compu e [5][11] is an ex ension o he CNN Uni e sal Machine [1]-[4]. The mul ilaye o highe o de elemen a y cells a e o - 2 ganized in o a 32×32 squa e g id. Each cell has second o de dynamics and local in e - connec ions o i s neighbo s. This s uc u e is especially sui able o compu ing a ce ain se o o dinal di e en ial equa ions. Simple PDEs can be ans o med so ha hey can be easily p og ammed on he a ay compu e . Wi h he p og ammable Complex Cell CNN Uni e sal Machine scien is s a e p o ided wi h a unique oppo uni y o s udy ac i e-wa e p opaga ion and o he eac ion-di usion di e en ial equa ion based phenomena in a p og ammable manne . Exploi ing he inhe en p og ammabili y o he CNN a chi ec- u e, complex wa e-compu ing analogic algo i hms can be designed. In his pape we show how o ans o m he ma hema ical model o he phenomena ou lined abo e in o a o m sui able o pa allel p ocessing on he Complex Cell CNN chip and we p esen he i s measu emen esul s. Fig. 1: spi al- and au owa es and pa e ns measu ed on he CACE1k chip (image size: 32×32) The ollowing sec ion desc ibes he ma hema ical model o he co e o he complex cell chip. In Sec ion 3 he p o o ype chip is in oduced. Sec ion 4 desc ibes he ela ion be- ween he eac ion-di usion equa ions and he s uc u e o he complex cell chip. Finally, Sec ion 5 con ains he eal- ime chip measu emen s. We should emphasize ha no simu- la ion esul s a e p esen ed he e. This chip was designed also o compu ing biologically inspi ed e ina modeling. The implemen a ion o he main e ec o a e ina model will be published in he nea u u e elsewhe e. 2 Compu a ional a chi ec u e The a chi ec u e o he chip ollows he design o he i s o de CNN-UM chip [10], bu i s i s o de cell co e is eplaced by a second o de one [5]. Wi h his change we a e able o ep oduce basic ac i e wa e phenomena (See Fig. 1). Fig. 2 shows he unc ional a chi ec u e o he 2nd o de , wo laye CACE1k co e. in p u la y e 1 la y e 2 a12 a21y2A1x1 A2x2x2 x1 τ ττ τ1 τ ττ τ2 z1 b2u2 z2 b1u1 τ ττ τ1 ≥ ≥≥ ≥ τ ττ τ2 ou p u Fig. 2: Func ional diag am o he CACE1k chip The e olu ion law o he complex cell CNN is he ollowing di e en ial equa ion sys em: 3 () () 10,10,1,1 )()()( )( )()()( )( ,22,121,2,2,2 ,2 2 ,11,212,1,1,1 ,1 1 1 1 ≤≤≤≤≤≤≤≤ ++++−= ++++−= ∑ ∑ ∈ ∈ ijij ijijij Nkl klklij ij ijijij Nkl klklij ij xuMjMi zub xa xA xg d dx zub xa xA xg d dx τ τ (2.1) Whe e:      −<− ≤ > =∞→ 1 1 1 lim)( ,, ,, ,, , ijnijn ijnijn ijnijn m ijn xi mx xi x xi mx xg (2.2) Func ion g(.) egula es he s a e so ha i s ays wi hin he +1..-1 in e al. Va iables u1, u2 a e he independen inpu s, b1, b2 a e hei weigh ac o s espec i ely; z1,ij, z2,ij a e space a ian bias maps.. Va iables x1, x2 deno es he s a e a iables o he laye s. Each xij co - esponds o one cell; i is one pixel o a M by N image i we conside he sys em’s ou pu as a pic u e. A1, A2 a e he weigh s o he in a-laye couplings, a1, a2 a e he in e -laye weigh s. Equa ion (2.1) u ilizes he so-called ull- ange model (FSR)[7], whe e he ol age o he s a e a iable is always he same as he ou pu . 3 Chip desc ip ion 3.1 A chi ec u e o he chip and basic p ocesso s uc u e The p o o ype chip consis s o an analog p og ammable a ay p ocesso o 32×32 iden i- cal cells (Fig. 3), su ounded by he bounda y condi ions o he CNN dynamics. The e is also an I/O in e ace, a iming and con ol uni and a p og am memo y. The in e ace consis s o a se ializing-dese ializing analog mul iplexo . The p og am memo y is com- posed o he analog p og am memo y, and he logic p og am memo y and he swi ch con igu a ion egis e s. The analog ins uc ions and e e ence signals need o be ansmi - ed o e e y cell in he ne wo k in he o m o analog ol ages. Thus, a bank o D/A con e e s in e aces he analog p og am memo y wi h he p ocessing a ay. Dis ibu ing analog e e ences ac oss la ge dis ances wi hin a chip is no a i ial ask. Apa om he p oblems caused by elec omagne ic in e e ence, ol age d ops in long me al lines ca y- ing cu en s can be qui e no iceable. Thus, in o de o a oid his signal bu e ing and low- esis ance pa hs mus be p o ided. Finally, he iming uni is made o an in e nal clock/coun e and a se o FSMs ha gene a e he in e nal signals which con ol he p ocesses o images up/downloading and p og am memo y accesses. 4 Fig. 3: A chi ec u e o he chip E e y elemen a y p ocesso o he Complex Cell CNN p og ammable a ay chip in- cludes wo coupled con inuous- ime CNN co es (Fig. 4(a)) belonging o each o he wo di e en laye s o he ne wo k. The synap ic connec ions be ween p ocessing elemen s o he same o di e en laye a e ep esen ed by a ows in he diag am. The basic p oc- esso also con ains a p og ammable local logic uni (LLU) and local analog and logic memo ies (LAMs and LLMs) o s o e in e media e esul s. All he blocks in he cell communica e ia an in a-cell da a bus, which is mul iplexed o he a ay I/O in e ace. Con ol bi s and swi ch con igu a ion a e passed o he cell di ec ly om he global p o- g amming uni . (a) (b) Fig. 4: Concep ual diag am o he (a) basic cell and (b) CNN laye node The in e nal s uc u e o each CNN co e is depic ed in Fig. 4(b). Each co e ecei es con- ibu ions om he es o he p ocessing nodes in he neighbo hood which a e summed and in eg a ed in he s a e capaci o . The wo laye s di e in ha he i s laye has a scal- able ime cons an , con olled by he app op ia e bina y code, while he second laye has a ixed ime cons an . The e olu ion o he s a e a iable is also d i en by sel - eedback and by he eed o wa d ac ion o he s o ed inpu and bias pa e ns. The e is a ol age 5 limi e o implemen ing he FSR CNN model [7]. The s a e a iable is ansmi ed in ol age o m o he synap ic blocks, in he pe iphe y o he cell, whe e weigh ed con- ibu ions o he neighbo s’ a e gene a ed. The e is also a cu en memo y ha will be employed o cancella ion o he o se o he synap ic blocks. Ini ializa ion o he s a e, inpu and/o bias ol ages is done h ough a mesh o mul iplexing analog swi ches ha connec o he cell’s in e nal da a bus. 3.2 P o o ype chip da a The p o o ype chip has been designed and ab ica ed using 0.5µm single-poly iple-me al CMOS echnology. I s dimensions a e 9.27 x 8.45 mm2 (Fig. 5). The cell densi y achie ed is 29.24cells/mm2. The ime cons an o he laye s is a ound 100ns (unscaled). The p o- g ammable dynamics o he chip pe mi he obse a ion o di e en phenomena o he ype o p opaga ion o wa es, pa e n gene a ion, e c (See Sec ion 5 o examples). Table 1 summa izes he mos ele an da a o he p o o ype chip. Technology 0.5 µ m CMOS 1-P 3-M Numbe o cells 32 x 32 Die a ea 9.27 x 8.45 mm2 Die a ea (w/o pads) 8.77 x 7.94 mm2 A ay a ea 5.98 x 5.93 mm2 Package ce amic PGA – 100 Powe supply ol age 3.3V Logic "0" / Logic "1" 0V / 3.3V Accu acy on he weigh s 8b Image samples esolu ion 7-8b I/O a es 10Ms/s CNN ime cons an below 100ns Table 1: Cha ac e is ic pa ame e s o he CACE1k p o o ype chip Fig. 5: Mic opho og aph o he p o o ype chip 6 4 Complex cell CNN-UM chip as a “PDE sol e ” – ypical wa es 4.1 Gene al PDE o mula ion Le us conside he ollowing PDE o mula ion o a coupled ec o alued nonlinea eac ion-di usion sys em (examples o nonlinea PDEs could be ound in [26]- [32]): 11 11111012 22 22222021 10 10 1 1 (,) [ (,) ( (,))] ( (,)) ( (, )) ( (,)) (,) [ (,) ( (,))] ( (,)) ( (, )) ( (,)) (, ) (); (,) (| ( (, dx di c x g ad x x x x d dx di c x g ad x x x x d x x c x g g ad x φφαφβφγφ φφαφβφγφ φφ φ −=++ −=++ == && & & & & && & & & & &&& * 20 20 2 2 )) | ) (, ) (); (,) (| ( (,))|)y x c y g g ad x φφ φ == &&& * (4.1) whe e 12 [(,) (,)]x x φφ  can be in e p e ed as he ime e olu ion o a ec o alued image in ensi y ( 10 20 [() ()]xx φφ  a e he ini ial images), he ec o  x ep esen s he spa ial coo dina es, he ime a iable can also be in e p e ed as he scaling pa ame e and c is he conduc ance pa ame e in he di usion e m. The igh hand side o he equa ions con- sis s o h ee eac ion e ms: (i) α (.) is he “sel - eac ing e m”; (ii) β (.) is a “spa ial con- s ain ” (calcula ed om he ini ial da a); and (iii) γ (.) is he “c oss-coupling” e m. Some p oposed unc ions o g(.) a e: () () gg adIx KK gg adIx K K 1 22 2 11 0 100 =− > =+     >> +− exp ( ( , )) / , ((,)) / , ,  αα (4.2) Decoupling he wo equa ions (γ1 = γ2 = 0) and choosing α (ξ) = - ξ he PDE o mula- ion is gi en in he o m o he so-called “biased” nonlinea aniso opic di usion equa- ion (No ds öm [27]) an ex ended e sion o he Pe ona-Malik o mula ion [26]. Ge ig e . al [29] i s p oposed he applica ion o coupled nonlinea di usion sys ems o ec- o - alued image p ocessing ha has been u he s udied by many o he s [32]. Fo ou pu poses – mo i a ed by silicon implemen a ion – we ocus on a simpli ied e sion o (4.1) ixing he di usion pa ame e o a cons an alue. 4.2 Spa ial disc e isa ion In wo spa ial dimensions and assuming c = cons . (4.1) and (4.2) educes o: 11 1 11 11012 22 2 22 22021 ( , , ) [ ( ( , , ))] ( ( , , )) ( ( , , )) ( ( , , )) ( , , ) [ ( ( , , ))] ( ( , , )) ( ( , , )) ( ( , , )) dxy cdi g ad xy xy xy xy d dxy c di g ad xy xy xy xy d φφαφβφγφ φφαφβφγφ −=++ −=++ (4.3) A e spa ial disc e isa ion using he ini e di e ence app oach one ob ains: 1, 1 1 2, 1 1, 1 1, 1, 1 1, 1 1, 1 1, 1 1 10, 2, 2 2 1, 2 2, 2 2, 2, 1 2, 1 2, 1 2, () ( ( )) ( ( )) ( ) ( ( ) ( ) ( ) ( )) ( ) 4 () (()) (()) () ( () () () 4 ij ij ij ij i j i j ij ij ij ij ij ij ij i j i j ij ij d c c d d c c d φ γφαφ φ φφφφ βφ φγφαφ φ φφφφ −+ −+ −+ −+ =+−+ ++++ = + −+ +++ 1220, ()) ( ) ij βφ + (4.4) Finally, wi h α (.)= α 0 and β (.) = β 0 a simpli ied o m (wi h spa ial symme y and iso - opy) o he CNN complex cell equa ion is de i ed: 7 1, 1 1 2, 1 1 1, 1, 1 1, 1 1, 1 1, 1 1, 2, 2 2 1, 2 2 2, 2, 1 2, 1 2, 1 2, 1 2, 1, 1 10, () () ( ) () ( () () () ()) 4 () () ( ) () ( () () () ()) 4 ij ij ij i j i j ij ij ij ij ij ij i j i j ij ij ij ij ij d c c z d d c c z d z φ γφ α φ φ φ φ φ φγφ α φ φ φ φ φ βφ −+ −+ −+ −+ =+− + ++++ =+− + ++++ =12,220,2 ;ij ij zz z βφ +=+ (4.5) Making φ ij explici ly depend on a s a e a iable ξ ij such as φ ij= ( ξ ij) = sigm( ξ ij) leads a good app oxima ion o he ull- ange CNN ci cui model ([7], sigm(.) could be a piece- wise linea o mono onic con inuous smoo h unc ion playing he ole o a signal- limi e ). The o m o he co esponding 2nd o de CNN empla e is as ollows: 1 11 1 1 1 1 12 1 11 1 1 1 1 2 22 2 2 2 2 21 2 22 2 2 2 2 0/40 /4 /4 ; ; ; 1; 0 0/40 0/40 /4 /4 ; ; ; 1; 0 0/40 c Ac cc A B z c c Ac cc A B z c αγβτ αγβτ   =− = = = =      =− = = = =    (4. 6) Analysis: I should be no ed ha he diagonal e ms could also be added esul ing in a much be e spa ial iso opy du ing he di usion p ocess. Symme y and spa ial iso opy o hese empla es a e due o he ac ha he e a e no con ec ion e ms in he o iginal o - mula ion (4.1)–(4.2). Wi h hese modi ica ions he sign and magni ude o he empla e en ies in (4.6) could be changed in any spa ial di ec ions. The ime cons an is also ixed and equal o he wo laye s. Ha ing wo di e en ime a iables (say '' ≠ ') in (4.1) leads o a "double ime-scale" in he desc i ized sys em '' / ' = τ 2 / τ 1 ≠ 1 which has p ac ical ele ance when looking a a ious second o de models. 4.3 Typical wa e classes The e a e a numbe o pa ame e se ings o special in e es ha lead o e y di e en quali a i e beha io s in his symme ic second o de sys em. We ha e examined and ex- plo ed p ima ily he ollowing simple cases (see also Sec ions 5.2): (i) igge -wa e gene a ion (di usion p ocess in "sa u a ion" a a ious speeds) c1 > 0 ; c2 > 0 ; α 1-(c1+1)>0; α 2-(c2+1)>0; γ 1 = γ 2 = 0; β 1 = β 2 = 0; τ 2 / τ 1 ≠ 1 (ii) a eling-wa e, spi al-wa e and au o-wa e gene a ion (spa ially in e ac ing ig- ge -wa es) c1 > 0 ; c2 > 0 ; α 1-(c1+1)>0; α 2-(c2+1)>0; γ 1 ≠ 0; γ 2 ≠ 0; β 1 ≠ 0; β 2 = 0; τ 2 / τ 1 ≠ 1 (iii) e ina e ec s (spa ially in e ac ing ecep i e ields) α 1 ≈ c1 >0; α 2 ≈ c2 > 0; γ 1 > 0; γ 2 < 0; β 1 ≠ 0; β 2 ≠ 0; τ 2 / τ 1 ≠ 1 5 Measu emen and compu a ion examples In he ollowing we p esen chip measu emen o he di e en ac i e wa e phenomena: igge wa e, a eling wa e, au owa e, and spa io- empo al edge de ec ion. The mo ie iles o he measu emen s a e a ailable a [33] 5.1 S o ed p og ammabili y The CACE1k chip can be p og ammed using he so-called AMC (ANALOGIC MACHINE CODE) language. This is simila o he assemble language bu i con ains addi ional high- le el ins uc ions, such as: image loading, ame g abbe ins uc ions, e c. and buil in im- 8 age p ocessing unc ions. In he Appendix is a simple AMC p og am which compu es he snapsho s o he au owa e example (Subsec ion 5.2.4). The ac ual compu a ion is pe o med in line 40. Be o e his line is he p epa a ion o he compu a ion: se ing up he pa ame e s. A e he compu a ion is done, he esul s a e ead ou and displayed om he chip’s local analog memo ies. The elemen a y p og am o he chip is he empla e. I con ains he weigh ac o s o he coupling be ween he cells and weigh s o he inpu and he bias map. 16..1,1:τ 2121 1221 2 1,1 2 0,1 2 1,1 2 1,0 2 0,0 2 1,0 2 1,1 2 0,1 2 1,1 2 1 1,1 1 0,1 1 1,1 1 1,0 1 0,0 1 1,0 1 1,1 1 0,1 1 1,1 1 ==           =           = − − −−−− − − −−−− τ 21 τ:τ AA zzbb aa aaa aaa aaa aaa aaa aaa (5.1) The ope a ion o he a ay compu e is comple ely de e mined by he 25 empla e alues, he ini ial s a es and bounda y condi ions. In (5.1) A1 and A2 ma ices deno es he weigh s in e laye connec ions o he slowe and he as e laye espec i ely. The s eng h o he in luence o he second laye on he i s is con olled by a21 and a12 s ands o he e e se case. Symbols b1, b2, z1 and z2 a e he weigh s o he independen inpu s and he space a ian bias maps. The a io o he ime cons an s o he wo CNN laye s a e con olled by τ1, τ2 is ixed. An analogic algo i hm is made up o combina ion o empla e execu- ions, logic ins uc ions and spa ial a i hme ic ope a ions. The example shown in he Ap- pendix is a e y simple one, i con ains only empla e execu ion ope a ion. 5.2 Wa e phenomenon This subsec ion con ains he on-chip measu emen s o he ac i e wa e phenomena. 5.2.1 T igge wa e This example shows a e y simple e ec : an ac i e (nonlinea ) wa e in he as e laye ini ia es a second wa e in he slowe laye h ough he posi i e in e laye coupling. The inpu is ed o he as e laye . Fig. 6 shows snapsho s o he wa e e olu ion. 1:165.16.030 03.3 6.08.06.0 8.06.08.0 6.08.06.0 111 15.11 111 2121 122121 =−==== ==           −=           = 21 τ:τ AA zzbb aa (5.2) Fig. 6: T igge wa e p opaga ion in he as e and in he slowe laye . The i s ilm shows he ou pu o he as e laye . 9 5.2.2 E asu e e ec This example is an “enhancemen ” o he igge wa e example: a nega i e coupling is in oduced om he slowe laye back o he as e one. As a esul , a e a while, he igge wa e on he as e laye is e ased by he igge wa e om he slowe laye . As in he case o he igge wa e he inpu is gi en o he as e laye . Fig. 7 shows he snap- sho s o he wa e e olu ion on he as e laye and Fig. 8 shows he e olu ion on he slowe laye . 1:167.26.030 4.53.3 6.08.06.0 8.06.08.0 6.08.06.0 111 15.11 111 2121 122121 =−==== −==           −=           = 21 τ:τ AA zzbb aa (5.3) Fig. 7: Re ina like wide- ield e asu e e ec , as e laye . Fig. 8: Re ina like wide- ield e asu e e ec , slowe laye . 5.2.3 T a eling wa e The cha ac e is ic o his wa e is ha a single wa e on a els ac oss he ac i e me- dium. The au owa e is simply o med om he igge wa e empla e by adding a nega- i e in a-laye coupling om he slowe o he as e laye . In Fig. 9 obse e he annihila- ion p ope y o his wa e ype.