Experimental demonstration of real-time image-processing using a VLSI analog programmable array processor

Expe imen al Demons a ion o Real-Time Image-P ocessing using a VLSI Analog P og ammable A ay P ocesso Gus a o Lj iáa, Ra ael DomInguez-Cas o, Se ando Espejo, Elisenda Rocaa,), Pe e Fo1desy and Angel Rod Iguez-Vazquez de Mic oelec ónica de Se illa, Cen o Nacional de Mic oelec ónica, A da. Reina Me cedes s/n, Campus Uni e sidad de Se illa, E-41012 Se illa (Spain). bA ea de Elec ónica, Escuela Supe io de Ingenie os, Uni e sidad de Se illa, Camino de los Descub imien os s/n, Isla de la Ca uja, E-41092 Se illa (Spain). Hunga ian Academy o Sciences, SzTAKI, Analog and Neu al Compu ing Labo a o y, Kende u ca 13- 17, H- 1 1 1 1 , Budapes (Hunga y) ABSTRACT1 This pape desc ibes a ull-cus om mixed-signal chip which embeds dis ibu ed op ical signal acquisi ion, digi ally- p og ammable analog pa allel p ocessing, and dis ibu ed image memo y —cache—on a common silicon subs a e. This chip, designed in a O.5p m CMOS s anda d echnology con ains a ound 1, 000, 000 ansis o s, 80% o which ope a e in analog mode; i is hence one he mos complex mixed-signal chip epo ed o now. Chip unc ional ea u es a e in acco dance o he CNN Uni e sal Machine pa adigm: cellula , spa ial-in a ian a ay a chi ec u e; p og ammable local in e ac ions among cells; andomly-selec able memo y o ins uc ions (elemen a y ins uc ions a e de ined by speci ic alues o he cell local in e ac ions); andom s o age/ e ie al o in e media e images; capabili y o comple e algo i hmic image p ocessing asks con olled by he use -selec ed s o ed ins uc ions and in e ac ing wi h he cache memo y, e c. Thus, as illus a ed in his pape , he chip is capable o comple e complex spa io- empo al image p ocessing asks wi hin sho compu a ion ime (200ns o linea con olu ions) and using a low powe budge (<1.2W o he comple e chip). The in e nal ci cui y o he chip has been designed o ope a e in obus manne wi h >7-bi equi alen accu acy in he in e nal analog ope a ions, which has been con i med by expe imen al measu emen s. Hence, o all p ac ical pu poses, p ocessing asks comple ed by he chip ha e he same accu acy han hose comple ed by digi al p ocesso s p eceded by 7-bi digi al- o-analog con e e s o image digi aliza ion. Such 7-bi accu acy is enough o mos image p ocessing applica ions. The pape b ie ly desc ibes he chip a chi ec u e and ocus mos ly on p esen ing expe imen al e idences o he chip unc ionali y. Mul iscale low-pass and high-pass il e ing o g ay-scale images, analog edges ex ac ion, image segmen a ion, h esholded g adien de ec ion, ma hema ical mo phology ope a ions, sho es pa h de ec ion in a laby in h, skele onizing, image econs uc ion, se e al non-linea ype image p ocessing aks like absolu e alue calcula ion o g ay-scale g adien de ec ion and eal- ime mo ion de ec ion in QCIF ideo sequences a e some o he e y in e es ing applica ions ha ha e been demons a ed as a ailable when using he p o o ype. 1. INTRODUCTION In mos mode n elec onic sys ems he ole o analog is basically limi ed o ha o an in e ace be ween eal-wo ld analog signals and he numbe s handled by he co e digi al p ocesso s. In he case o sys ems in ended o p ocessing uni- dimensional signals, in e ace unc ions comp ise basically impedance ma ching, ampli ica ion, il e ing and analog- o- digi al con e sion. P ocessing is ealized in digi al domain by subsequen s ages o he p ocessing chain. Howe e , in he case o sys ems in ended o p ocessing la ge, mul i-dimensional, in e ela ed signals — such as image lows — i may be in e es ing o inco po a e pa allel p ocessing a he analog in e ace in o de o educe he amoun o da a been ans e ed o he digi al sec ion, and hence o imp o e e iciency o he whole p ocessing chain. i. This wo k has been pa ially unded by ONR-NICOP N68171-98-C-9004, DICTAM IST-1999-19007 and TIC 990826. In Applica ions o A i icial Neu al Ne wo ks in Image P ocessing V, Nasse M. Nas abadi, Aggelos K. Ka saggelos, Edi o s, P oceedings o SPIE Vol. 3962 (2000) • 0277-786X1001$15.00 235 Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020 Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use Ac ually, such analog pa allel p ep ocessing is obse ed al eady in he e ina — he senso y on -end o na u al "image p ocessing" sys ems. I embeds pho o ecep o cells o acqui e images, and p ocessing cells o pe o m non-linea spa ial- empo al p ocessing ope a ions on he incoming low o images h ough a sequence o laye s. Mo i a ed by he e iciency o na u al ision sys ems, uni e si ies and companies ha e ocused hei e o s on he de elopmen o new gene a ions o de ices capable o o e coming he d awbacks o adi ional ones h ough he inco po a ion o dis ibu ed pa allel p ocessing, and, by making his p ocessing ac concu en ly wi h he acquisi ion o he signal. One possible s a egy o achie e ha is h ough lip-chip bonding o sepa a e sensing and p ocessing de ices; ano he possibili y is o inco po a e he senso y and he p ocessing ci cui y on he same semiconduc o subs a e. "Silicon e inas", "sma -pixel chips" and " ocal-plane a ay- p ocesso s" a e membe s o his la e class o ision chips 2Thei de elopmen is expec ed o ha e a signi ican impac in qui e di e se scena ios. Howe e , indus ial applica ions demand chips capable o lexible ope a ion, wi h p og ammable ea u es and s anda d in e acing o con en ional equipmen . A powe ul me hodological amewo k o a sys ema ic de elopmen o hese ypes o chips is using he pa adigm o analog cellula compu ing whose mos ad anced implemen a ion is he so-called CNN uni e sal machine (CNN-UM) . Main p ocessing componen s o his a chi ec u e a e: a) pa allel a chi ec u e consis ing o an a ay o locally-connec ed analog p ocesso s; b) a means o s o ing, locally, pixel-by-pixel, he in e media e compu a ion esul s, and 3) s o ed on-chip p og ammabili y. When implemen ed as a mixed-signal VLSI chip wi h embedded dis ibu ed op ical senso s, image p ocessing a es o illions o ope a ions pe second can be achie ed wi h e y small size and low powe consump ion. Ac ually, his has been al eady demons a ed by he au ho s h ough a O.8 m mixed-signal chip con aining 20 x 22 cells and capable o only bina y image p ocessing . This pape desc ibes, om a high le el poin o iew, ano he chip belonging o he same amily which ou pe o ms p e ious ealiza ions ega ding bo h size, and unc ionali y; among o he hings his new chip is capable o g ay-scale p ocessing. The pape is o ganized as ollows: Sec ion 2 desc ibes he chip om a sys em poin o iew. Sec ion 3 p o ides he in o ma ion abou he speci ic elec onic implemen a ion ocussing in o he mapping o he cell s a e equa ion and in o he desc ip ion o he enhanced unc ionali ies ha ha e been added o each elemen a y p ocessing uni o inc ease he inal pe o mances o he p o o ype. Sec ion 4 shows wo applica ion examples while Sec ion 5 deals wi h he implemen a ion o non-linea image p ocessing asks. 2. CHIP DESCRIPTION The chip consis s basically o a squa e a ay o 64 x 64 iden ical cells (plus a su ounding ing o bo de cells which a e ob iously di e en ), each o which in e ac s wi h i s nea es neighbo s in acco dance o he so called ull signal ange model (FSR4). dx'J( ) = —g[x,j )]+ :A(i,j;k,l).xkl( )+ :B(i,j;k,l).ukl+z (1) C(k,I) E S (1,J) C(k, 1) E S (E,J) whe e x, ep esen s he s a e o he gene ic i — h cell; Xkl ep esen s he s a e o he cells loca ed in he in e ac ion neighbo hood o he i — h cell; A(i,j;k, 1) , B(i,j;k,1) and z a e p og ammable pa ame e s; and, In addi ion o he ci cui y needed o implemen (I), each cell con ains he ollowing: a) an op ical senso ; b) ou andomly-add essable analog memo y poin s plus o he ou digi al memo y poin s (i means ha he chip can s o e ou g ay-scale images plus ou bina y images); c) a p og ammable local logic uni o ealiza ion o logical ope a ions among bina y images; d) inpu /ou pu and con ol ci cui y. mJ ,x1( )<—l mR ,x1( )> I mL mR (2) 236 Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020 Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use Fo gi en inpu image ( ep esen ed by he ma ix o u1 alues), ei he cap u ed by he op ical senso s o e ie ed om some o he cache memo ies, and gi en alue ze o-s a e image ( ep esen ed by he ma ix o x(O) alues, he chip can ealize a la ge a ie y o basic p ocessing asks (ins uc ions) h ough adjus men o he p og ammable pa ame e s in (1); namely he bias e m z and he in e ac ion pa ame e s which can be g ouped in o he so-called eedback empla e and con ol empla e, A = [A(i,j;k,l)] =B = [B(i,j;k,1)] (3) whe e means op, c cen e , b bo om, 1 le and igh . Thus, o ins ance, A1 deno es he scaling coe icien o he con ibu ion o he cen e -le cell s a e a iable in o equa ion (1). Basic chip ins uc ions a e hence de ined by 19 (ac ually 20 ) alues. In he chip hese alues a e p og ammable wi h a esolu ion o se en bi s plus sign. F om an ex e nal poin o iew, images may be analog (g ay-scale) o bina y (black & whi e). In e nally, pixel alues a e ea ed as analog in gene al, wi h black & whi e images ha ing ex eme analog le els co esponding o he limi s o he linea egion. Su ounding bo de cells a e used o es ablish he necessa y spa ial bounda y condi ions o p ocesses. O he miscellaneous unc ions like analog and digi al bu e ing, con ol, and I/O asks, a e also included wi hin he bo de cells. Figu e 1 (a) shows he chip a chi ec u e. In addi ion o he co e a ay p ocesso , he chip includes some global con ol and p og amming ci cui y loca ed in he pe iphe y o he cell a ay. This includes memo y o 32 a bi a y se s o coe icien s, which a e being p og ammed can be a bi a ily selec ed om he ou side. Some o he analog alues ela ed o he p ocessing ci cui y, like he limi s o he linea egion and o he s, can also be p og ammed. Digi al o analog (DA) con e e s gene a e he analog-p og am signal le els ansmi ed o he cell a ay om he selec ed se o coe icien s. Also, 64 a bi a y se s o 35 digi al signals ha a e used as digi al ins uc ions o con igu e p ope ly he cell in o de o pe o m di e en ask anging om unning a p ocess o con igu e he cell I/O ci cui y. These memo ies can be andomly add essed om he hos ing pla o m once hey ha e been p og ammed. Figu e 1 (b) shows he chip mic opho og aph. The ex e nal con ol is comple ely digi al. The in e ace has been designed o be easily embedded in con en ional digi al sys ems cen e ed a ound a CPU o a DSP uni . Two bidi ec ional da a-buses, one analog and one digi al, a e employed o image ans e ences. Figu e 1. (a) Chip A chi ec u e. Figu e 1. (b) Chip Mic opho og aphy 237 Digi al I/O Da a bus - - - - 9.145mm Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020 Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use The p o o ype has been designed and manu ac u ed in a 0.5 im, single-poly, 3-me al laye s, CMOS echnology. Table 1 shows he mos impo an physical and elec ical da a o he p o o ype. 3. ARRAY CIRCUITRY In an elec onic implemen a ion, equa ion (1) mus be eplaced by a simila one in which a iables and pa ame e s ha e physical ____________________ ___________________ meaning. The p ocessing ci cui y employed in he p o o ype ep esen s he s a e a iable x and he cell' inpu u as ol ages ac oss capaci o s. Con ibu ions om cell o cell a e in he o m o cu en , and he e o e, empla e elemen s ha e he dimension o conduc ances. In his manne , incoming con ibu ions, in he o m o cu en s, a e easily added a a common node, and easily in eg a ed in ___________________ a capaci o , while cell' s a e and inpu a iables, in he o m o ol age, a e easily dis ibu ed o he synapses. The cell s a e equa ion can he e o e be w i en, in in eg al o m, as ollows : ( ) = (O)+[-I( ( )) +d1G ( ) + d=1 G + JD]d (4) ______________ _____________ The nonlinea dissipa i e e m Ig(V) . ske ched in Figu e 2, pe o ms an e ec i e ol age limi a ion o he s a e a iable x wi hin he linea egion [Vsa Vsa ] by "elimina ing" wha e e cu en is _____________________ necessa y. The same signal- ange is employed o he inpu signal U, which is sampled and s o ed in a capaci o C iden ical o ha employed o he in eg a ion. I , a we assign he index 0 o an imagina y neighbo gene a ing he bias, o o se , e m 'D and se i s inpu and s a e a iables o he sa u a ion le el Vsa 238 ii. The index d e e s o he eigh nea es neighbo s o he cell and he cell i sel Table 1. P o o ype Da a. # o Cells 4096 (64 x 64 A ay) # o T ansis o s —l.OOO.OOO # T ansis o s on he cell I 72 Cell Size l2O m x lO2.2m Cell Densi y -82 cells/mm2 Signal Swing [0.6, l.41V (P og am- mable.) Weigh Swing [2.15, 2.95]V (P og am- mable.) Time Cons an -l.2 s Time Cons an o Linea Con olu ions 2OOns Spa ial Uni o mi y on he Weigh Signals. 7.6-bi s 110 Digi al Ra e 10MHz 110 Analog Ra e 1MHz Powe Supply 3.3V Powe pe cell 250 W Powe Consump ion 1 .2W (wo s case) # o Templa es Memo- ized 32 # o lns uc ions 64 Die Size 9l45.10 m x 9534 j m C mL ,Vx<Vsa mJ mR —* C mR ,Vx>Vsa (5) —V Figu e 2. Cell' Non-Linea i y. C Vx Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020 Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use 00 x = u =Vsa (6) hen, (4) can be ew i en as: ( ) = (O)+;L[- J( ( ))+ dO d=O1 (7) whe e he o se e m is now exp essed as: 'D (G +G)Vsa (8) which is now gene a ed in he same o m (i.e., wi h he same ci cui blocks) and wi h he same e e ence (sa u a ion) le el Vsa han any o he con ibu ion o he in eg andin (4). The use o he sa u a ion le el as a e e ence o he gene a ion o he bias e m p o ides some addi ional homogenei y, since con ibu ions om sa u a ed cells a e p opo ional o hesa u a ion le el. No e also ha wo addi i e coe icien s a e employed o de ine he o se e m hus p o iding a double ange o he o se e m. 3.1 Enhanced Func ionali ies Some addi ional unc ionali ies ha e been inco po a ed which ha e speci ic applica ion in ele an p ocessing unc ions. 3.1.1 Image Memo ies E e y cell has he capabili y o s o ing ou analog (g ay-scaled) and ou bina y (black & whi e) pixel alues. A a sys em le el, his means ha he chip allows he s o age o a o al o eigh images. The use o any o hese images as inpu da a o some p ocess, o o some speci ic memo y as des ina ion o he esul o he p ocessing unc ion, can be ealized in a e y sho ime (a ound 0. 1 s). In u n, his esul s in a signi ican imp o emen in he compu a ion ime o mo e complex algo i hms equi ing i e a i e empla e applica ions, wi h he possibili y o s o ing in e media e esul s o bi u ca ed- low algo i hms. Bina y memo ies employ con en ional digi al la ches, while analog memo ies elay on "bo om-pla e sampling" swi ched-capaci o s ages ollowing he guidelines gi en in . 3.1.2 Local Logic Uni The local logic uni (LLU) is a p og ammable boolean ga e whose u h able is de ined as pa o he digi al ins uc ions s o ed in he p og amming ci cui y. I allows a comple ely pa allel ealiza ion o a bi a y bi - o-bi logic ope a ions be ween images s o ed a wo speci ic bina y memo ies. The esul ing image can be down-loaded o s o ed in anyo he ou bina y memo ies. Con en ional digi al ci cui y is employed o his pu pose. 3.1.3 F eezing Mask Ha ing a " eezing" mask means ha he con en o one speci ic bina y image memo y can (op ionally) be used as a lag which disables he e olu ion o he ma ked pixels du ing p ocessing ansien s, keeping hei s a e a iables ime-in a ian . The ealiza ion o his unc ion equi es jus a ew analog swi ches. 11.4 Global Ga es In many cases i is in e es ing o ind ou i some speci ic image ( o ins ance he ou come o some p ocess) is comple ely whi e o comple ely black, wi hou was ing he ime equi ed o download he whole image. The p o o ype inco po a es wo global ga es, one NAND and one NOR, o pe o m hese logic ope a ions o e he pixel alues o one speci ic bina y memo y. Wi h his unc ionali y, he ime equi ed o check i some image is comple ely black o whi e is abou 3ps. 239 Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020 Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use 3.1.5 Op ical Inpu In many eal-li e high-speed applica ions, he in o ma ion o be p ocessed by he ne wo k is an image ha is a ailable in op ical o m while he ou pu con ains only a ew de ails ex ac ed om he inpu . In hese si ua ions, he eadou p ocess is ex emely simpli ied and hence speeded up. Howe e , he hpu image is always a comple e ame and he e o e, he ime needed o ans e he image o he a ay can cons i u e a bo leneck. In hose cases, he capabili y o combining he senso y and he p ocessing planes, p o ides a d ama ic inc ease o he sys em pe o mances, since i p oduces sys ems ha do no only exploi he ad an ages o he ully pa allel p ocessing bu also hose o he ully pa allel image acquisi ion ha a e p o ided by a ma ix o pho osenso s me ged wi h ha o p ocesso s. Acco dingly o his, he chip inco po a es a pho osensing de ice wi hin each cell ha allows he acquisi ion o images ha a e di ec ly p ojec ed o e he silicon su ace. The sensing scheme is based on he in eg a ion, in he capaci o o any o he analog image memo y, o he cu en ha is gene a ed by di usion-subs a e pho odiode. 4. APPLICATION EXAMPLES 4.1 Mo emen De ec ion One o he key-ideas when coding a ideo sequence is he in e ame mo ion compensa ion 6• This s a egy codi ies he i s image on he sequence as an independen i em while he emaining ames a e codi ied as a p edic ed ame and he p edic ion e o . I he mo ion o he objec s in he scene is small enough and mos ly ansla ional, his scheme comp ises he ideo sequence in a e y e icien manne . Ou om all asks needed o MPEG-1,2 ideo comp ession, block mo ion es ima ion is c ucial in e ms o ime consump ion. Block mo ion es ima ion is based on sys ema ic block-ma ching sea ch. This sea ch is based on inding he candida e block, conside ing a Mean Absolu e Ji e ence c i e ion (MAD) which is he mos simila o he p edic ed one. In his subsec ion, we will show, only wi h example pu pose, how he i s low-le el, bu e y in ensi e asks, a e pe o med by he new p o o ype. Namely, he ex ac ion o blocks ha a e changed. A i s s ep in he block mo ion es ima ion, is he de ec ion o he changes among wo consecu i e ames. Du ing his ask, image subs ac ion, low-pass il e ing, and double- h esholding a e pe o med on chip (see Figu e 3). The ime needed o accomplish his i s s ep is l1Oxs , ha is, he p ocessing ime is 25ns pe pixel, while he o al numbe o ope a ions pe pixel is I subs ac ion (eigh bi s o esolu ion) + 1 hea di usion (N x (10 mul iplica ions+ 10 addi ions) whe e N is he numbe o i e a ions needed o achie e he desi ed equilib ium poin ) + 1 logic ope a ion. The second s ep is o ma k he changed blocks. The goal o his s ep is o ma k he blocks whe e a leas one pixel has changed. This is done by using he ixed s a e map and black wa e p opaga ion. The wa e ills up he block-sized a eas whe e a black pixel exis s, while he ixed s a e map s ops he wa e e olu ion a he bo de o each block. The equi ed ime is 26 s , hus p o iding a p ocessing ime pe pixel o abou 6ns .The esul o his s ep can be seen also in Figu e 3. .N ________ (b) Di e ence om 4 h ame (c) Hea Di usion (Low (d) Ma ked blocks Pass Fil e ing) + Double h esholding+ OR Figu e 3. De ec ion o he changes in a ideo sequence. 240 , (a) 5 h F ame Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020 Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use 4.2 Impulse Noise Remo al The p oblem o emo ing noisy pixels om an inpu image can be o mula ed in a gene al way as ollows. Gi en a se o pixel in ensi ies I(i,j) whe e 1 I NROWS 1Nc015 I ld a maximum a e o change among pixels R ,all hose pixels pi whe e he local a e o change is la ge han R should be subs i u ed by he local a e aging M o he in ensi y o i s neighbo s, whe e: m1 k1 : M=m-4k--1 8(9) The low cha o he algo i hm ha we a e using in o de o implemen his unc ion is shown in Figu e 4 and i wo ks as ollows: •Fi s o all, he use has o de ine wha is noise, ha is, he use has o de e mine which is he maximum alue o he di e ence among wo pixels ha makes a pixel o be conside ed as co ec ins ead o as noisy'. •Second, he local maxima and minima pixels a e de ec ed. Du ing his calcula ion, only hose pixels whe e he di e ence be ween i s in ensi y and ha o any o i s neighbo s is la ge (o lowe , i depends on whe he we look o maxima o minima loca ions) han he p e iously de ined h eshold will u n o black. •The esul o he maxima and minima de ec ions a e agg ega ed by using a logic NOR ia he Local Logic Uni . A e his las ope a ion, he esul ing image con ains whi e do s only whe e a local maxima o minima was de ec ed. •This las image is sen o he ixed-s a e image memo y, and a hea di usion (a e aging) p ocess is pe o med o e he inpu image. Due o ixed s a e unc ion, ha is included wi hin each cell, only hose pixels ha a e no ma ked (black) will be p ocessed. Since hose pixels a e conside ed as noise con ibu ions o he image, he a e eplaced by he local a e aging o hei neighbo s. Figu e 4 shows he low cha o his algo i hm while .. .....Figu e 4. Algo i hm o Impulse Noise Remo al. Figu e 5 shows an example o applica ion o a noisy image. I can be obse ed how noisy pixels a he inpu a e eplaced by he a e aging o he in ensi ies o hei nea es neighbo s hus inc easing signi ican ly he quali y o he image. The inpu image is an s anda d QCIF (176 x 144 ) size image ( he image is cu in o nine o e lapped pieces ha a e p ocessed one a e he o he ) and he o al ime consump ion (including he ime o 9loading and 9 downloading p ocess) is sligh ly below 5.2ms hus yielding a pixel a e o 200ns .Among his ime, he p ocessing akes 23ns . Du ing his 23ns each pixel pe o ms 18 compa isons, I logic ope a ion and a hea di usion (N x (10 mul iplica ions+ 10 addi ions)) whe e N is he numbe o i e a ions needed o achie e he desi ed equilib ium poin . iii.We conside ha noisy pixels a e always sepa a ed by co ec pixels ( hus dis inguishing noise om sha p edges) 241 Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020 Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use Figu e 5. l'he Impulse Noise Remo al Applica ion 5. IMPLEMENTATION OF NON-LINEAR INTERACTIONS 5.1 Theo e ical Backg ound Non-linea cell in e ac ions can be ealized by decomposing hem in o a sequence o se e al linea ones . We will deal wi h he decomposi ion o empla es whe e only he B e m is a non-linea unc ion. Fu he mo e, we will assume ha he non-linea i ies appea ing on he eed o wa d e m a e piecewise-linea unc ions and ha he inpu image is ime in a ian . Le us suppose ha he non-linea piecewise unc ion can be exp essed as: (10) whe e a and 3 a e eal numbe s, and ha he linea egions a e de ined by a se o m b eaking poin s E '2''' . In ha case, ha is also he mos common in p ac ice. he non-linea empla e can be decomposed in o a sequence o linea empla e execu ions. The algo i hm8 uns as ollows: •The p ocess s a s by selec ing he i s linea egion o he non-linea unc ion. Le us call R1 his egion ha is de ined by he b eaking poin s E •The nex s ep is o selec which a e he cells belonging o ha egion. This calcula ion is ealized by wo empla es exe- cu ions and a logic ope a ion (all o hem a e done on-chip). Wi h he i s empla e, he so called h eshold empla e, we d i e o black all hose cells ha ing >E,while wi h he second one, he so called in e se h eshold, we d i e o black all hose cells ha ing <.Finally a logic AND ope a ion o bo h esul s will selec hose pixels whe e < <'2 " l •Equa ions (Ii). (12). show he h eshold and he in e se h eshold empla e'. 000 30() A= 010 B= OaO (II) 000 000 i . Keep in mind ha = au11 + and he subindex ki deno es he cell' neighbo s. . These a e he FSR e sion o he empla es . 242 (a) Noisy Inpu Image (b) Ou pu Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020 Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use 000 —IO A= B= O—aO =2 (12) 000 000 •The non-selec ed cells a e " ozen", by using he eezing mask p o ided by he chip, while in heselec ed ones he co - esponding con ibu ion o he s a e equa ion is e alua ed and s o ed as a "bias map" ha will be upda ed (o no ) in he nex i e a ion by adding he new esul o he one ha was p e iously s o ed. The upda inglaw o he s a e a iables o he cells ha a e selec ed mus be gi en by he equa ion o a s aigh line (due o he ac ha kP() is linea be ween each wo b eaking poin s) c ossing he poin s E and 2 All he poin s belonging o his line sa is y: E—1 =qI()kp(1) (13) 21 '(2)'(1) and om heoiy, i can be demons a ed ha his ela ionship is ob ained i he ollowing empla e is execu ed"1: 000 00 A= _i B= k-aO (14) 000 0 00 z= whe e, k = (2)'1) (15) •The p ocess con inues o he nex linea egion. •Finally, a empla e execu ion is needed. In his empla e he eedback e m is he same as in he o iginal one, he eed o - wa d e m is se o ze o (modi ied B empla e), since i has been al eady calcula ed, and he o se e m is he addi ion o he o iginal one z , and he "bias map " ha is s o ed in some memo y on he cell. 5.1.1 Absolu e Value Calcula ion In his subsec ion we conside only pixel-wise ans o ma ions, o wi h o he wo ds, B empla es wi h he size o lx 1 As a consequence o missing neighbo connec ions he decomposi ion me hod can be simpli ied, a oiding he accumula ion o he pa ial esul s. Mo eo e , he selec ion o cells belonging a gi en in e al is done by he wo h eshold empla es, which also con ain only cen al elemen s (whe e a = 1, 0 and = 0): 000 000 A= 020 B= OaO Z1 (16) 000 000 i. The posi ion o he coe icien mus be o a ed in o de o pe o m his ope a ion o each o he neighbo s o he cell appea ing as a non-linea connec ion on he o iginal B empla e. The e o e, each linea egion could equi e up o 16 em- pla es and 8 logic ope a ions o be selec ed, 8 empla es o upda e he s a e a iable, and 8 empla es o pe o m he addi ion o he esul s, ha is 32 empla es and 8 logic ope a ions. 243 Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020 Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use