Synthetic generation of address-events for real-time image processing

Syn he ic Gene a ion o Add ess-E en s o Real-Time Image P ocessing A. Lina es-Ba anco, R. Senhadji-Na a o, I. Ga cia-Va gas, F. Gomez-Rod iguez, G. Jimenez and A. Ci i . A qui ec u a y Tecnologia de Compu ado es. Uni e sidad de Se illa. AV. Reina Me cedes sin, 41012-Se illa SPAIN Absnac - Add ess-E en -Rep esen a ion (AER) is a s eam. The ecei e pixel in eg a es he pulses and commnnica lon p o ocol ha emula es he nenons sys em’s econs mc s he o iginal low equency COn inUOUS- ime neu ons communica ion, and ha is ypically used o wa e o m, like a neu on does in he ne ous sys em. Pixels ans e ing Images be ween chips. I was o iginally de eloped ha a e mo e ac i e a e accessing he bus mo e equen ly o bio-inspi ed and eal- ime image p ocessing sys ems. Such han hose less ac , e. sys ems may consis o a complica ed hie a chical s nc n e ~~~~~~i~~i~~ he add esses pe o ming exba ope a ions on he images while hey a el om one chip o wi h many chips ha ansmi Images among hem in eal ime, while pe o ming some p ocessing ( o example, con olu ions). s eams om images s o ed in a compu e ’s memo y a e (le. EEPROM) allows ans o ma ion (ie. shi ing and p esen ed. A ha dwa e e sion will wo k in eal- ime. ~11 o o a ion) o images. Also, he image ansmi ed by one chip hem ha e been e alua ed and compa ed. can be ecei ed by many ecei e chips in pa allel, by p ope ly handling he asynch onous communica ion p o ocol. The peculia na u e o he AER p o ocol also 1. INTRODUCTION allows o e y e icien con olu ion ope a ions wi hin a 1991 by Si ilo i [SI o ans e ing he s a e o an a ay o The e is a g owing communi y o AER p o ocol use s o analog ime dependan alues om one chip o ano he . I bio-inspi ed applica ions in ision and audi ion sys ems, as uses mixed analog and digi al p inciples and exploi s pulse demons a ed by he success in he las yea s o he AER densi y modula ion o coding in o ma ion. Fig. 1 explains g oup a he Neu omo phic Enginee ing Wo kshop se ies he p inciple behind he AER basics. [I]. The goal o his communi y is o build la ge mul i-chip and mul i-laye hie a chically s uc u ed sys ems capable o pe o ming complica ed a ay da a p ocessing in eal ime. The success o such sys ems will s ongly depend on he a ailabili y o obus and e icien de elopmen and debugging AER- ools. One such ool is a compu e in e ace ha allows no only eading an AER s eam in o a compu e and displaying i on i s sc een in eal- ime, bu also he opposi e: om images a ailable in he compu e ’s memo y, In his pape se e a] so wa e me hods o gene a ing AER ano he ’ Fo p ope ly coded memo ies Add ess-E en -Rep esen a ion (AER) was p oposed in ecei e chip [81. [U ’ gene a e a syn he ic AER s eam in a simila manne as would do a dedica ed VLSI AER emi e chiu 121141151161. Fig. 1: lllus mlion o AER in ci-chip communic ion scheme The Emi e chip con ains an a ay o cells (like, o example, a came a o a i icial e ina chip) whe e each pixel shows a con inuously a ying ime dependan s a e ha changes wi h a slow ime cons an (in he magni ude o de o milliseconds). Each cell o pixel includes a local oscilla o (VCO) ha gene a es digi al pulses o minimum wid h (a ew nanoseconds). The densi y o pulses is p opo ional o he s a e o he pixel (o pixel in ensi y). Each ime a pixel gene a es a pulse (which is called “e en ”), i communica es o he a ay pe iphe y and a digi al wo d ep esen ing a code o add ess o ha pixel is placed on he ex e nal in e -chip digi al bus ( he AER bus). Addi ional handshaking lines (Acknowledge and Reques ) a e also used o comple ing he asynch onous 1 . _I .. .. ~ This wo k is in ol e in he amewo k o he Eu opean Resea ch p ojec CAVIAR, one o he objec i es o CAVIAR is de elop an AER-compu e in e ace. This pape p esen s se e al me hods o syn he ic AER s eams gene a ion (sec ion 11). I also e alua es hese me hods a ending o he execu ion ime compa ison and o he e o o dis ibu ion o he e en s along he ime associa ed o an image: dis ance o he equency o an in ensi y le el. The me hods a e e alua ed o images wi h di e en a e age in ensi y le els wi h a Gaussian his og am ( om IO-90% cha ge o e en s) and o a ypical image (a ound 50 % cha ge o e en s) (sec ion 111). Conclusions p esen ed in his pape a e no de ini i e. A ha dwa e implemen a ion and i s s udy is necessa y o a eal- ime compa ison. communica ion. The in e -chip AER bus ope a es a he maximum possible speed. In he ecei e chip he pulses a e 11. SYNTHETIC STREAM GENERATION di ec ed o he pixels o cells whose code o add ess was on he bus. This way, pixels wi h he same code o add ess in he emi e and ecei e chips The e a e many so wa e o ans o m a he Same pulse bi map image in o an AER s eam o pixel add esses. In all 0-7803-7937-3/03/$17.00 02003 IEEE 462 o hem he equency o appea ance o he add ess o a This me hod can be enhanced by applying a small shi gi en pixel mus be p opo ional o he in ensi y o ha be o e placing he e en s in he ime pe iod. This shi can pixel. No e ha he p ecise loca ion o he add ess pulses is he he add ess o he pixel. Fo example, he pixel no c i ical. The pulses can be sligh ly shi ed om hei (i,j)=(lO,lO) will be shi ed 1290 posi ions espec o he nominal posi ions; he AER ecei e s will in eg a e hem o beginning o he ime pe iod (N=M=I28). These posi ions eco e he o iginal pixel wa e o m. The ha dwa e implemen a ion o his me hod is unde add esses ha will he sen o an AER ecei e chip ia an s udy. Bul, he ime pe iod is no gene a ed sequen ially, so, AER bus. I we ha e an image o NxM pixels and each pixel he i s implemen a ion seems o need a memo y o sa e he can ha e a g ey le el alue om 0 o K, one possibili y is o comple e ime pe iod be o e ansmi ing i . place each pixel add ess in he add ess sequence as many imes as he alue o i s in ensi y, and dis ibu ed in D. TheRandom me hod a e ansla ed as a delay in he ansmission o he e en . Wha e e algo i hm is used, i will gene a e a ec o o equidis an posi ion. In he wo s case (all pixels wi h alue This me hod places he add ess e en s in he slo e, he add ess sequence would be illed wi h NxMxK posi ions ob ained by a pseudo- andom numbe gene a o based on Linea Feedback Shi Regis e s (LFSR) [71[9]. add esses. The ime used o ansmi ing an image needs o be he same o any image in ensi y cha ge, lea ing blank Due o he p ope ies o he LFSR used, each slo posi ion is slo s i non-e en has o he sen . Each algo i hm would gene a ed only once and no collisions appea . I a pixel in implemen a pa icula way o dis ibu ing hese add ess he image has he in ensi y p, hen he me hod will ake p e en s. Le us p opose some algo i hms: he Scan me hod, he Scan Slice, he Uni o m me hod, he Random me hod, alues om he pseudo- andom numbe gene a o and places he pixel add ess in he co esponding p slo s o he he Random-Squa e me hod and he Exhaus i e me hod. add ess sequence. They will no be equidis an bu will appea along he comple e add ess sequence. This me hod is as , because he image is swep only once, and because he In his me hod he image is scanned many imes. Fo each algo i hm does no need o pe o m sea ches o emp y Slo s. scan, e e y ime a non-ze o pixel is eached i s add ess is Due o he LFSR can ob ain wo consecu i e, o e y pu on he add ess sequence in he i s a ailable slo , and close , add esses in a ew calls, The LFSR-based me hod he pixel alue is dec emen ed by one. This me hod is e y can be enhanced using a b-bi coun e o he high pa O as , due o i does no need o look o emp y slo s, al hough he add ess. So, o each call o he add ess gene a o , 2' he image needs o be scanned many imes (K imes in he add esses equally dis ibu ed a e ob ained. Fo pixels wi h a wo s case). Howe e , a non-well e en dis ibu ion is g ey le el alue nea o 2', he dis ibu ion ob ained is ob ained due o all pulses o he pixels wi h low in ensi y simila o he ideal dis ibu ion. Howe e , o di e en g ey alues will appea only a he beginning o he sequence. le el, he dis ibu ion o he e en s ge s wo se again. Tuning he co ec size coun e alue o each image, he andom B. The Scan-Slice me hod me hod could be imp o ed. A. The Scan me hod Fig. 2 shows he LFSR s mc ll e wi h a 2-bi coun e o a 128x128 images wi h a 256 g ey le els. The Scan me hod can be enhanced in he dis ibu ion o e en s i an blank slo is le when all he e en s o a pixel LSB .>21 ha e been sen . The image is scanned many imes, as he h,SB L FIR Scan me hod does, hu when a pixel's alue is ze o, a blank " -1 '7 ''1 ''1 "1 7 1 '1 7 7 I I I I y slo IS le in he ime pe iod. --- I I - Bo h he Scan me hod and he Scan-Slice me hod can be easily implemen ed in ha dwa e, because he ime pe iod is gene a ed sequen ially. So, he equi ed ha dwa e could be a memo y o s o e he image, plus he con ol ci cui y C. The UniJo m nielhod Fig. 2: LFSR wi h a 2-bi coun e o pseudo- andom numbe s gene a ion. E. The Randoni-Squa e me hod Using he Random me hod wi h a ixed size coun e om In his me hod, he image is scanned pixel by pixel only I o he maximum g ey le el, he e en dis ibu ion o high once. Fo each pixel, he gene a ed pulses mus be le el pixels is accep able, bu poo o low le el alues. dis ibu ed a equal dis ances. As he sequence is ge ing Subs i u ing he coun e by ano he LFSR, he dis ibu ion illed, he algo i hm may wan o place add esses in slo s could he imp o ed. ha a e al eady occupied. This si ua ion is called 'collision' Fo a 128x128 image wi h maximum g ey le el o 255, and i appea s wi h he AER app oxima ion o in e -chip 8-bi LFSR (LFSR-8) is used o selec ing 255 slices o communica ion. In his case, i will pu he pulse in he 128x128 posi ions. and ano he 14-bi LFSR (LFSR-14) nea es emp y slo o he ime pe iod. So, his me hod, selec s he posi ion inside he slice. The image is scanned appa en ly, will make mo e mis akes a he end o he only once. Fo each pixel a 14-bi numbe is gene a ed by p ocess han he beginning. The execu ion ime g ows he LFSR-14, and he LFSR-8 is called as many imes as he conside ably because he collisions ha e a high-cos in ime in ensi y le el o he pixel would indica e. Fig. 3 shows he o esol e i . LFSRs used by he Random-Squa e me hod. 463 mlm7 h,SB LNR-8 “R-I, LSB Fig. 3: LFSR-8 and LFSR-I4 used by he Random-Squa e me hod. F. The Exhaus i e me hod This algo i hm also di ides he add ess e en sequence in K slices o NxM posi ions o an image o NxM pixels wi h a maximum g ey le el o K. This me hod is based on he idea ha each possible e en (as maximum NxMxK) has assigned one ixed posi ion in he add ess e en sequence. In such way, o he slice k, an e en o he pixel (i,j) is sen on he ime i he ollowing condi ion is asse ed: (k. c.,j)mod K + c,j 2 K N, M .(k -l)+ (i - I) .M + j = I whe e c,j is he in ensi y alue o he pixel 0,j) (see Fig. 4). and N.M K Fig. 5: Tes image se (10% o 90% e en cha ge) All o he me hods ha e been implemen ed in C++ language and an applica ion has been de eloped which allows he gene a ion o he add ess e en s om a g ey- scale image and calcula e he pa ame e s p e iously desc ibed. The Fig. 6 shows he sc eensho o his so wa e in e ace. Fig. 4 The e en dis ibu ion made by he Exhaus i e mu had. The Exhaus i e me hod ies o dis ibu e he e en s o each pixel in o he K slices a equal dis ances. Fo his p opose, he algo i hm scan he image K imes. In he i e a ion k, i he p e ious condi ion is ue, hen he co esponding e en is sen , o he wise he algo i hm will wai o he ollowing e en (no e en is sen in ime ). Ill. SIMULATION RESULTS This sec ion is de o ed o compa e he me hods p oposed abo e and o es ima e how he pe o mance o he me hods is a ec ed by he ype o image. To ca y ou his analysis a se o andom images we e gene a ed, which ep esen a small popula ion o images. In o he s popula ion o images, wi h di e en his og am pa e ns and cha ge o e en s, he esul s can a y in some me hods, al hough he beha iou A, will be close o he esul s he e p esen ed. Theses images was ob ained conside ing wo aspec s: i s The in e es o his poin is no o compa e he execu ion his og am mus be close o Gaussian dis ibu ion and he ime o each me hod, because he me hods ha e o be numbe o e en s needed o ansmi hem, his is called implemen ed in ha dwa e o a mo e eal compa ison. “image e en cha ge” and his way. 100% e en cha ge The e o e he in e es yields in he beha iou o he co esponds o an image wi h all o pixel wi h inaximum execu ion ime o each me hod espec o he image e en alue. In such a way, a image wi h 10% o e en cha ge, cha ge. The es has been made by using he nine images ep esen an image ha used 10% o he possible e en . Fig. showed in Fig. 5. 5 shows he images used. Fig. 6 So wa e in e ace The execu ion 464 ( p j) and he ollowing e en ( p$' ). The dis ance o he las e en is calcula ed supposing ha he nex e en is he i s o a new sequence o he same image (we suppose ha he image is con inuously e-sen ). i. Then we can measu e he mean e o o a pixel as he a e age o he di e ences be ween he ideal and eal dis ance. The e o exp ession is: gIQ., -d& e , = '.I k=I G.i I is easy o see ha he wo s case o his e o measu emen is which all he e en s a e oge he in he add ess sequence (see Fig. 8). The e o e, in o de o compa e he e o ob ained o di e en me hods and images, he e o o each pixel mus be no malized espec o he maximum e o associa ed o he pixel. The ollowing exp ession is he maximum e o --. I ).(I--) wi h e.,j#l 4.j Scan Slice and Exhaus i e me hod. Fo y,, = 1, he dis ibu ion e o is ze o, due o only one Fig. 7 shows he execu ion ime e sus he cha ge o e en has o be sen . e en s in he image. Scan and Exhaus i e me hods ollow an almos cons an ela ion because he cha ge o e en s is aduced in o an insigni ican inc easing in ime execu ion, al hough his a ec o he Exhaus i e me hod in images wi h hal cha ge o e en s. Random, Random Squa e and Scan Slice me hods ollow a g owing up beha iou , due o he inc easing numbe o e en s, wi h a less e ec in he Scan Slice. Uni o m me hod is also a ec ed h collisions (c ows Fig. 8: The wo s e en dis ibu ion. _- up as e han o he s), hu i seems o be a ela ion be ween he numbe o collisions and he his og am, as can be seen compa ing heses esul s wi h he ypical image. Finally, we de ine a ma ix wi h he same dimensions O he image, whe e he @,j) elemen ep esen s he e o no malized o he pixel @,j): B. The dis ibu ion e o In an ideal AER dis ibu ion all e en s o one pixel and image could be equidis an in ime: cons an equency o e en s. In his sec ion, he dis ibu ion o e en ob ained wi h each me hod is e alua ed. The dis ibu ion e o p oposed measu es how much he e en dis ibu ion ene a ed by a me hod de ia es om he ideal dis ibu ion. NE= - Le suppose Du is he ideal dis ance be ween e en s o he Fig. 9 shows he no malized dis ibu ion e o calcula ed o he nine es images using he me hods p oposed. The x- axis ep esen s he image e en cha ge and he y-axis is he e o . pixel 0,j) o a NxM image wi h K g ey le el alues. N.WK D. .=- 4.j '.J whe e 4., is he in ensi y alue o he pixel @,j). 465 Oi5 nbu18on e o VE wen cha ge + Scan-slice 8 Random 4 Random-Squa e B Exhaus i e 06 02 01 Fig. I I: Typical image his og am o 20 30 40 y1 60 70 00 90 Image wen cha ge (%) Fig. 9 Dis ibu ion e o Me hod Mean S d. De~ls lon 69.75 17.14 99.22 49.19 35.58 99.22 84.95 14.69 98.43 17.25 Uni o m Random (2bi s) 34.92 As i can he seen, he minimum e o is ob ained wi h he The e o dec eases wi h he image e en cha ge io all exhaus i e me hod. me hods excep Random. In he scan me hods he pixels wi h low in ensi y has a highe e o o dis ibu ion inside he ime pe iod selec ed o ansmi! ing he image, The Uni o m me hod has he bes esul s o he popula ion o image selec ed, because he e a e e y ew In his pape six so wa e me hods o gene a ing AER collisions. This me hod ep esen he close solu ion o he s eams om images s o ed in a compu e 's memoly ha e ideal dis ibu ion in he con ex p esen ed. been p esen ed. The di e en me hods ha e been analysed Le us conside a ypical case, Fig. IO shows a ypical and es ed by simula ion so wa e. I is necessa y he g ey le el image and Fig. 11 i s his og am. ha dwa e implemen a ion o eal- ime, and he A ha dwa e pla o m ha exploi s hese echniques is cu en ly unde de elopmen . Fo his goal. an analysis o di e en a chi ec u es o eal ime is being made. A dedica ed ha dwa e o w i e and ead oi om an AER bus is being de eloped using a s anda d FPGA-based p o o yping boa d. I is s ill soon o conclude ha he Uni o m me hod is he mos adequa e me hod, as i can be expec ed. The ha dwa e s udy will complemen he wo k p esen ed in his pape . 1V. CONCLUSIONS AND FUTURE WORKS . ... co esponding s udy. ., Fig. 10 Typical image Table I shows he maximum and mean no malized e o s yields a signi ican imp o emen o e he o he me hods. I can be Obse ed ha he alues o Uni o m Fo mo e speci ic popula ions o images, he me hods need o be analysed o cla i y he mos app op ia e. The e a e se e al ac o s ha can de e mine he selec ion o he me hod o he desi ed popula ion: he necessi y o eal- '' pe cen . As i can be seen, he Uni o m me hod p oposed ime, he dis ibu ion o e en s, whose ha e been analysed he e. Bu i can be in e es ing o analyse he e ec o he me hod sequence o e en s in a ic e ec in he AER bus, whe e co espond o an image be ween he 80 and 90 o cha ge he densi y o in ime could make he eSul S o o he Gaussian his og am popula ion showed in Fig. he me hods. Fo example, in biomedical applica ions, he x- Be ween he me hods ha di ide he ime pe iod in K ay o windows o slice, he Exhaus i e me hod ob ain he hes his og am. esul s. The Random-Squa e esul s a e be e han hose ob ained by he Random me hod. images ha e a non-gaussian 466 V. ACKNOWLEDGMENTS The au ho s wish o exp ess hei g a i ude o he suppo gi en o his wo k by he Eu opean Commission h ough p ojec CAVlAK ‘Con olu ion AEK Vision A chi ec u e o Real-Time” (IST-2001-341024), unded by he Eu opean Commission, V F amewo k P og amme “In o ma ion Socie y Technologies (IST) P og amme”. VI. REFERENCES Ill A. Cohen, R. Douglas, C. Koch, T. Sejnowski, S. Shamma, T. Ho iuchi, and G. lndi e i. Repo o :he Nu ional Science Fmmda:ion: Wu l hop on Neu umo phic Enginee ing. Tellu ide, Colo ado, USA, June-luly 2001. [w *l .ini.uni2h.chi ellundc] [2] A. Lina cs-Ba anco. “Es udiu y e oluocid,~ de in e&ces pu a lo conexi& de isiemm neu omd iios azedianie Add es -E m - Rep ~,~en u io ”. Ph.D. Thesis, Uni u si y o Se ille, Spain, 2003 [3] Cha les M. Higgins and Ch is o Koch. Ud i-Chip Neu onw phic Mo ion P ocessing. Janua y 1999. [4] Kwabena A. Baahen. Commcmicu:ing Nne.onul Enn embles beween N u omo phic ChipA. Neu umo phic Sys ems. Kluwe Academic Pubiishen, Bos on 1998. M. Si iloni, Wi ing Conside o ions in onnl g VLSl Sys ems waiih .4pplicu!ion io Field-P og ummobie Newo h, Ph.D. Thesis, Cali o nia Ins i u e o Technology, Pasadena CA, I99 I [6] Misha Mahowaid. VLSl Analogs oJ Neu onol Vi~uol P ocessing: A Syn hase o/Fo,a and Func ion. Ph.D. Ihesis. Cali o nia Insli U e o Technology Pasadena, Caii omia 1992. Pie e L’Ecuye , Fianqois Panne on. A hie!+ Cluss q Lineo Feedback Sh@ Regis e Gene a o s. P oceedings o he 2000 Win e Simula ion Con e ence. 181 Te esa Semno-Go ac edona, And eas G. And eou, Bemabe Lina es- Ba anco. AER hip Fil e ing Amhi ec ue k Vi io -P ocessing Sys ems. IEEE Tmnsac ions on Chui s and Sys ems. Fundamm al Theo y and Applica ions, Vol. 46, NO. 9. Sep embe 1999. [Y] Linea Feedback Shih Regis e V2.0. Xiiinx Inc. Oc obe 1, 2001. h m:llw~.xilinx.ca~i~~~~ ~~. [SI [7] 467