A P og ammable VLSI Fil e A chi ec u e o Applica ion in
Real-Time Vision P ocessing Sys ems
Te esa Se ano-Go a edona1, And eas G. And eou2, and Be nabé Lina es-Ba anco1
1Ins i u o de Mic oelec ónica de Se illa (IMSE), Cen o Nacional de Mic oelec ónica (CNM),
Ed. CICA, A . Reina Me cedes s/n 41012 Se illa, SPAIN. Phone: 34-5-4239923,
Fax: 34-5-4231832, E-mail: [email p o ec ed]
2Dep . o Elec ical and Compu e Enginee ing, The Johns Hopkins Uni e si y,
Bal imo e, MD 21218, USA
Abs ac
An a chi ec u e is p oposed o he ealiza ion o
eal- ime edge-ex ac ion il e ing ope a ion in an
Add ess-E en -Rep esen a ion (AER) ision sys em.
Fu he mo e, he app oach is alid o any 2D
il e ing ope a ion as long as he con olu ional ke nel
F(p,q) is decomposable in o an x-axis and a y-axis
componen , i.e. F(p,q)=H(p)V(q), o some o a ed
coo dina e sys em {p,q}. I i is possible o ind a
coo dina e sys em {p,q}, o a ed wi h espec o he
absolu e coo dina e sys em a ce ain angle, o which
he abo e decomposi ion is possible, hen he
p oposed a chi ec u e is able o pe o m he il e ing
ope a ion o any angle we would like he ke nel o be
o a ed. This is achie ed by aking ad an age o he
AER and manipula ing he add esses in eal ime.
The p oposed a chi ec u e, howe e , equi es one
app oxima ion: he p oduc ope a ion be ween he
ho izon al componen H(p) and e ical componen
V(q) should be able o be app oxima ed by a signed
minimum ope a ion wi hou signi ican pe o mance
deg ada ion. I is shown ha o edge-ex ac ion
applica ions his il e does no p oduce pe o mance
deg ada ion. The p oposed a chi ec u e is in ended
o be used in a comple e ision sys em known as he
Bounda y-Con ou -Sys em and Fea u e-Con ou -
Sys em Vision Model, p oposed by G ossbe g and
collabo a o s. The p esen pape p oposes he
a chi ec u e, p o ides a ci cui implemen a ion using
MOS ansis o s ope a ed in weak in e sion, and
shows beha io al simula ion esul s a he sys em
le el ope a ion and elec ical simula ion and
expe imen al esul s a he ci cui le el ope a ion o
some c i ical subci cui s.
I. In oduc ion
Human beings ha e he capabili y o ecognizing
objec s, igu es,andshapese eni heyappea embedded
wi hin noise, a e pa ially occluded o look dis o ed. To
achie e his, he human ision p ocessing sys em is
s uc u ed in o a numbe o massi ely in e connec ed
neu allaye s wi h eed o wa dand eedbackconnec ions
among hem. Neu ons communica e by means o
elec ical s eams o pulses. Each neu on b oadcas s i s
ou pu o a la ge numbe o o he neu ons, which can be
inside he same o a di e en laye s, and he way his is
doneis h oughphysicalconnec ionscalledsynapses.
One big p oblem encoun e ed by enginee s when i
comes o implemen bio-inspi ed ( ision) p ocessing
sys ems is o o e come he massi e in e connec ions. An
in e es ing way o ying o sol e his is by Add ess E en
Rep esen a ion(AER)[1]-[3].InAEReachneu oncodes
i s ac i i y as a pulse s eam signal wi h e y low du y
cycle, i.e. pulse wid h mus be minimum bu sepa a ion
be ween pulses should be ai ly la ge. Each neu on has a
code o add ess, and e e y ime i p oduces a pulse i will
y o w i e i s code on a common digi al bus. A ecei ing
sys em will con inuously be eading his bus and send he
pulse o hose neu ons who ough o be connec ed o he
sending neu on. In his manne he ac i i y o a la ge
numbe o neu onscan be imemul iplexedon acommon
bus. This p inciple allows o s uc u e hie a chically a
e y complex neu al sys em. Fo example, a e ina chip
wi h AER ou pu is con inuously pu ing add esses on a
bus ep esen ing he sensed images. Se e al chips, each
wi h an AER ecei e sys em, can be eading he same
bus, doing some specialized p ocessing and b oadcas ing
he ou pu s o all hei neu ons using again AER on
ano he ex e nal bus, and so on. Fu he mo e, ex a
p ocessing can be added easily while he “add esses” go
om one chip o he nex . Fo ins ance, image o a ion o
ansla ion can be pe o med in a s aigh o wa d manne
by inse ing an EEPROM o which he ans o ma ion
ope a ionhasbeenp og ammedpixelbypixel(o add ess
by add ess). In he a chi ec u e p oposed in his pape we
akead an ageo his ac osimpli y hep ocessingchip.
As neu oscien is s manage o un old he in e nal
s uc u e and unc ions o he ision sys em, i becomes
mo e easible o ma hema iciansandcompu e scien is s
op oposeandunde s andbio-inspi ed isionmodelsand
algo i hms, and o enginee s o build bio-inspi ed
a i icial ision sys ems. One powe ul ision model
p oposed ecen lybyG ossbe ge al.[5]is heBounda y-
Con ou -Sys em (BCS) and Fea u e-Con ou -Sys em
(FCS) ision model. I consis s o nine laye s which a e
local illumina ion no maliza ion and con as
enhancemen o an inpu image, pe o ms local edge
ex ac ion o di e en spa ialo ien a ionsandscales,and
hen is able o iden i y consis en long ange con ou s o
he shapes in he inpu image h ough p ocessing laye s
wi h eed o wa d and eedback connec ions. In his
ision model one o he s ages pe o ms a 2D il e ing
ope a ion o edge ex ac ion, and o he s ages pe o m
o he 2D il e ing ope a ions.Thep ocessinga chi ec u e
p oposedin hispape isin ended obeused in his ision
model o pe o m a simpli ied e sion il e doing he
edge-ex ac ion ope a ion. The same p ocessing
a chi ec u ecanbe ep og ammed ope o msomeo he
o he 2D il e sneededin heBCS-FCS isionmodel.
Thep esen pape iss uc u edas ollows.In henex
Sec ion we will b ie ly desc ibe he s uc u e,
unc ionali y, and ope a ions pe o med by he BCS-FCS
ision model. In Sec ion III we in oduce modi ica ion o
he edge-ex ac ion ke nel which subs i u es a p oduc
ope a ion by a minimum ope a ion in he o iginal ke nel.
Sec ion IV desc ibes b ie ly he essence o AER, and in
Sec ion V we in oduce a VLSI a chi ec u e capable o
implemen ing a 2D p og ammable il e . Sec ion VI
p o idessys emle elbeha io alsimula ion esul so his
a chi ec u ep og ammedwi hake nel odoanex ac ion
o e ically o ien ed edges, and inally Sec ion VII
indica es heconclusionsand u u ewo k.
II. The Bounda y-Con ou -Sys em and
Fea u e-Con ou -Sys em Vision Model
Fig. 1 shows a schema ic ep esen a ion o he
s uc u eo heBCS-FCSmodel[5].TheBCSconsis so
se e al iden ical subsys ems ( h ee in he case o Fig. 1)
each o which is uned o a di e en spa ial scale. Each
BCS spa ial subsys em consis s o 8 laye s. Consecu i e
laye s ha e been d awn in Fig. 1 as connec ed by hick
shaded a ows. We may hink o hese a ows as he
ep esen a ion o a con olu ion (o il e ) ope a ion
applied o he s a e o he p e ious laye and esul ing in
he s a e o he nex laye . Fo ins ance, he 2D inpu
image su e s h ee di e en il e ing o con olu ional
ope a ions, each o which is he s a ing poin o a BCS
subsys em. F om he e on, each BCS subsys em ope a es
au onomously. F om Laye 1 oLaye 3 he e a e only
eed o wa d il e ing ope a ions, while Laye s 4 o 8 a e
connec edina eedback loopcon igu a ion, whichmeans
hesys em will eacha s eadys a e a e a ce ainnumbe
o i e a ions(i hesys emisimplemen edsequen iallyon
a compu e ) o a e a ce ain ime cons an (i he sys em
ope a es asynch onously and ully pa allel, like in
biological b ains). The ou pu s o Laye s 1 and 5 o he
h ee BCS subsys ems a e ed o he FCS. Nex we will
b ie lydesc ibe hep ocessing pe o medon hedi e en
laye s.
A.S age 1: Cen e -ON OFF-Su ound
Le usassume
(1)
is an N×Minpu image p o ided by a ision sensing
on end. This inpu image is applied o a 2D il e
whose impulsi e esponse o ke nel o adial symme y
is shown in Fig. 2(a). We can see ha pixels close o he
cen e egion o he ke nel a e going o con ibu e wi h
posi i e weigh s o he con olu ion, while pixels u he
away will con ibu e nega i ely. The esul o such a
con olu ion is local illumina ion no maliza ion and
con as enhancemen . The ma hema ical exp ession o
his ke nel is
(2)
whe e , a e posi i e pa ame e s, , and
con ols he spa ial scale o he il e ing. In he case o
Fig. 1 he e a e h ee BCS subsys ems, which means
h ee Cen e -ON OFF-Su ound il e s a e applied in
pa allel o he same inpu image, each wi h a speci ic
( ). F om now on he p ocessing in each BCS
subsys em is independen .
B.S age 2: Simple Cells
The second s age o he BCS sys em applies an
o ien a ion speci ic con olu ion o de ec ing edges
o ien ed wi hin a na ow angle ange. This is pe o med
bycon ol ing heou pu o Laye 1wi h heke nelshown
in Fig. 2(b) o di e en o ien a ions. This is why he
ou pu o Laye 1 in Fig. 1 su e s se e al con olu ions in
pa allel, one o each o ien a ion, esul ing in as many
“Laye 2” as o ien a ions ha e been conside ed. The
ke nel o Fig. 2(b) is ma hema ically desc ibed by he
di e encebe ween wodisplacedgaussians
Laye 6
Laye 7
Laye 2
Laye 3
Laye 4
Laye 1
Laye 5
Laye 8
Inpu Image
Laye 9
Ou pu Image
Fig. 1: Schema ic Rep esen a ion o G ossbe g’s e al.
Bounda y Con ou Sys em (BCS) and Fea u e Con ou
Sys ems (FCS) Vision Model
BCS
FCS
Ipq,() p1…N,=
q1…M,=



S1pq,()A1e
p2q2
+
σg
-----------------
A2e
p2q2
+
ασg
-----------------
–=
A1A2
>α1>σ
g
σg
g123,,=
(3)
whe e he coo dina e sys em is o a ed a ce ain
angle wi h espec o he coo dina e sys em o he inpu
image ,
(4)
wi h being he o al numbe o o ien a ions o be
conside ed.
C.S age 3: Complex Cells
A e applying he il e ing o S age 2, a pixel in
Laye 2 o o ien a ion kwilldisplayahighposi i e alue
i he inpu image p esen s a posi i e change in con as
wi h espec o he k- h o ien a ion axis. I he change in
con as is nega i e, he ou pu o his pixel would be a
highnega i e alue.Ino de ode ec whe he o no he e
is an edge a ha o ien a ion a ound he gi en pixel he e
is no need o dis inguish be ween posi i e and nega i e
alues. The e o e, he pu pose o his p ocessing s age is
simply o ec i y heou pu o hep e iousone.
D.S age 4: Hype complex Cells, Compe i ion ac oss
Space
A Laye 3 o o ien a ion k, pixels will p esen a
posi i e alue i a ound ha pixel he e is an edge a ha
o ien a ion. The highe he pixel alue, he clea e he
edge was. A his s age, and independen ly o each
o ien a ion, a 2D Cen e -ON OFF-Su ound il e is
applied o con as enhance he p e ious image. This is
equi alen o pe o ming a spa ial compe i ion among
pixels, a o ing hosewi hhighe alues.
E.S age 5: Hype complex Cells, Compe i ion ac oss
O ien a ions
A his s age, all Laye 4 pixels o he same spa ial
posi ion bu o all possible o ien a ions k, a e going o
compe e among hem o con as enhance hose
o ien a ions wi h highe pixel alues. This is done by
applying a 1D Cen e -ON OFF-Su ound il e o pixels
o Laye 4 o he same spa ial posi ion bu o all k
o ien a ion alues.
F.S age 6: Bipole Cells, Long-Range Coope a ion
Theope a iono hiss ageis hemos complica ed.I
ies o iden i y long e m “Con ou s”, which can be
de ined as edges ha emain consis en o e la ge space
anges. This is achie ed by pe o ming o each
o ien a ionk he ollowingsumo con olu ions,
(5)
whe e is he esul ing s a e o pixel o Laye
6 o spa ial scale gand o ien a ion k, subsc ip
deno es o ien a ion, and each con olu ion is gi en by
(6)
whe e deno es he s a e o pixel o Laye
5 o o ien a ion , deno es o ien a ion pe pendicula
o , and he ke nel is de ined by
(7)
wi h , , and being posi i e pa ame e s. Fig. 2(c)
depic s his ke nel o he case .
G.S age 7: Hype complex Cells, Compe i ion ac oss
Space
Thiss agepe o ms hesameope a ion hanS age4.
(a)
Fig. 2: Con olu ional Ke nels used in he BCS sys em o
Fig. 1: (a) Cen e -ON OFF Su ound Ke nel used by
S ages 1, 4, and 7. (b)Edge-Ex ac ion Ke nel used by
S age 2. (c) Bipole Ke nel used by S age 6.
(b)
(c)
010 20 30 40 50 60 70
0
20
40
60
80
−0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
010 20 30 40 50 60 70
0
20
40
60
80
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
010 20 30 40 50 60 70
0
20
40
60
80
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Fgpkqk
,() 1
2πσghσg
------------------------ e
1
2
---pk
σgh
--------


2
–
e
1
2
---qk
σg
--------1
2
---+


2
–
e
1
2
---qk
σg
--------1
2
---–


2
–
–=
pkqk
,()
pq,()
pkpπk
nR
------cos qπk
nR
------sin–=
qkpπk
nR
------sin qπk
nR
------cos+=
nR
Apqk
gCpqk ,,,()
1=
nR
∑
=
Apqk
gpq,()
Cpqk ,,,()y pq,()y
ˆpq,()–[]Z pkqk
,()⊗=
y pq,() pq,()
ˆ
Z pkqk
,() pk
()sgn eβpk
2qk
2
+()–eµqk/pk
2
()
2
–×=
k –()π
nR
------------------- pk
()sgn 2 2qk
pk
-------- pk
,


a an–



cos×
βµ γ
k 0==
H.S age 8: Hype complex Cells, Compe i ion ac oss
O ien a ions
Thiss agepe o ms hesame ope a ion han S age 5.
The ou pu o S age 8, Laye 8,is combined wi h he
ou pu o S age 3, Laye 3, o o m he inpu image o
S age 4. This way a eedback loop is o med, which once
se led will yield he p ope ou pu o each BCS
subsys em.
I.S age 9: Fea u e Con ou Sys em (FCS)
The in o ma ion abou consis en long ange
con ou s can be aken om Laye 5 (once he eedback
loop has se led), o all compu ed o ien a ions. The FCS
akes he o iginal (local illumina ion no malized and
con as enhanced)imagep esen inLaye 1andpe o ms
a selec i e di usion ope a ion be ween pixels, using he
con ou in o ma ion p esen a Laye 5: he con ou s a
Laye 5 ac as ba ie s o he di usion ope a ion. The
esul o all his p ocessing is a clean noise- ee image
wi hclea andconsis en long angecon ou s.
III. An Edge-Ex ac ion Fil e
In he es o his pape wewill concen a eonS ages
2 and 3, he il e o edge-ex ac ion and subsequen
ec i ica ion.Wewill i s in oduceasimpli ica iono he
ke nel o eq. (3) which will allow us o p opose a e y
compac and e icien ha dwa e ha akes ad an age o
heAERaswell.
The ke nel o eq. (3) is decomposable in o wo
ac o s, each o which depends only on ei he he x-
coo dina e o hey-coo dina e ,
, (8)
wi h (9)
The simpli ica ion p oposed he e consis s in subs i u ing
he p oduc ope a ion be ween and by he
signed minimum,
(10)
Fig. 3(b) shows he esul o applying he il e ing o eq.
(8) o he inpu image o Fig. 3(a), while Fig. 3(c) esul s
when using he ke nel o eq. (10). As can be seen he e
is no app eciable di e ence in he esul ing images.
To e alua e quan i a i ely he e ec o he p oposed
app oxima ion we can use he No malized Squa e E o
de inedas,
. (11)
This quan i y helps us o e alua e he di e ence be ween
he o iginal ke nel and he modi ied ke nel
ob ained when he p oduc ope a ion is
subs i u ed by he signed minimum. Table 1 gi es he
compu ed NSE o se e al ke nels. All ke nels in Table
1 a e decomposable in he p oduc o wo unc ions ha
depend sepa a ely on he and componen s.
IV. Using Add ess E en Rep esen a ion
(AER)
Fig.4 shows aschema ic igu e ou lining heessence
behind he AER. Suppose we ha e an “emi e ” chip
con ainingala genumbe o neu onso cellsD1,D2,D3,
... whose ac i i y changes in ime wi h a “ ela i ely slow”
ime cons an . Fo example, i Chip 1 is a e ina chip and
each neu on’s ac i i y ep esen s he illumina ion sensed
by a pixel, he ime cons an wi h which his ac i i y
pkqk
Fgpkqk
,()
1
2π
------ Hgpk
()Vgqk
()=
Hgpk
() 1
σgh
-------- e
1
2
---pk
σgk
--------


2
–
=
Vgqk
() 1
σg
-------- e
1
2
---– qk
σg
--------1
2
---+


2
e
1
2
---– qk
σg
--------1
2
---–


2
–=
Hg.() Vg.()
Fg'pkqk
,()
1
2π
------ Hgpk
()[]sgn Vgqk
()[]×sgn=
min Hgpk
()Vgqk
(),{}×
NSE Fxy,()Fmxy,()–2xdyd
∫∫
Fxy,()
2xdyd
∫∫
-----------------------------------------------------------------------=
100 200 300 400 500 600
50
100
150
200
250
300
350
400
450
100 200 300 400 500 600
50
100
150
200
250
300
350
400
450
100 200 300 400 500 600
50
100
150
200
250
300
350
400
450
(a)
(b)
Fig. 3: Beha io al Simula ion Resul s when pe o ming
2D Fil e ing o e ical edge-ex ac ion. (a) Inpu Image,
(b) using he edge-ex ac ion il e ke nel o eq. (8), (c)
using he modi ied il e ke nel o eq. (10)
(c)
Fxy,()
Fmxy,()
xy
changesis,a hemos ,equi alen oF ame-Ra e(i.e.,25-
30 changes pe second o a ime cons an o abou 30-
40ms).
The pu pose o an AER based communica ion
scheme is o be able o ep oduce he ime e olu ion o
each neu on’s ac i i y inside a second o “ ecei e ” chip,
usinga as digi al buswi ha smallnumbe o pins. In he
“emi e ” chip he ac i i y o each pixel has o be
ans o med in o a pulse s eam signal such ha pulse
wid h is minimum and he spacing be ween pulses is
easonably high o ime mul iplex he ac i i y o a
ela i ely la ge numbe o neu ons. E e y ime a neu on
p oduces a pulse i s add ess o code should be w i en on
he bus. Fo he case mo e han one pulses a e p oduced
simul aneously by se e al neu ons, a classical a bi a ion
ee can be in oduced [1]-[3], o one based in Winne -
Take-All(WTA) ow-wise compe i ions[6], o simplyby
making no neu on accessing he bus in case o a
“collision”[7].Wha e e me hodisused he esul willbe
he p esence o a con inuous sequence o add esses o
codes on he digi al bus ha one o mo e ecei e chips
can ead. Each ecei e chip mus con ain a decoding
ci cui y so ha a pulse eaches he neu on (o neu ons)
speci ied by he add ess ead on he bus. I each neu on
in eg a es he sequence o pulses p ope ly, he o iginal
ac i i y o he neu ons in he emi e chip will be
ep oduced.No e ha inAER hoseneu ons ha a emo e
ac i e access he bus mo e equen ly. This p ope y
allows o op imize he use o he bus, since neu ons wi h
low ac i i y will no consume much communica ion
bandwid h.
This is he simples AER based communica ion
schemeamong chips.Howe e , AERallowseasily oadd
mo ecomplica edp ocessing. Fo example, inpu images
can be ansla ed o o a ed by emapping he add esses
while hey a el om one chip o he nex . By p ope ly
p og amminganEEPROMasalook-up ableanyadd ess
emapping can be implemen ed, by simply inse ing he
EEPROM be ween he wo chips. Fu he mo e, many
EEPROMs can be connec ed in pa allel each pe o ming,
o example, a o a ion a a speci ic angle, and each
deli e ing he emapped add esses o a se o specialized
p ocessing chips. I is also possible o include synap ic
weigh ingbyha ing heEEPROMs o e heweigh alue,
dumping i on a da a bus, ha e he “ ecei e ” chip ead
bo h headd essand heda abus,andpe o maweigh ed
in eg a ion in he des ina ion(s) neu on(s). I is also
possible o implemen “p ojec i e ields”, i.e. o e e y
add ess ha appea s on he bus a small digi al sys em
couldgene a easequenceo add essesa oundi andsend
i o he“ ecei e ”chip.Thiswouldbea ime-mul iplexed
p ojec ion ield gene a ion. In he a chi ec u e p oposed
in his pape , we implemen a synap ically weigh ed
p ojec ion ield o each add ess ead on he bus, and no
in a ime-mul iplexed manne bu in pa allel. As we will
see, he ecei e chip will pe o m he ollowing
ope a ions: o e e y add ess ead on he bus i will send
pulses o a bubble o neu ons a ound ha add ess. The
wid h o hose pulses is modula ed acco ding o some
weigh s s o ed on chip. Time in eg a ion o hose pulses
o he comple e a ay o neu ons in he ecei e chip
implemen s a con olu ion ope a ion. In he es o he
pape we will concen a e on desc ibing he ci cui s able
oimplemen suchacon olu ionalo il e ingchip.
V. Sys em Design
Fig. 5 shows he basic ope a ing p inciple o he
p oposed a chi ec u e. The add ess bus p o ides he
coo dina es o heneu on(o pixel)a oundwhich
he ke nel o eq. (10) should be applied. Pulses will be
applied o all ows wi h y-coo dina e in he in e al
, and all columns wi h x-coo dina e in he
in e al , whe e is he wid h
conside ed o heke nel.
Table 1
ke nel pa ame e s NSE
(dB)
Gaussian -24.92
E en
Gabo -19.04
Odd
Gabo -19.03
Displaced
Gaussians -22.73
Fxy,()Hx()Vy()=
e
1
2
---– x
σx
-----


2
e
1
2
---– y
σy
-----


2σx10=
σy15=
e
1
2
---– x
σ
---


2
e
1
2
---– y
σ
---


2
2πy
ys
----


sin
σ15=
ys20=
e
1
2
---– x
σ
---


2
e
1
2
---– y
σ
---


2
2πy
ys
----


cos
σ15=
ys20=
e
1
2
---– x
σx
-----


2
e
1
2
---– yy
s
–
σy
-------------


2
e
1
2
---– yy
s
+
σy
-------------


2
–




σx15=
σy5=
ys5=
D1
D2
D3
CHIP1
ARBITER+ENCODER
DIGITAL
BUS
DECODER
D1
D2
D3
CHIP2
Fig. 4: Add ess E en Rep esen a ion In e chip Communica ion Scheme
x0y0
,()
y0L–y0L+,[]
x0L–x0L+,[]2L1+

Pulses will be modula ed in wid h acco ding o
unc ion (see eq. (8)) o he ows, and unc ion
o he columns. A each pixel he e is an AND
ga e which p o ides a pulse o wid h equal o he
minimumo and .Thispulsewillgene a ea
ixed magni ude cu en pulse o he same wid h which
will be in eg a ed on a capaci o . Each pixel con ains wo
in eg a o s. One o hem, called he “posi i e in eg a o ”,
in eg a es he pulse o leng h when
; while he o he , called he
“nega i e in eg a o ”, in eg a es he pulse when
. The alues o and
( ) a e s o ed digi ally on chip on a
smallRAM.
Fig. 6 shows he loo plan diag am o he sys em. I
consis s o wo inpu decode s ha decode he add ess o
he a i ing pulses, a C-elemen equi ed o he AER
communica ion p o ocol [1]-[3], an a ay o
in eg a o cells , wo se s o p og ammable
monos ables and
whose pulse wid hs a e con olled
by he bi s s o ed in wo RAMs, and
(which s o e he digi al wo ds and
, espec i ely), wo a ays o
and selec ingcells and
, espec i ely, wo ou pu decode s o selec he
min H x() Vy(),()
Vy()
Hx()
Hx()
y0
x0
x0
BUS X
BUS Y
y0
Fig. 5: Schema ic Rep esen a ion o he Basic Ope a ion
P inciple behind he p oposed A chi ec u e
Vy()
Vy()
Hx()
Vy() Hx()
min H x() Vy(),()
Hx()()sgn Vy()()sgn 0>
Hx()()sgn Vy()()sgn 0<Vx() Hy()
xy,L…0…L,,–=
c1M
c11
cNM
cN1
cij
x1xN
xi
Px+1Px-1Px+iPx-iPx+NPx-N
Mx-L
MxL
Tx-L
Sx-L
TxL
SxL
My-L
MyL
Ty-L
Sy-L
TyL
SyL
yM
yj
y1
Py+M
Py-M
Py+j
Py-j
Py+1
Py-1
Ack
Rqs
Vmonob
C-elemen
La ch X
La ch Y
Inpu Decode X
Inpu Decode Y
IM+
+
IM-
Ij+
Ij-
I1+
I1-
ScanM
Scanj
Scan1
Io
Ou pu Decode X
Ou pu Decode Y
Scx1ScxN
Scxi
ScyM
Scyj
Scy1
Cx1,-L
Cx1L
CxN,-L
CxNL
La ch
Rqs m
Rqs
Ack
La ch
Rqs
Ack
Mys
Mxl
CyM,-L
CyM,L Cy1,-L
Cyj0
Cxi0
Rqs m
RyL
Ry-L
Rys
Rxl
Rx-L
RxL
Random
Scan
Bus
8
4
4
Add ess
Bus
4
4
8
4
4
Cxi-l,l
Cyj-s,s
Cy1,-L
RAM Y
RAM X
Fig. 6: Floo plan o Comple e 2D Fil e ing Sys em
Tys
Sys
Txl
Sxl
NM×
cijMx L–... Mx0... MxL
,, ,,
My L–... My0... MyL
,, ,, RAM X RAM Y
Rx L–... Rx0... RxL
,, ,,
Ry L–... Ry0... RyL
,, ,,
2L1+()N×2L1+()M×Cxil–l,
Cyjs–s,
cells o be scanned, and a column o scanning ci cui s
o eadou hein eg a o sanalogou pu cu en .
No e ha in he p esen p o o ype o Fig. 6 he sys em
does no gene a e an AER ou pu . This can be sol ed by
ei he adding he necessa y ci cui y o each pixel [1]-[3]
which will dec ease he cell densi y, o by adding a pos -
p ocessing chip ha scans sequen ially all cells in he
a ayo Fig.6andgene a esanAERou pu .
The ope a ion o he sys em in Fig. 6 is as ollows. In
and digi alwo dso bi sa es o ed
( and ). The i s
bi (o )indica es hesigno he unc ion (o
). The ollowing bi s indica e he absolu e alue
(o ).These bi slinea lycon ol heleng h
o he pulse igge ed by monos ables (o ). The
pulses gene a ed by he monos ables a e sen h ough
lines (o )anda e igge ed whene e an ex e nal
pulsea i es o he sys em(whene e signalRqs pulses).
When an ex e nal pulse a i es, he inpu decode s
ac i a e lines and co esponding o he add ess o
he a i ing pulse. The selec ion cells con olled by
(cells in Fig. 6, ) connec he
pulse in line o line i he sign bi is ‘1’. I
hesignbi is‘0’line isconnec ed o henega i e
line .Thisway,pulses (o )a esen h ough
lines o ( o )dependingon he
signo heweigh s o edin (o ).
Each neu on has wo in eg a o s. The posi i e
in eg a o accumula es cha ge when pulses a e
simul aneously a i ing h ough ho izon al and e ical
lines o he same sign. Tha is, i in eg a es a pulse when
lines and (o lines and ) a e
simul aneously high, o equi alen ly i pe o ms he
ope a ion . Hence, he
posi i e in eg a o in cell compu es along ime he
ollowingsum
(12)
whe e , , is he (lossy)
in eg al o e ime o he numbe o pulses pixel
is ecei ing, and is he ixed magni ude o he cu en
pulses being in eg a ed.
Simila ly, henega i ein eg a o accumula escha ge
whenpulsesa i ing h oughho izon aland e icallines
o opposi e sign and (o and ) a e
simul aneously high, ha is, i pe o ms he ope a ion
. Hence, along ime i
compu es he ollowingsum
(13)
Consequen ly, he di e ence be ween he ou pu s o he
posi i e and nega i e in eg a o s is gi en by,
, (14)
which is he il e ope a ion we wan o implemen . In
wha ollows we will desc ibe he ci cui componen s
and ope a ions o each block in Fig. 6.
A.Communica ion P o ocol: The C-elemen
To pe o m a p ope communica ion be ween wo
chips a communica ion p o ocol mus be implemen ed
[1]-[3]. In he AER scheme, he sende chip indica es
when he add ess o a pulsing neu on is eady on he bus
and he ecei e chipmus acknowledge ha hepulsehas
been ecei edand ha i is eady o ecei eanewpulse.
Fig. 7 shows he iming diag am o a alid
communica ion p o ocol o he wo chips. The sende
chip gene a es a eques signal and he ecei e
gene a es an acknowledge signal . When he sende
has pu he add ess on he bus i pulls he eques signal
o a high alue. Once he ecei e de ec s a high
signal i la ches he ecei ed add ess and pulls he
acknowledge signal high. The sende can pu now
low and begin o p ocess he ollowing pulse. The
ecei e mus wai un il all he monos ables ha e sen
hei pulses o he co esponding neu ons and he
has gone low o pu he signal low. Once he
signal is low he sende can ac i a e he signal o a
high alue again. Fig. 8 shows he schema ic o he cell
used in he ecei e chip o gene a e he signal. This
cell is known as “C-elemen ”. This elemen ecei es wo
inpu signals: a eques signal gene a ed by he
sende sys em, and signal which is he wi ed-
NOR o all he monos able ou pu pulses
. The C-elemen gene a es an
ou pu acknowledge signal which is sen back o he
sende sys em. When no pulses a e being ecei ed,
is low. Signal is high as no pulses a e being
ScanjIo
RAM X RAM Y n 1+
Rx L–... Rxl... RxL
,, ,, Ry L–... Rys... RyL
,, ,,
SxlSysHx()
Vy() n
Hx() Vy() nMxlMys
TxlTys
xiyjxi
Cxil–l,lL…0…L,–=
TxlPxil–
+Sxl
SxlTxl
Pxil–
-TxlTys
Pxil–
+Pxil–
-Pyjs–
+Pyjs–
-
RxlRys
cij
Pxi
+Pyj
+Pxi
-Pyj
-
Pxi
+Pyj
+
∩()Pxi
-Pyj
-
∩()∪cij
Iij
+Iwmin H(xpi)V(yqj),()npq
pq,
Hx
pi
()()sgn Vy
qj
()()sgn 0>
∑
=
xpi xpxi
–= yqj yqyj
–= npq xpyq
,()
Iw
Pxi
+Pyj
-Pxi
-Pyj
+
Pxi
+Pyj
-
∩()Pxi
-Pyj
+
∩()∪
Iij
-Iwmin H(xpi)V(yqj),()npq
pq,
Hx
pi
()()sgn Vy
qj
()()sgn 0<
∑
=
IwHx
pi
()()sgn Vy
qj
()()sgn min H(xpi)V(yqj),()npq
pq,
∑
CHIP1 CHIP2
Add ess
Rqs
Ack
Rqs
Add ess Bus
Pulse
Ack
Sende Recei e
Fig. 7: Timing diag am o he add ess-e en
communica ion p o ocol
Bus
Rqs
Ack
Rqs
Rqs Ack
Rqs
Rqs
Ack Ack
Rqs
Ack
Rqs
Vmonob
Tx L–…TxLTy L–…TyL
,, , ,, Ack Rqs
Vmonob
gene a ed by he monos ables. Consequen ly, he
signal is low. When a alid add ess pulse a i es he
sende pu s he signal high. The ising edge o his
signal is used o igge he monos ables so ha
signal becomes low. Once is high and
has been se o low he C-elemen se s o a
high alue, meaning ha he pulse has been
ecei ed. The high alue o is used o la ch he
p esen bus add ess and signal ha igge s he
monos ables. La ching he add ess assu es ha he
co esponding neighbo hood is kep selec ed un il all
monos ables inish hei pulses. By la ching we
assu e ha he monos able pulses do no end i signal
goes low be o e he monos able pulses ha e
inished.TheC-elemen wai sun il goeslowandall
he monos able pulses inish ( ) o pu he
signal low again. Once he acknowledge is low he
sende is allowed o pull up again and a new
communica ioncyclecanbegin.
B.The Monos ables
The schema ic o a monos able wi h con olling
bi s is shown in Fig. 9. T ansis o s and a e
equally sized, as well as ansis o s and .
Swi ches a e con olled by a digi al n-bi wo d
ha se he capaci ance connec ed o node .
When no add ess is being ecei ed, and a e
bo h low and hence is also low. T ansis o is
cu o so ha node is low. Node is also se low
h ough hose ansis o s wi h a high alue. I all
bi s a e low will always be high (by ) and no
pulse will be gene a ed. When an inpu pulse a i es
signal and hence become high. As soon as
goeshighnode goeshigh.Cu en begins
o low h ough he swi ch o med by ansis o s and
cha ging node a he a e se by bi s . When
node eaches ol age alue , he cu en
h ough ansis o becomes highe han he cu en
supplied by so ha he ou pu node lips om
high o low. The leng h o he pulse a is he ime
aken by cu en o cha ge node up o a ol age o
.This imeisgi enby
(15)
whe e is he o al capaci ance p esen a node
and is se by he bi s s o ed in he
co esponding RAM wo d o . Wi h his
scheme, he leng h o he monos able pulses is linea ly
con olled be ween 0 and , wi h
being he numbe o bi s con olling each monos able
pulse leng h, and he uni capaci ance in Fig. 9. Fig.
10 depic s he pulse wid hs ob ained wi h Hspice e sus
he alue o he digi al con ol wo d, o a monos able
con olled by bi s. Fo his simula ion, alues o
, and we e used.
C.The Selec ion Cell
Fig. 11 depic s he schema ic o he selec ion cell
(o ) used o selec he neighbo hood o
cells whe e he monos able pulses ha e o be sen . Each
selec ioncell consis so woNANDga escon olling he
ga es o wo PMOS ansis o s ( and ) ha
beha e like swi ches, and wo NMOS pull down
ansis o s ( and wi h a cons an ga e ol age
).Eachselec ioncell( o example, inFig.6)
has wocon olsignals( hedecode ou pu and hesign
bi om ), one inpu signal ( he monos able
ou pu ) and wo ou pu s ( and ). When a
pulse a i es wi h add ess , i ac i a es he
decode s ou pu and , espec i ely. The decode
ou pu con ols all he selec ion cells wi h
. When is high, i he sign bi is ‘1’,
heselec ion cell connec s he monos ableou pu
line o he posi i e line . I he sign bi is
‘0’, line is connec ed o he nega i e line . The
sameis alid o he coo dina eselec ioncells.
D.The Co e Cell
Theschema ico cell o Fig.6isshowninFig.12.
I consis s o wo diode-capaci o in eg a o s [3]. The
Rqs
Vmonob
Ack
Ack
Fig. 8: Schema ic o he C-elemen used o he
a bi a ion o he add ess-e en inpu s
Ack
Rqs
Rqs Vmonob Rqs
Vmonob Ack
Rqs
Ack
Rqs m
Rqs m
Rqs Rqs
Vmonob 1=
Ack Rqs
n
Rqs m
Rqs m Rqs m
Rqs m
Rqs m
V hm
b1bn
Vm
IT
M1 M2
M5
M3 M4
Vou
Fig. 9: Schema ic o a Monos able Cell
S1Sn
20Cu2n-1Cu
Mb1Mbn
M6 M7
M8
OR{bi}VT
M1M2
M3M4
S1…Sn
,,
b1…bn
,, Vm
Ack Rqs
Rqs m M5
Vou Vm
Mbibi
biVmM8
Rqs Rqs m
Rqs m Vou ITM6
M7Vmbi
VmV hm
M2
M1Vou
Vou
ITVm
V hm
TCmono
IT
--------------- V hm
=
Cmono Vm
b1…bn
,,{}
RxlRys
2n1–()CuV hm/ITn
Cu
0 5 10 15 20 25 30 35
0
50
100
150
200
250
Fig. 10: Monos ables pulse leng h exp essed in nanoseconds
e sus alue o con olling digi al wo d ob ained h ough
Hspice simula ion
Pulse Wid h (ns)
n5=
Cu0.2pF=IT75µA=V hm 2.5V=
Cxil–l,Cyis–s,
MP+MP-
MN+MN-
VPD Cxil–l,
xi
SxlRAM X
TxlPxil–
+Pxil–
-
xiyj
,()
xiyj
xiCxil–l,
lL–…L,,[]∈xiSxl
Cxil–l,
TxlPxil–
+Sxl
TxlPxil–
-
Y
cij
posi i e in eg a o in eg a es he ANDED pulses ha
a i e in ow and column lines wi h he same sign, ha is,
and (o and ). The nega i e in eg a o
in eg a es he ANDED pulses ha a i e in ow and
column lines o opposi e sign, ha is, and (o
and ). Each diode-capaci o in eg a o consis s
o wo ansis o s and ( and )ope a ingin
he sub h eshold egion, a capaci o and a ansis o
(o ) ac ing as a cu en sou ce o alue
(con olled by bias ol age ) wi h i s sou ce pulsed by
heou pu o heNORga e.The inpu andou pu cu en s
and o he posi i e in eg a o a e ela ed o he
ol age a node h ough he ollowing di e en ial
equa ions( he ea men o henega i ein eg a o would
be hesame o cu en s , and ol age )[3],
(16)
(17)
whe e is he he mal ol age and , a e model
pa ame e s o he MOS ansis o ope a ing in he
sub h eshold egion. F om eqs. (16) and (17) we can ge
an exp ession ha ela es he ou pu and inpu cu en s
o he diode
, (18)
whe e
(19)
No e ha cu en mi o gain Ais con olled by ol age
. Du ing he ime in which and (o
and ) a e simul aneously high, he sou ce o
ansis o is low, and his ansis o is ac ing as a
cu en sou ce sinking a cons an cu en om he
in eg a ion node . In his case in eq. (18).
Suppose ha a ain o pulses o cons an equency
, pulse wid h and in e spike in e al (as
depic ed in Fig. 13) is applied simul aneously o lines
and (o and ). In eg a ing equa ion
(18) om o wi h , esul sin
, (20)
whe e he in eg a o ime cons an is gi en by
. When he ANDED pulses a e ze o, he
sou ce o ansis o becomes high and . I
he pulses go low a ime and s ay low o an
in e spike ime (see Fig. 13), he ou pu cu en a
ime jus be o e a new pulse is applied, is gi en
by
. (21)
When pulses o wid h a e applied a a cons an
equency as shown in Fig. 13, a s eady s a e is
eached in which he cha ge injec ed by he diode du ing
he inac i e pe iods equals he cha ge sank by he
xi
SxlTxl
Pxi-l+
Pxi-l-
VPD
VPD
Fig. 11: Schema ic o a neighbo hood selec ion cell
MP+
MP-
MN+
MN-
Pxi+
Pxi-
Pyj+
Pyj-
Vw
VA
Scxi
Pxi+
Pxi-
Pyj-
Pyj+
Vw
VA
ScxiIj-
VREF
C
C
Iij-
Fig. 12: Schema ic o he Co e Cell wi h he wo
diode-capaci o in eg a o s
g-
g+
Mw+
Mw-
M1+M2+
M1-M2-
Iin+
Iin-
VREF
Iij+
Ij+
Pxi
+Pyj
+Pxi
-Pyj
_
Pxi
+Pyj
-
Pxi
-Pyj
+
M1
+M2
+M1
-M2
-
C
Mw
+Mw
-Iw
Vw
Iin
+Iij
+
g
+
Iin
-Iij
- g
-
Iin
+C g
+
d
d
--------
–Iop
VAκ g
+
–
----------------------



exp+=
Iij
+Iop
Vdd κ g
+
–
-------------------------



exp=
Iop κ
QT
Iij
+
d
d
-------- Iij
+Iin
+1
A
--- Iij
+
–


=
AVdd VA
–
-----------------------


exp=
QT
C
κ
--------=
VAPx+iPy+jPx-i
Py-jMw+
Iw
g
+Iin
+Iw
=
1T⁄ThTl
Px+iPy+jPx-iPy-j
Pxi+
1 2 2+Tl
ThTl
T
nT (n+1)T
Fig. 13: Timing diag am o he pulses applied o lines
, , o Pxi
+Pxi
-Pyj
+Pyj
-
1 2Iin
+Iw
=
1
Iij
+ 1Th
+()
----------------------------1
AIw
----------1
Iij
+ 1
()
---------------1
AIw
----------–


Th
τ
------–


exp+=
τC /κIw
=
Mw+Iin
+0=
2
Tl
2Tl
+
1
Iij
+ 2Tl
+()
---------------------------1
Iij
+ 2
()
---------------Tl
AQT
-----------+=
Th
1T⁄