ma e ials
Re iew
Neu omo phic Spiking Neu al Ne wo ks and Thei
Mem is o -CMOS Ha dwa e Implemen a ions
Luis A. Camuñas-Mesa * , Be nabé Lina es-Ba anco and Te esa Se ano-Go a edona
Ins i u o de Mic oelec ónica de Se illa (IMSE-CNM), CSIC and Uni e sidad de Se illa, 41092 Se illa, Spain
*Co espondence: [email p o ec ed]
Recei ed: 5 July 2019; Accep ed: 10 Augus 2019; Published: 27 Augus 2019


Abs ac :
Inspi ed by biology, neu omo phic sys ems ha e been ying o emula e he human b ain o
decades, aking ad an age o i s massi e pa allelism and spa se in o ma ion coding. Recen ly, se e al
la ge-scale ha dwa e p ojec s ha e demons a ed he ou s anding capabili ies o his pa adigm o
applica ions ela ed o senso y in o ma ion p ocessing. These sys ems allow o he implemen a ion o
massi e neu al ne wo ks wi h millions o neu ons and billions o synapses. Howe e , he ealiza ion
o lea ning s a egies in hese sys ems consumes an impo an p opo ion o esou ces in e ms
o a ea and powe . The ecen de elopmen o nanoscale mem is o s ha can be in eg a ed wi h
Complemen a y Me al–Oxide–Semiconduc o (CMOS) echnology opens a e y p omising solu ion
o emula e he beha io o biological synapses. The e o e, hyb id mem is o -CMOS app oaches
ha e been p oposed o implemen la ge-scale neu al ne wo ks wi h lea ning capabili ies, o e ing a
scalable and lowe -cos al e na i e o exis ing CMOS sys ems.
Keywo ds:
neu omo phic sys ems; spiking neu al ne wo ks; mem is o s; spike- iming-dependen
plas ici y
1. In oduc ion
The ou s anding e olu ion o compu e s du ing he las 50 yea s has been based on he
a chi ec u e p oposed by Von Neumann in he 1940s [
1
]. In his model o s o ed-p og amme compu e ,
da a s o age and p ocessing a e wo independen asks pe o med in sepa a ed a eas wi h a high
need o da a communica ion be ween hem. Wi h he de elopmen o in eg a ed ci cui s, Go don
Moo e p edic ed in he 1960s ha he numbe o ansis o s in an in eg a ed ci cui would double
e e y
18 o 24 mon hs
[
2
]. This exponen ial e olu ion allowed o he de elopmen o mo e e icien
compu ing sys ems, wi h inc easing p ocessing speed and dec easing powe consump ion. Howe e ,
e en he cu en echnologies o semiconduc o manu ac u ing a e eaching he limi s o Moo e’s
law [
3
],
so di e en
solu ions ha e been p oposed o keep he u u e e olu ion o p ocessing sys ems [
4
].
Two di e en
s a egies sugges he de elopmen o new p ocessing pa adigms and no el de ices
beyond con en ional Complemen a y Me al–Oxide–Semiconduc o (CMOS) echnologies.
In pa allel wi h he de elopmen o compu ing pla o ms, in he 1960s some esea che s used he
eme ging elec onic echnologies as a mechanism o modeling neu al sys ems, om indi idual
neu ons [
5
–
10
] o mo e complex ne wo ks [
11
]. The inc easing unde s anding o he s uc u e
and undamen al p inciples o beha io o he human b ain e ealed a e y di e en p ocessing
pa adigm om he adi ional compu e a chi ec u e wi h a much be e pe o mance. E en when
compa ing wi h cu en supe compu e s which excel a speed and p ecision, he human b ain is s ill
much mo e powe ul when dealing wi h no el y, complexi y and ambigui y o p ac ical asks like
isual ecogni ion and mo ion con ol, while p esen ing a negligible powe consump ion a ound
20W [
12
]. This compa ison be ween con en ional compu e s and he b ain led o he eme gence o
neu omo phic compu ing. The e m neu omo phic enginee ing was i s coined by Ca e Mead
Ma e ials 2019,12, 2745; doi:10.3390/ma12172745 www.mdpi.com/jou nal/ma e ials
Ma e ials 2019,12, 2745 2 o 28
o e e o de eloping mic oelec onic in o ma ion p ocessing sys ems mimicking he ope a ion o
hei biological coun e pa s [
13
,
14
]. Du ing he 1980s, Ca e Mead highligh ed he analogy be ween
he physics in biological neu ons and he beha io o ansis o s in sub- h eshold egime [
13
,
14
],
de eloping neu al ne wo ks based on analog ci cui s; leading o he implemen a ion o he i s
silicon e inas [
15
] and p oposing a new compu ing pa adigm whe e da a and p ocessing asks a e
pe o med by indi isible en i ies, aking inspi a ion om biological neu al sys ems. Along he yea s,
he neu omo phic enginee ing ield has b oaden i s inspi a ion. Today’s neu omo phic compu ing
enginee s no only y o mimic he highly pa allel a chi ec u e o biological b ains and he use
o in-memo y compu ing a chi ec u es as a way o imp o ing he speed and ene gy pe o mance,
bu also ha e deeply s udied he signal in o ma ion encoding, compu a ional p inciples and lea ning
pa adigms ha enable e en simple biological b ains wi h admi ing pe o mance in he in e ac ion and
adap a ion o complex and unexpec ed en i onmen s wi h high eac ion speeds and minimal powe
consump ion despi e elying on e y simple and highly un eliable compu a ion uni s [16].
Al e na i ely, many no el beyond-CMOS echnologies ha e been p oposed o o e come he
limi s o Moo e’s law. One o he mos p omising a ailable de ices is he nanoscale mem is o .
The mem is o was i s desc ibed heo e ically by Chua in he 1970s as he ou h passi e elemen
es ablishing a ela ionship be ween elec ic cha ge and magne ic lux [
17
]. Much la e in 2008,
a eam a HP Labs claimed o ha e ound Chua’s mem is o expe imen ally based on a hin ilm o
i anium oxide [
18
]. This 2- e minal de ice beha es as a a iable esis o whose alue can be modi ed
by applying ce ain ol ages o cu en s. The mos common s uc u e o his de ice is a union
me al-dielec ic(s)-me al, whe e he dielec ic laye can be as hin as a ew nanome e s. The applica ion
o elec ic ields and con olled cu en s ac oss he dielec ic p oduces an al e a ion o i s esis ance
by g owing a ilamen o o he mechanisms like ba ie modula ion.
Cu en ly a ailable
mem is o s
a e mos ly bina y de ices, as hey can swi ch be ween wo esis ance alues: HRS (High-Resis ance
S a e) and LRS (Low-Resis ance S a e) [
19
]. Since he appea ance o he mem is o s, many logic
amilies based on mem is o s o digi al compu a ion ha e been p oposed [
20
,
21
], hei po en ial as
digi al long- e m non- ola ile memo y echnology has also been demons a ed [
22
–
25
], and hei
use as biosensing de ices looks also p omising [
26
]. In he ield o neu omo phic enginee ing,
he mem is o s ha e a ac ed a special in e es due o i s pa icula plas ici y beha iou which
essembles he adap a ion ules obse ed in biological synapses. Mem is o s can adap and change i s
beha iou o e ime in esponse o di e en s imula ion pa e ns as i happens in he human b ain.
In pa icula , i has been demons a ed ha i s imula ed wi h pulse- ains simula ing he inpu om
spiking neu ons, mem is o s may exhibi a biologically inspi ed lea ning ule [27–30] esembling he
spike- iming-dependen plas ici y (STDP) obse ed in biological neu ons [
31
–
36
]. Hence, mem is o s
ha e been conside ed as a i icial ino ganic synapses.
In his pape , we analyze he cu en end owa ds using mem is o s o e CMOS pla o ms
o implemen neu omo phic sys ems, demons a ing a new pa adigm which o e comes cu en
limi a ions in con en ional p ocessing sys ems. In Sec ion 2, we gi e a gene al o e iew o
he basis o neu omo phic compu ing, while in Sec ion 3we e iew he main la ge-scale CMOS
ha dwa e implemen a ions o neu omo phic sys ems. In Sec ion 4, we desc ibe p oposed hyb id
Mem is o -CMOS app oaches, while in Sec ion 5we emphasize he sui abili y o his s a egy o
implemen lea ning algo i hms in neu al sys ems. Finally, in Sec ion 6we gi e ou u u e pe spec i e
o his ield.
2. Neu omo phic Compu ing
As al eady s a ed, neu omo phic compu ing sys ems ake inspi a ion on he a chi ec u e,
he echnology and he compu a ional p inciples o biological b ains. Mo phologically, he human
b ain is composed o app oxima ely 10
11
elemen a y p ocessing uni s called neu ons, massi ely
in e connec ed by plas ic adap able in e connec ions called synapses. Each neu on connec s
app oxima ely o 10
3
–10
4
o he neu ons h ough synap ic connec ions. The neu ons a e known
Ma e ials 2019,12, 2745 3 o 28
o be dis ibu ed in laye s, and mos o he synap ic in e connec ions a e de o ed o in e connec
neu ons belonging o successi e laye s.
The i s compu ing sys ems inspi ed by his s uc u e o biological b ains we e published in he
1940s–1950s and we e called A i icial Neu al Ne wo ks (ANNs) [
37
,
38
]. They appea ed as powe ul
compu a ional ools ha p o ed o sol e, by i e a i ely aining algo i hms ha adap ed he s eng h
o he in e connec ion weigh s, complex pa e n ecogni ion, classi ica ion o unc ion es ima ion
p oblems no amenable o be sol ed by analy ic ools. The i s gene a ions o neu al ne wo ks did no
in ol e any no ion o ime no any empo al aspec in he compu a ion.
Mc Culloch and Pi s, p oposed in 1943, one o he i s compu a ional models o he biological
neu ons. Figu e 1illus a es he ope a ion o each p oposed neu al compu a ional uni . As illus a ed
in Figu e 1, a neu on
Nj
ecei es inpu s om
n
o he p e ious neu ons
x1
,
x2
, ...,
xn
. The ou pu
o each neu on
x1
,
x2
, ...,
xn
in he p e ious laye is mul iplied by he co esponding synap ic
weigh
w1j
,
w2j
, ...,
wnj
, also know as synap ic e icacy. The combined weigh ed inpu is ans o med
ma hema ically using a ce ain non-linea ans e unc ion o an ac i a ion unc ion
ϕ
, gene a ing an
ou pu
oj
. In he o iginal Mc Culloch and Pi s’ neu al model he ac i a ion unc ion was a h esholding
ga e, gi ing as neu al ou pu a digi al signal [
37
]. This digi al ou pu neu on was he co e o he i s
gene a ion o neu al ne wo ks.
Figu e 1.
Diag am o an a i icial neu on wi h n inpu s wi h hei co esponding synap ic weigh s.
All weigh ed
inpu s a e added and an ac i a ion unc ion con ols he gene a ion o he ou pu signal.
In 1958, Rosenbla p oposed he pe cep on. The a chi ec u e o he pe cep on is shown in
Figu e 2a. In Figu e 2, he compu a ional uni s o neu ons a e ep esen ed by ci cles, in e connec ed
h ough ainable weigh s ep esen ing he synap ic connec ions. The o iginal pe cep on consis ed o
a single laye o inpu neu ons ully in e connec ed in a eed o wa d way o a laye o ou pu neu ons.
A lea ning hebbian ule [
39
] o adap he weigh s was p oposed [
38
]. This single laye pe cep on was
able o sol e only linea ly sepa able p oblems [40].
In he 1950–60s, a second gene a ion o compu a ional uni s a ose we e he h esholding ac i a ion
unc ion was eplaced by a con inuous analog alued ou pu like a smoo h sigmoid, adial basis
unc ion o a con inuous piece-wise linea unc ion [
41
,
42
]. Recen ly, he ec i ying non-linea ac i a ion
unc ion, also known as ReLU has become e y popula o i s be e aining con e gence and i s
ha dwa e iendly implemen a ion [
43
]. Fu he mo e, g adien descen based lea ning algo i hms
could be now applied o op imize he ne wo k weigh s. Al e na i e lea ning ules we e p oposed
as he del a ule based on he Leas Mean Squa es (LSM) algo i hm published by Wid ow [
44
,
45
].
This second gene a ion p o ed o be uni e sal app oxima o s o any analog con inuous unc ion,
ha is, any analog con inuous unc ion could be app oxima ed by a ne wo k o his ype wi h a single
hidden uni [41].
Ma e ials 2019,12, 2745 4 o 28
Figu e 2.
(
a
) A chi ec u e o a single laye pe cep on. The a chi ec u e consis s o a laye on inpu
neu ons ully connec ed o a single laye o ou pu neu ons. (
b
) Ex ension o a mul i-laye pe cep on
including mo e han one laye o ainable weigh s. In his example, he ne wo k includes 3 laye s:
inpu , hidden and ou pu laye . Each connec ion be ween wo neu ons is gi en by a ce ain weigh .
The backp opaga ion algo i hm ex ended he applica ion o he g adien descen echniques
o ne wo ks wi h any numbe o hidden laye s, popula ly known as Deep Neu al Ne wo ks
(DNNs) [46–48].
Figu e 2b illus a es a case wi h 3 laye s: a i s laye o inpu neu ons, a second
laye o hidden neu ons, and a hi d laye o ou pu neu ons, al hough a gene al a chi ec u e can
con ain any gi en numbe o hidden laye s.
The ANN a chi ec u es shown in Figu e 2a,b a e pu e eed o wa d a chi ec u es as he signal
p opaga es om inpu o ou pu in an unidi ec ional way. O he a chi ec u es, known as ecu en
neu al ne wo ks, including eedback connec ions om uppe laye s in he a chi ec u e o lowe
laye s, ha e been p oposed. The Adap i e Resonance Theo y (ART) a chi ec u es by G ossbe g [
49
],
he Kohonen sel -o ganizing maps [
50
] o he Hop ield models [
51
] can be ci ed among he
pionee ing ones.
The p esen ed ANNs ha e been ypically de eloped in so wa e, and ained o line. The aining
o DNNs equi es a as amoun o anno a ed da a o co ec ly gene alize he p oblem wi hou
o e i ing [
52
] and in ensi e compu a ion esou ces. Howe e , in ecen yea s, he inc ease in he
compu a ion capabili ies o mode n compu e s and he a ailabili y o as amoun s o in o ma ion
ha e made DNN e y popula allowing he de elopmen o many DNN-based applica ions [
53
,
54
]
ha use complex a chi ec u es like LeNe o handw i en digi ecogni ion [
55
], Mic oso ’s speech
ecogni ion sys em [
56
] o AlexNe o image ecogni ion [
43
]. As a consequence we ha e wi nessed
he explosion o DNNs and machine lea ning.
Despi e he imp essi e ad ances ha DNNs ha e demons a ed in ecen yea s, hei pe o mance
in e ms o e iciency (speed and powe consump ion) compa ed wi h he human b ain is s ill low
as i is low hei esemblance o he human b ain in e ms o in o ma ion coding. In he biological
b ain, he in o ma ion is p ocessed in a con inuous way in ime, no jus as a sequence o s a ic ames
as DNNs ecogni ion sys ems do. Fu he mo e, in con en ional DNNs, he ou pu o he di e en
neu al laye s a e compu ed in a sequen ial way. Each laye has o wai un il he ou pu o he p e ious
laye has been compu ed o pe o m i s compu a ion, hus in oducing a signi ican ecogni ion delay
in he ne wo k. On he con a y, biological neu ons ansmi hei in o ma ion o he nex neu onal
laye s in he o m o spikes. Whene e a neu on emi s a spike, he spike is ansmi ed o i s a e en
connec ed neu ons and p ocessed wi h jus he delay o he synap ic connec ion. In 1996, Tho pe
demons a ed ha he human b ain was able o ecognize a isual amilia objec in he ime ha
jus one spike p opaga es h ough all he laye s o he isual co ex [
57
]. Simila isual p ocessing
Ma e ials 2019,12, 2745 5 o 28
speeds ha e been measu ed in he macaque monkeys by Rolls [
58
]. These expe imen s e eal an
ex emely e icien in o ma ion coding in he biological b ains. In his con ex , he 3 d gene a ion o
neu al ne wo ks, spiking neu al ne wo ks (SNNs), aims o b idge he gap be ween neu oscience and
machine lea ning, using biologically- ealis ic models o neu ons o ca y ou in o ma ion coding and
compu a ion ying o ully exploi he e iciency in he spa io- empo al signal coding and p ocessing
and he co esponding powe e iciency obse ed in he biological b ains. SNNs ope a e using spikes
in a simila way as biological neu ons do. Tha way, in addi ion o he s a e o he neu on and he
synap ic weigh , SNNs also inco po a e he concep o ime in o hei model o ope a ion. In hese
neu ons, he e is no p opaga ion cycle, so each neu on i es an ou pu spike only when i s s a e
eaches a ce ain h eshold. The e o e, he in o ma ion lows in hese ne wo ks a e spike ains which
p opaga e be ween neu ons asynch onously, and empo al co ela ion be ween spikes is c ucial [
41
].
Spike ains o e he possibili y o exploi ing he ichness o he empo al in o ma ion con ained in
eal-wo ld senso y da a. This allows SNNs o be applied o sol e asks which dynamically changing
in o ma ion like isual ges u e ecogni ion o speech ecogni ion in a mo e na u al way han cu en
con en ional (non spiking) a i icial in elligen sys ems do. When dealing wi h dynamic in o ma ion
(as ideo sequences), con en ional a i icial sys ems pe o m compu a ions using sequences o s a ic
images sampled a a cons an pe iodic ime (pho og am ime in he case o ision). Recogni ion o
dynamic sequences may in ol e he use o ecu en neu al ne wo k a chi ec u es o he esolu ion o
con inuous ime di e en ial equa ions. These compu a ions a e qui e in ensi e using con en ional
amed ANN. Howe e , he use o SNN whe e compu a ion is d i en in a con inuous ime way
na u ally and d i en only by he occu ence o spikes de ec ing ce ain spa io- empo al co ela ions
can be much mo e ad an ageous.
Many di e en coding me hods o hese spike ains ha e been p oposed. Many au ho s ha e
p oposed o code he ac i i y le el o he neu ons as he equency o he i ing a e. Howe e , his ype
o coding does no bene i om he spike spa si y ha should cha ac e ize SNN p ocessing and hus,
i does no enable he co esponding low powe communica ion and compu a ion due o he spa si y o
he spike coding. Rega ding he as compu a ion capabili y expec ed om SNN, his i ing a e coding
in oduces a la ency in he compu a ion o he ou pu i ing a e. Fu he mo e, i is no biologically
plausible as e idenced by he expe imen s o Tho pe [
57
] and Rolls [
58
] which demons a ed ha he
compu a ion o a single co ical a ea is comple ed in 10–20 ms while he i ing a e o he neu ons
in ol ed in he compu a ion is below 100 Hz, which does no make possible he compu a ion based on
he coding o analog a iables in i ing a es. Howe e , as discussed by Tho pe e al. [
59
], he e a e
many o he biologically plausible and mo e e icien coding s a egies. O he coding schemes ha ha e
been conside ed a e in he iming be ween spikes [
60
], in he delay ela i e o a gi en synch oniza ion
ime also known as ime o i s spike (TFS) [
59
] encoding, jus coding he alues in he o de o spikes
which is known as ank o de coding [61], o synch onous de ec ion coding [59].
Rega ding he SNN neu on models, he e a e many neu on models ha desc ibe he beha iou o
biological neu ons wi h di e en le els o complexi y [
5
–
10
]. The classic Hodgkin-Huxley model [
5
] is
a 4- h o de biophysical model ha desc ibes he beha iou o he cu en s lowing in o he neu on ion
channels in a biologically ealis ic way. Howe e , due o i s complexi y, di e en 2nd o de simpli ied
models ha e been p oposed like he one p oposed by Fi zHugh and Nagumo [
6
,
7
] and he Mo is-Leca
model [
8
], among o he s. In he las yea s, he Izhike ich model [
10
] and he Adap i e Exponen ial
In eg a e and Fi e (AdEx) model [
9
] ha e become e y popula o hei abili y o ep oduce a la ge
a ie y o spiking egimes obse ed in he biological neu ons jus by a ying a educed numbe o
model pa ame e s. Howe e , while de ailed biophysical models can ep oduce elec ophisiological
ac i i y o biological neu ons wi h g ea accu acy, hey a e di icul o analyze compu a ionally and no
iendly o ha dwa e implemen a ions. Because o hese easons, o compu a ional pu poses simple
i s -o de phenomenological models like he In eg a e and Fi e model a e equen ly used.
The beha io o a single in eg a e-and- i e spiking neu on is illus a ed in Figu e 3. A spiking
neu on ecei es inpu spikes om se e al dend i es and sends ou spikes om i s ou pu axon,

Ma e ials 2019,12, 2745 6 o 28
as shown in Figu e 3a. E e y ime an inpu spike a i es, he s a e o he neu on is upda ed, and when
i eaches he h eshold, i gene a es an ou pu spike and ese i s s a e, as seen in Figu e 3b. In his
case, spikes a e ully cha ac e ized by hei i ing ime. In Figu e 3, i can be obse ed ha he e is a
cons an slope decay o he memb ane po en ial be ween wo a i ing spikes as i is he case o a leaky
in eg a e and i e neu on. Ma hema ically, a leaky in eg a e-and- i e neu on can be desc ibed as:
iin( ) = mem( )− es
R+Cd mem( )
d (1)
whe e
mem( )
ep esen s he memb ane po en ial,
iin( )
he injec ed cu en ,
es
he es ing
alue o he memb ane po en ial,
C
he equi alen capaci ance o he memb ane, and
R
he leak
esis ance. A leaky in eg a e-and- i e neu on can be easily implemen ed in ha dwa e ollowing he
esis ance-capaci ance (RC) " ex book" concep scheme p esen ed in Figu e 4, whe e an inpu cu en
iin
is in eg a ed in capaci o
C
wi h leak esis ance
R
. The in eg a ed ol age
mem
is compa ed wi h
a e e ence
h
, gene a ing an ou pu gi en by
ou
. Addi ionally, in eg a e-and- i e neu ons may
conside a e ac o y pe iod ha o ces a minimum ime in e al be ween wo consecu i e spikes o a
neu on. A comp ehensi e o e iew o ci cui ealiza ions o spiking neu ons wi h di e en le els o
complexi y can be ound in [62].
Figu e 3.
Illus a ion o he beha io o a leaky in eg a e-and- i e spiking neu on. (
a
) A spiking neu on
ecei es spikes om se e al inpu s, p ocesses hem, and gene a es ou pu spikes om i s ou pu node.
(
b
) Tempo al e olu ion o he neu on s a e while i ecei es inpu spikes. When he h eshold is eached,
i gene a es an ou pu spike.
Figu e 4. Example o a ha dwa e implemen a ion o an RC leaky in eg a e-and- i e neu on.
Ma e ials 2019,12, 2745 7 o 28
In e ms o connec i i y, he mos gene al ype o neu al ne wo k is ully connec ed, meaning
ha each single neu on in laye
i
is connec ed o all neu ons in laye
i+
1. This scheme applies no
limi a ion o he lea ning capabili ies o he ne wo k; howe e , i p esen s some di icul ies o p ac ical
implemen a ions. A e y popula way o educing he amoun o in e connec ions is ep esen ed by
Con olu ional Neu al Ne wo ks (Con Ne s), whe e each neu on in laye
i
is connec ed o a subse
o neu ons in laye
i+
1 ep esen ing a p ojec i e ield. This ecep i e ield can be ep esen ed as
a con olu ional ke nel, wi h sha ed weigh s o each laye [
63
]. This scheme is inspi ed by biology,
as i has been obse ed in he isual co ex [
64
]. In a simila way o he biological isual co ex,
his con olu ional
neu al ne wo k a chi ec u e is commonly used o image p ocessing applica ions in
he ea lie mo e massi e pa allel ea u e ex ac ion laye s, as i implies an impo an educ ion o he
numbe o connec ions.
Table 1(adap ed om [
65
]) con ains a compa ison o he main dis inc i e ea u es be ween ANNs
and SNNs. As p e iously s a ed, he la ency in each compu a ion s age in an ANN is high as he whole
compu a ion in each s age has o be comple ed on he inpu image o gene a e he co esponding
ou pu . On he con a y, in an SNN p ocesso he compu a ion is pe o med spike by spike so ha ,
ou pu spikes in a compu a ional laye a e gene a ed as soon as enough spikes e idencing he exis ence
o a ce ain ea u e has been collec ed. In ha way, he ou pu o a compu a ion s age is a low o spikes
ha is almos simul aneous wi h i s inpu spike low. This p ope y o SNN sys ems has been called
“pseudo-simul anei y” [
65
,
66
]. The la ency be ween he inpu and ou pu spike lows o a p ocessing
SNN con olu ion laye has been measu ed o be as low as 155 ns [
67
]. Rega ding he ecogni ion speed,
whe eas in an ANN he ecogni ion speed is s ongly dependen on he compu a ion capabili ies o
he ha dwa e and he numbe o o al ope a ions o be compu ed (which is dependen on he sys em
complexi y), in an SNN, each inpu spike is p ocessed in almos eal ime by he p ocessing ha dwa e
and he ecogni ion is pe o med as soon as he e a e enough inpu e en s ha allow he sys em o
ake a decision. This ecogni ion speed s ongly depends on he inpu s a is ics and signal coding
schemes as p e iously discussed. In e ms o powe consump ion, he ANNs powe depends on he
consump ion o he p ocesso and he memo y eading and w i ing ope a ions bu o a gi ing inpu
sampling equency and size does no depend on he pa icula isual s imulus. Howe e , in an SNN,
he powe consump ion depends also s ongly on he s a is ics o he s imulus and coding s a egies.
I e icien coding s a egies a e used, he sys em should bene i om he powe e iciency o spa se
spike ep esen a ions.
On he nega i e side, as i has been al eady poin ed ou , he addi ion o he ime a iable
makes SNN neu on models mo e complex han ANN ones. Also, as he compu a ion o ANN is
ime-sampled, in each sampling ime he algo i hmic compu a ion is pe o med using he a ailable
ha dwa e esou ces ha can be ime mul iplexed by e ching da a and s o ing in e media e a iables.
Howe e , in ue SNN he spikes should be p ocessed as hey a e gene a ed in eal ime, equi ing
pa allel ha dwa e esou ces which canno be mul iplexed. The scaling up o he sys em can be done by
modula expansion o he ha dwa e esou ces.
Howe e , whe e SNN should ha e majo ad an age is in applica ions equi ing ecu en neu al
a chi ec u es, such as, in ecogni ion o dynamic s imulus. The compu a ion o ecu en connec ions in
ANN equi es compu a ionally in ensi e i e a ions un il con e gence is eached, while he con e gence
o ecu en connec ions in SNN is almos ins an aneous due o hei pseudo-simul anei y p ope y.
In e ms o accu acy, as i will be discussed in Sec ion 5, he lea ning me hods ha ha e been
de eloped o ANN a e no di ec ly applicable o SNN. Al hough he lea ning heo y o SNN s ill lacks
behind i s equi alen me hods o ANN, some ecen wo k epo s o he same a chi ec u e an e o
inc emen o only 0.15% o he ImageNe da ase and 0.38% o he CIFAR10 da ase [
68
]. Howe e ,
he empo al dependence in oduces complexi y so ha once a SNN has been ained, i s accu acy
d ops i he inpu empo al coding changes. Bu i also in oduces he po en ial o ecognize dynamic
sequences in a mo e e icien way.
Ma e ials 2019,12, 2745 8 o 28
Table 1. Table compa ing di e en ea u es o ANNs and SNNs.
Fea u e ANN SNN
Da a p ocessing F ame-based Spike-based
La ency High Low
Pseudo-simul anei y
Time esolu ion Low High
P ese a ion o spa io- empo al
co ela ion
Time p ocessing Sampled Con inuous
Neu on model complexi y Low High
Recogni ion accu acy Highe Lowe
Ha dwa e mul iplexing Possible No possible
Sys em scale-up Ad hoc Adding modules
Recogni ion speed Low High
Independen on inpu s imulus Dependen on inpu s a is ics
Dependen on ha dwa e esou ces
Dependen on sys em complexi y No dependen on sys em complexi y
Powe consump ion De e mined by p ocesso powe De e mined by powe -pe -e en
and memo y e ching p ocessing in modules
Independen on inpu s imulus Dependen on s imulus s a is ics
Recu en opologies Need o i e a e un il con e ge Ins an aneous
3. CMOS Neu omo phic Sys ems
Simula ing SNNs on no mal ha dwa e is e y compu a ionally-in ensi e since i equi es simula ing
coupled di e en ial equa ions o la ge neu on popula ions unning in pa allel. Fully exploi ing he
coding and compu a ion capabili ies o biological b ains equi es he adequacy o he co esponding
ha dwa e pla o m o he peculia i ies o he algo i hm a di e en le els: om signal coding up o high
le el a chi ec u es. A he a chi ec u al le el, he in insic pa allelism o neu al ne wo ks lends o he
de elopmen o neu omo phic cus om pa allel ha dwa e esembling he a chi ec u e o he biological
b ain o emula e i s compu ing capabili ies [
62
,
69
,
70
]. Fu he mo e,
a he signal
le el, SNNs a e be e
sui ed han ANNs o ha dwa e implemen a ion, as neu ons a e ac i e only when hey ecei e an
inpu spike, educing powe consump ion and simpli ying compu a ion.
One o he majo issues when ying o implemen in a pa allel ha dwa e la ge a ays o neu al
popula ions is he implemen a ion o he synap ic in e connec ions. In a pa allel 2D ha dwa e,
he physical wi ing does allow o implemen connec ions be ween jus neighbou ing neu ons,
while he
biological neu ons a e dis ibu ed in 3D and massi ely in e connec ed among popula ions.
Add ess-E en -Rep esen a ion (AER) [
71
] is an asynch onous communica ion p o ocol ha was
concei ed o massi ely in e connec neu on popula ions ha can be loca ed in he same o di e en
chips as a ‘ i ual wi ing’ sys em. Figu e 5illus a es wo neu al popula ions communica ed h ough
an AER bus. In he pa icula case o his igu e, neu ons in he emi e popula ion code hei ac i i y
as a densi y o ou pu pulses which is p opo ional o hei ac i a ion le el. Howe e , he AER
communica ion scheme can be applied o any ype o pulse signal encoding [
59
]. Whene e a neu on
in he emi e popula ion gene a es a spike, i codes i s physical coo dina es
(x
,
y)
o add ess in a
digi al wo d in a as digi al bus and ac i a es an asynch onous eques (Rqs ) signal. The coded
add ess is sen h ough he as digi al bus o he ecei e popula ion. Upon ecep ion o an ac i e
eques , he ecei e decodes he a i ing neu on add ess and ac i a es he acknowledge (Ack) signal.
The ecei ed pulse can be sen o he co esponding neu on whe e he o iginal ac i i y o he sending
neu ons can be ep oduced (as illus a ed in Figu e 5) o o a g oup o i ually connec ed neu ons in
he ecei ing popula ion implemen ing a p ojec ion ield [
72
]. The high-speed o he in e -popula ion
digi al bus (in he o de o nanoseconds) compa ed o he in e spike in e al o biological neu ons
(in he o de o milliseconds) allows o mul iplex he connec ions o a million neu ons in a sha ed
ime-mul iplexed digi al bus. Mos o he de eloped la ge-scale CMOS neu omo phic compu ing
Ma e ials 2019,12, 2745 9 o 28
pla o ms make use o his AER communica ion p o ocol. As neu omo phic sys ems ha e scaled up in
size and a chi ec u al complexi y, many a ia ions o he o iginal poin - o-poin AER communica ion
scheme [
71
,
73
,
74
] ha e been p oposed ying o imp o e he o e all sys em communica ion bandwid h.
The b oadcas -mesh-AER [
75
–
77
] p oposes a gene ic app oach o in e connec a mesh o AER de ices
using a global mappe and in e connec ing he de ices in a chain a chi ec u e. The p e-s uc u ed
hie a chical AER app oach [
78
] uses he knowledge o he ne wo k opology o in e connec AER
de ices h ough di e en AER links. Mappe s can be used in e e y link, howe e , once he
AER de ices ha e been physically in e connec ed he changes in he con igu a ion a e limi ed.
The Hie a chical-F ac al AER [
79
] p oposes di e en le els o in e connec ion by adding add ess
bi s a highe le el based on he idea ha he a ic o spikes is going o be mo e in ense a a
local le el. The ou e -mesh AER [
80
] p oposes o a oid an ex e nal mappe by placing a ou e
wi h a mapping able inside e e y AER module aking ideas om adi ional NoC opologies [
81
].
The mul icas ing-mesh AER app oach [
82
] p oposes a simpli ica ion o he ou e -mesh AER by
employing ou ing ables ha con ain only in o ma ion o he connec i i y be ween modules ins ead
o allowing ull neu on o neu on connec i i y p og amming. Ano he app oach de eloped o allow
p og ammable in e connec ions inside he same chip o a wa e scale has been o implemen massi e
p og ammable c oss-poin in e connec s o con igu e he ne wo k opology [
83
] and including o -wa e
e ou ing o longe ange in e connec s [
84
]. Recen ly, he Hie a chical Rou ing AER has been
p oposed ha es ablishes di e en hie a chical le els o nes ed AER links whe e each link has a
dynamically econ igu able synap ic ou ing able which allows p og ammable connec i i y o he
neu ons wi hou es ic ion on he spa ial ange o connec i i y [
85
]. Mo adi e al. ha e p oposed
a mixed-mode hie a chical-mesh ou ing scheme ha exploi s a clus e ed connec i i y s uc u e o
educe memo y equi emen s and ge a balance among memo y o e head and econ igu abili y [86].
Figu e 5.
Illus a ion o wo neu al popula ions communica ed h ough a poin - o-poin AER bus. Each
neu on in he emi e popula ion can be i ually connec ed o e e y neu on in he ecei e popula ion.
The abo e men ioned spike ou ing schemes ha e allowed he implemen a ion o highly pa allel
massi ely in e connec ed spiking neu al ne wo ks and he mul ichip in eg a ion o SNN ha dwa e
de o ed o ealize di e en speci ic pa s o he cogni i e unc ion including in eg a ion o spike-based
senso s and neu al p ocesso s.
CMOS spike-based ision senso s ha e been de eloped since he e y beginning o he
neu omo phic enginee ing ield [
15
]. Since hen, a a ie y o AER isual senso s can be ound in he
li e a u e ha use di e en app oaches o encode he luminance such as simple luminance o equency
ans o ma ion senso s [
87
], Time- o-Fi s -Spike (TFS) coding senso s [
88
–
91
], o ea ed senso s [
92
,
93
],
senso s encoding he spa ial con as [
94
,
95
], spa ial and empo al il e ing senso s ha adap o
illumina ion and spa io- empo al con as [
96
] and empo al ansien de ec o s [
97
–
104
]. Among hem,
he empo al ansien de ec o s also know as Dynamic Vision Senso s (DVSs) ha e ecen ly become
Ma e ials 2019,12, 2745 16 o 28
damage due o ansien high-cu en s. When he ansis o de ice is omi ed, he cu en limi a ion
is done in he pe iphe al CMOS ci cui y, limi ing he size o he a ay o educe he isk o local
high pa asi ic ansien cu en s. In he 1T1R s uc u es, he ansis o also ac s as a selec ion de ice
o upda e indi idually each mem is o a oiding al e a ion o he nea by de ices. As a summa y,
al hough mem is o s a e a e y p omising echnology o implemen high-densi y analog memo ies
close o he compu ing sys em ha could po en ially implemen high-speed low powe lea ning
cogni i e sys em, he e a e s ill some echnological limi a ions ha a e cu en ly being in es iga ed
ha ha e no allowed o implemen such la ge scale sys ems.
5. Lea ning wi h Mem is o s (STDP)
Gi en ha hese SNNs a e mo e powe ul, in heo y, han 2nd gene a ion ne wo ks, i is na u al o
wonde why we do no see widesp ead use o hem. One main issue ha cu en ly lies in p ac ical use
o SNNs is ha o aining. Lea ning mechanisms a e c ucial o he abili y o neu omo phic sys ems
o adap o speci ic applica ions. In gene al, he goal o a lea ning algo i hm is o modi y he weigh s o
he synap ic connec ions be ween neu ons in o de o imp o e he esponse o he ne wo k o a ce ain
s imulus. Two main ca ego ies can be conside ed: supe ised o unsupe ised lea ning. In supe ised
lea ning, he da ase samples a e labeled wi h he iden i ica ion o he expec ed ‘co ec ’ ne wo k
ou pu . The measu ed de ia ion be ween he desi ed ou pu and he eal one is used o modi y he
synap ic weigh s. In unsupe ised lea ning, he e is no labeled in o ma ion, so he own cha ac e is ics
o he inpu da a a e analyzed by he ne wo k in o de o sel -o ganize.
As explained in Sec ion 2, in he ANN ield, he powe ul compu a ional capabili ies o mode n
GPUs and CPUs and he a ailabili y o la ge amoun o anno a ed da a ha e made possible o ain
complex deep lea ning a chi ec u es using he supe ised backp opaga ion lea ning algo i hm [
48
] o
sol e complex cogni i e p oblems in some cases wi h be e accu acy han humans. Howe e , he e
a e no known e ec i e supe ised aining me hods o SNNs ha o e highe pe o mance han 2nd
gene a ion ne wo ks. The popula backp opaga ion lea ning s a egies a e no di ec ly usable in SNN
ne wo ks. On he one hand, i spikes a e ep esen ed compu a ionally as he occu ence o an ou pu
e en a a pa icula ime (as ep esen ed in Figu e 3) hey a e no di e en iable; on he o he hand,
di e en ia ing he e o back ac oss he spa ial laye s (as i is done in he backp opaga ion algo i hm)
looses he p ecise empo al in o ma ion con ained in he spike imings. The e o e, in o de o p ope ly
use SNNs o eal-wo ld asks, we would need o de elop an e ec i e supe ised lea ning me hod
ha akes space and ime simul aneously in o accoun [
163
]. Se e al app oaches o SNN aining
ha e been adop ed:
T aining an ANN and con e sion o SNN
[
66
,
164
–
167
]. Some au ho s ha e p oposed ANN o
SNN di ec con e sion me hods which a e based on he aining o ANN using s a ic inpu images
and di ec ly mapping he ne wo k o an SNN con e ing he inpu s imulus o spikes using equency
a e encoding [
164
,
165
,
167
]. Bodo e al. implemen ed se e al op imiza ions achie ing o a a e coded
inpu simila pe o mance han equi alen ANN implemen a ions [
165
]. Howe e , such encoding
educes he powe e iciency o SNN. O he au ho s ha e p oposed o ain SNN wi h senso y da a
coming di ec ly om a spike-based senso (as a DVS eco ding). Fo ha pu pose, an equi alen
ANN using s a ic images gene a ed om his og ams o he inpu eco dings o spiking s imulus
is ained. A e wa ds, a me hod o con e he weigh s o he ANN o he co esponding SNN is
de ised [
66
]. The addi ional iming pa ame e s as leakage ime o e ac o y pe iod cha ac e is ics
o SNN a e op imized as hype -pa ame e s in he SNN esul ing on di e en op imized pa ame e
alues o di e en inpu dynamics. Bodo e al. ecen ly p oposed an ANN o SNN con e sion me hod
based on ime- o- i s -spike inpu con e sion code [
166
]. In all o hese me hods, aining is done on
s a ic images and hus hey do no ully exploi di ec ly all he spa io- empo al in o ma ion con ained
in he e en s.
Supe ised aining in he spiking domain
. Fo he abo e men ioned eason, some me hods
o di ec supe ised lea ning in he spiking domain ha e been p oposed [
168
–
179
]. Some o he

Ma e ials 2019,12, 2745 17 o 28
ea lie SNN aining me hods we e based on an adap a ion o he Del a Lea ning Rule [
44
] and we e
app op ia e o ain single laye a chi ec u es [
169
,
171
,
172
]. Mo e ecen SNN lea ning me hods ha e
been epo ed ha y o apply he backp opaga ion lea ning ules o SNN wi h se e al lea ning
laye s. They include coding he spike imes o ha e a di e en iable ela ionship wi h a subse o
p e ious spikes and hence compa ible wi h he g adien descen back-p opaga ion ule in he empo al
domain [
180
], o app oxima ing he spike shape esponse ac i i y o be di e en iable ac oss neu al
laye s [
174
,
175
,
177
]. Wu e al. in oduced an SNN Spa io-Tempo al BackP opaga ion algo i hm [
177
].
No only do hey app oxima e he spike shape as a con inuous di e en iable unc ion, bu also hey
use a back-p opaga ion- h ough- ime (BTT) [
163
] which backp opaga es he e o in he space as
well as he ime dimension epo ing he bes ecogni ion accu acy achie ed by p e iously epo ed
SNN on he MNIST and N-MNIST da ase s and equi alen o he s a e-o - he-a o ANNs. Simila ly,
he SLAYER me hod [
178
] conside s back-p opaga ion in space and ime and ains bo h weigh s and
delays o he synap ic connec ions.
Unsupe ised aining in he spiking domain
. The unsupe ised SNN aining me hods a e
mos ly based on he well known Spike-Timing-Dependen Plas ici y (STDP) lea ning ule [
31
,
32
]. STDP
is a Hebbian lea ning ule. The adi ional Hebbian synap ic plas ici y ule was o mula ed in 1940
sugges ing ha synapses inc ease hei e iciency i hey pe sis en ly ake pa in i ing he pos -synap ic
neu on [
39
]. Much la e in 1993, STDP lea ning algo i hms we e epo ed [
31
,
32
] as a e inemen o his
ule aking in o accoun he p ecise ela i e iming o indi idual p e- and pos -synap ic spikes, and no
hei a e age a es o e ime. In compa ison wi h adi ional Hebbian co ela ion-based plas ici y,
STDP p o ed o be be e sui ed o explaining b ain co ical phenomena [
181
,
182
], and demons a ed
o be success ul in lea ning hidden spiking pa e ns [
183
] o pe o ming compe i i e spike pa e n
lea ning [
184
]. In e es ingly, sho ly a e ha , in 1997, STDP lea ning was expe imen ally obse ed
in biological neu ons [
33
–
35
]. Figu e 9a,b illus a e he STDP lea ning ule as obse ed in biological
synapses. Figu e 9a plo s a p esynap ic neu on wi h a memb ane po en ial
Vp e
which is connec ed
h ough a synapse wi h synap ic s eng h
w
o a pos synap ic neu on wi h memb ane po en ial
Vpos
.
The p esynap ic neu on emi s a spike a ime
p e
which con ibu es o he gene a ion o a pos synap ic
spike a ime
pos
. The biological lea ning ule obse ed by Bi and Poo is illus a ed in Figu e 9b.
When he wo connec ed neu ons gene a e spikes close in ime, i
∆T= pos − p e
is posi i e, meaning
ha he p esynap ic pulse con ibu ed causally o gene a e he pos synap ic pulse, he e is a posi i e
a ia ion in he e icacy o he synap ic connec ion
ξ(∆T)>
0; on he con a y, i
∆T= pos − p e
is
nega i e, he a ia ion in he e icacy o he synap ic connec ion
ξ(∆T)<
0 is nega i e. Being STDP
a local lea ning ule, and mem is o s wo- e minal de ices exhibi ing plas ici y con olled by he
local applied ol age/cu en o hei e minals con e s mem is o s as ideal candida es o implemen
high-densi y on-line STDP-based neu omo phic lea ning sys ems [
27
]. Lina es e al. [
28
] showed ha
by combining he mem is ance model o mula ed in Equa ion (2) wi h he elec ical wa e signals
o neu al impulses (spikes) as shaped in Figu e 9c applied o he p e- and pos -synap ic e minals
o he mem is i e synap ic-like de ice, he STDP beha io shown in Figu e 9d eme ges na u ally.
Conside ing he ma hema ical equa ion desc ibing he spike shape shown in Figu e 9c e sus ime
spk( ) =















A+
mp e /τ+−e +
ail/τ+
1−e +
ail/τ+i − +
ail < <0
A−
mp e− /τ−−e− −
ail/τ−
1−e− −
ail/τ−i 0< < −
ail
0o he wise
(5)
and a mem is i e synapse-like de ice whe e a p esynap ic spike
spk( )
wi h a enua ion
αp e
a i es a
ime
o i s nega i e e minal and a pos synap ic spike
spk( +∆T)
wi h a enua ion
αpos
a i es a
ime +∆T o i s posi i e e minal, a ol age di e ence
Ma e ials 2019,12, 2745 18 o 28
MR( +∆T) = αposspk( +∆T)−αp espk( )(6)
is gene a ed among he de ice e minals. The o al change in he mem is ance pa ame e
w
can hus
be compu ed as,
∆w(∆T) = Z MR( MR( +∆T))d =ξ(∆T)(7)
In e es ingly, o he mem is o model conside ed in Equa ion (2) and he spike shape conside ed
in Equa ion (5), he mem is ance lea ning ule shown in Figu e 9d
ξ(∆T)
is ob ained which esembles
he STDP ule obse ed by Ge s ne in biological neu ons. By playing wi h he spike shapes, many
o he STDP upda e ules can be uned as demons a ed by Zama eño e al. [29,30].
Figu e 9.
Illus a ion o STDP lea ning ule. (
a
) P e-synap ic neu on gene a ing a spike
Vp e
a ime
p e
ha a i es o a pos -synap ic neu on ha gene a es a spike
Vpos
a ime
pos
, being
∆T= pos − p e
,
and (
b
) illus a es he a ia ion o he synap ic e icacy
ξ(∆T)
Vs
∆T
, STDP lea ning ule, as he
obse ed by Bi and Poo in biological synapses. (
c
) Illus a es he spike shape ha applied o he
mem is i e de ices desc ibes in Sec ion 4 ep oduces he STDP lea ning ule shown in (d).
In he las decade, many di e en wo ks ha e demons a ed he eme gence o STDP lea ning in
mem is i e de ices o di e en kinds o ma e ials [
137
,
180
,
185
–
189
]. Howe e , as al eady s a ed in
Sec ion 4, a a sys em le el, he cu en limi a ions o he mem is o echnology in e ms o con ol
o he esolu ion o he weigh upda ing, ha e no made possible he implemen a ion o wo king
STDP mem is i e lea ning sys ems wi h analog synap ic elemen s. P ecision in he weigh upda e
is di icul o con ol and mos o he mem is i e de ices ope a e changing be ween bina y s a es.
Fo ha eason, s ochas ic STDP lea ning ules ha ope a e wi h bina y weigh s du ing in e ence
and upda ing ope a ion ha e been p oposed. Seo e al. [
190
] applied his idea o simple classi ica ion
p oblems, bu hey ound ha hey could no lea n o sepa a e mo e han 5 pa e ns. Recen ly,
Youse zadeh e al. [
191
] we e able o classi y mo e elabo a ed da abases (as MNIST) by in oducing
some o he echniques ha imp o ed he pe o mance.
Combining unsupe ised ea u e ex ac ion me hods wi h supe ised ca ego iza ion aining
.
While supe ised lea ning me hods like backp opaga ion a e no ene gy e icien , a e no app op ia e
Ma e ials 2019,12, 2745 19 o 28
o on-line chip lea ning, and do no look like biologically plausible, unsupe ised lea ning ules
a e app op ia e o ex ac epe i i e s uc u es in he aining da a bu no app op ia e o ake
decisions
[192,193]
. Fo example, Moza a i e al. p opose o combine unsupe ised STDP laye
wi h supe ised Rein o cemen Lea ning STDP laye s [
193
]. The esul ing ne wo k is mo e obus o
o e i ing compa ed o backp oga ion aining as i ex ac s common ea u es and pe o ms well wi h
educed numbe o aining samples.
6. Fu u e Pe spec i e
I is well known ha he human b ain con ains abou 10
11
neu ons in e connec ed h ough
1015 synapses
, and wi h a powe consump ion o a ound 20 W i is capable o pe o ming complex
sensing and cogni i e p ocessing, sophis ica ed mo o con ol, lea ning and abs ac ion, and i
can dynamically adap o changing en i onmen s and unp edic ed condi ions. Fo his eason,
neu omo phic enginee s ha e been using he b ain as a p ocessing pa adigm o se e al decades
in o de o ab ica e a i icial p ocessing sys ems wi h simila capabili ies. A e he ini ial
a emp s o building he i s spike-based p ocessing sys ems demons a ed hei easibili y and
showed hei p omising po en ial [
78
], i became e iden he need o scaling up hese sys ems
in e ms o numbe o neu ons and synapses [
62
]. Se e al wo ks de eloped by bo h academic
ins i u ions [
86
,
121
–
125
,
127
–
129
] and indus ial playe s like IBM [
120
] o In el [
126
] ab ica ed
neu omo phic chips wi h up o 1 M neu ons and 256 M synapses, which could be ensembled in
mul i-chip boa ds and mul i-boa d pla o ms, opening he way o implemen la ge sys ems in he nea
u u e wi h numbe s o neu ons and synapses simila o he b ain. Howe e , hese sys ems, based on
di e en CMOS echnologies, will be limi ed by he hei la ge oom-scale size. Besides, he complexi y
o cu en implemen a ions o lea ning algo i hms in CMOS limi s hei scalabili y.
The eme gence o mem is o s and hei synap ic-like beha io opened he possibili y o o e come
he limi a ions o CMOS echnologies. Mem is o s can be a ew nanome e s size and can be packed
densely in a wo-dimensional laye wi h nanome e - ange pi ch, po en ially o e ing highe neu on
and synap ic densi y. Wi h a ab ica ion p ocess much cheape han CMOS, mem is o laye s can be
s acked in 3D. Assuming a easonable 30-nm pi ch, he supe posi ion o 10 mem is i e laye s could
heo e ically p o ide a memo y densi y o 10
11
non- ola ile analog cells pe cm
2
. This app oach could
in p inciple each he neu on and synap ic densi y o he human b ain in a single boa d, including
lea ning capabili ies [
194
]. Fu he mo e, he close 3D dense packaging be ween he CMOS neu al
compu a ion uni s and he mem is i e adap i e memo y synap ic elemen s can signi ican ly educe
he cu en consump ion o he esul ing sys ems.
Cu en a ailable mem is o s a e desc ibed as 1T1R de ices, meaning ha hey a e o med by he
se ies connec ion o a MOS ansis o and a mem is i e elemen . This ansis o is used o limi he
cu en lowing h ough he mem is o du ing each ope a ion (Fo ming, W i ing, E asing, Reading)
o a oid damaging he de ice. Howe e , his s uc u e is limi ing he densi y o mem is o s, as hey
a e also consuming a ea in he CMOS subs a e. An al e na i e o o e come his limi a ion is gi en
by 1S1R de ices (1-selec o -1- esis o ), whe e a ola ile mem is o (1S) is connec ed in se ies wi h a
non- ola ile mem is o (1R), eluding any CMOS a ea consump ion [195].
Hyb id sys ems wi h mem is o laye s ab ica ed on op o a CMOS subs a e can p o ide highly
pa allel massi e s o age igh ly coupled o CMOS compu ing ci cui y. The e o e, compu ing and
lea ning p ocesses in he b ain can be imi a ed by combining mem is o s wi h spiking p ocesso s
and in eg a e-and- i e neu ons in silicon. Using mesh echniques [
82
], g ids o ens o chips can
be assembled modula ly on a P in ed Ci cui Boa d (PCB), allowing o scaling up he numbe s o
neu ons and synapses in a neu al sys em [
65
]. The combina ion o all hese echniques oge he
wi h he esolu ion o he mul iple echnical challenges cu en ly associa ed o dense mem is i e
laye s ( eliabili y, epea abili y, ep og ammabili y) could p o ide an impo an s ep owa ds he
ha dwa e implemen a ion o b ain-scale low-powe neu omo phic p ocessing sys ems wi h online
STDP lea ning.
Ma e ials 2019,12, 2745 20 o 28
Au ho Con ibu ions:
W i ing—o iginal d a p epa a ion, L.A.C.-M. and T.S.-G.; w i ing— e iew and edi ing,
L.A.C.-M., T.S.-G. and B.L.-B.; supe ision, B.L.-B. and T.S.-G.; unding acquisi ion, T.S.-G. and L.A.C.-M.
Funding:
This wo k was unded by EU H2020 g an s 687299 ”NEURAM3” and 824164 ”HERMES”, and by
Spanish g an om he Minis y o Economy and Compe i i i y TEC2015-63884-C2-1-P (COGNET) (wi h suppo
om he Eu opean Regional De elopmen Fund). Luis A. Camuñas-Mesa was unded by he VI PPIT h ough he
Uni e sidad de Se illa.
Con lic s o In e es : The au ho s decla e no con lic o in e es .
Re e ences
1.
Von Neumann, J. Fi s D a o a Repo on he EDVAC. IEEE Ann. His . Compu .
1945
,15, 27–75. [C ossRe ]
2.
Moo e, G.E. C amming mo e componen s on o in eg a ed ci cui s. Elec onics
1965
,38, 114–117. [C ossRe ]
3. Wald op, M.M. The chips a e down o Moo e’s law. Na u e 2016,530, 144–147. [C ossRe ] [PubMed]
4.
Kau , J. Li e Beyond Moo e: Mo e Moo e o Mo e han Moo e—A Re iew. In . J. Compu . Sci. Mob. Compu .
2016,5, 233–237.
5.
Hodgkin, A.L.; Huxley, A.F. Cu en s ca ied by sodium and po assium ions h ough he memb ane o he
gian squid axon o loligo. J. Physiol. 1952,116, 449–472. [C ossRe ] [PubMed]
6.
Fi zHugh, R. Impulses and physiological s a es in models o ne e memb ane. Biophys. J.
1961
,1, 445–466.
[C ossRe ]
7.
Nagumo, J.S.; A imo o, S.; Yoshizawa, S. An ac i e pulse ansmission line simula ing ne e axon. P oc. IRE
1962,50, 2061–2070. [C ossRe ]
8.
Mo is, C.; Leca , H. Vol age oscilla ions in he ba nacle gian muscle ibe . Biophys. J.
1981
,35, 193–213.
[C ossRe ]
9.
B e e, R.; Ge s ne , W. Adap i e exponen ial in eg a e-and- i e model as an e ec i e desc ip ion o neu onal
ac i i y. J. Neu ophysiol. 2005,94, 3637–3642. [C ossRe ]
10.
Izhike ich, E.M. Simple Model o Spiking Neu ons. IEEE T ans. Neu al Ne w.
2003
,14, 1569–1572. [C ossRe ]
11.
Runge, R.G.; Uemu a, M.; Viglione, S.S. Elec onic syn hesis o he a ian e ina. IEEE T ans. Biomed. Eng.
1968,15, 138–151. [C ossRe ] [PubMed]
12. Fu be , S. La ge-scale neu omo phic compu ing sys ems. J. Neu al Eng. 2016,13, 051001. [C ossRe ] [PubMed]
13. Mead, C. Analog VLSI and Neu al Sys ems; Addison-Wesley: Bos on, MA, USA, 1989.
14. Mead, C. Neu omo phic Elec onic Sys ems. P oc. IEEE 1990,78, 1629–1636. [C ossRe ]
15. Mahowald, M.A.; Mead, C. The silicon e ina. Sci. Am. 1991,264, 76–82. [C ossRe ] [PubMed]
16. Smi h, L.S. Neu omo phic Sys ems: Pas , P esen and Fu u e. B . Inspi . Cogni . Sys . 2008, 167–182.
17.
Chua, L.O. Mem is o —The Missing Ci cui Elemen . IEEE T ans. Ci cui Theo y
1971
,18, 507–519. [C ossRe ]
18.
S uko , D.B.; Snide , G.S.; S ewa , D.R.; Williams, R.S. The missing mem is o ound. Na u e
2008
,453,
80–83. [C ossRe ]
19.
Hashem, N.; Das, S. Swi ching- ime analysis o bina y-oxide mem is o s ia a non-linea model.
Appl. Phys. Le . 2012,100, 262106. [C ossRe ]
20.
K a insky, S.; Belouso , D.; Liman, S.; Sa a , G.; Wald, N.; F iedman, E.G.; Kolodny, A.; Weise , U.C.
MAGIC—Mem is o -Aided Logic. IEEE T ans. Ci cui s Sys . II Exp ess B . 2014,11, 895–899. [C ossRe ]
21.
K a insky, S.; F iedman, E.G.; Kolodny, A.; Weise , U.C. Mem is o -based ma e ial implica ion (IMPLY)
logic: Design p inciples and me hodologies. IEEE T ans. Ve y La ge Scale In eg . (VLSI)
2013
,10, 2054–2066.
[C ossRe ]
22.
Su, F.; Chen, W.H.; Xia, L.; Lo, C.P.; Tang, T.; Wang, Z.; Hsu, K.H.; Cheng, M.; Li, J.Y.; Xie, Y.; e al.
A 462 GOPs/J RRAM-based non ola ile in elligen p ocesso o ene gy ha es ing IoE sys em ea u ing
non ola ile logics and p ocessing-in-memo y. In P oceedings o he 2017 Symposium on VLSI Technology,
Kyo o, Japan, 5–8 June 2017.
23.
Liu, Y.; Wang, Z.; Lee, A.; Su, F.; Lo, C.; Yuan, Z.; Lin, C.; Wei, Q.; Wang, Y.; King, Y.; e al. A 65 nm
ReRAM-Enabled Non ola ile P ocesso wi h 6
×
Reduc ion in Res o e Time and 4
×
Highe Clock F equency
Using Adap i e Da a Re en ion and Sel -W i e-Te mina ion Non ola ile Logic. In . Con . Solid-S a e Ci cui s
2016,59, 84–86.
Ma e ials 2019,12, 2745 21 o 28
24.
Onuki, T.; Uesugi, W.; Tamu a, H.; Isobe, A.; Ando, Y.; Okamo o, S.; Ka o, K.; Yew, T.; Lin, C.; Wu, J.; e al.
Embedded memo y and ARM Co ex-M0 co e using 60-nm C-axis aligned c ys alline indium-gallium-zinc
oxide FET in eg a ed wi h 65-nm Si CMOS. IEEE Symp. VLSI Ci cui s 2017,52, 925–932.
25.
Shulake , M.M.; Hills, G.; Pa k, R.; Howe, R.; Sa aswa , K.; Wong, H.; Mi a, S. Th ee-dimensional in eg a ion
o nano echnologies o compu ing and da a s o age on a single chip. Na u e 2017,547, 74–78. [C ossRe ]
26.
Ca a a, S.; Sacche o, D.; Doucey, M.A.; Baj-Rossi, C.; De Micheli, G.; Leblebici, Y. Mem is i e-biosenso s:
A new de ec ion me hod by using nano ab ica ed mem is o s. Sens. Ac ua o s B Chem.
2012
,171–172,
449–457. [C ossRe ]
27.
Snide , G.S. Spike- ime-dependen Plas ici y in Mem is i e Nano echnologies. In P oceedings o he IEEE/ACM
In e na ional Symposium on Nanoscale A chi ec u es, Washing on, DC, USA, 12–13 June 2008.
28.
Lina es-Ba anco, B.; Se ano-Go a edona, T. Mem is ance can explain spike- ime-dependen -plas ici y in
neu al synapses. Na . P eced. 2009. [C ossRe ]
29.
Zama eno-Ramos, C.; Camuñas-Mesa, L.A.; Pé ez-Ca asco, J.A.; Masquelie , T.; Se ano-Go a edona, T.;
Lina es-Ba anco, B. On spike- iming-dependen -plas ici y, mem is i e de ices, and building a sel -lea ning
isual co ex. F on . Neu osci. 2011,5, 26. [C ossRe ]
30.
Se ano-Go a edona, T.; Masquelie , T.; P od omakis, T.; Indi e i, G.; Lina es-Ba anco, B. STDP and STDP
a ia ions wi h mem is o s o spiking neu omo phic lea ning sys ems. F on . Neu osci.
2013
,7, 2. [C ossRe ]
31.
Ge s ne , W.; Ri z, R.; Hemmen, J.L. Why spikes? Hebbian lea ning and e ie al o ime- esol ed exci a ion
pa e ns. Biol. Cybe n. 1993,69, 503–515. [C ossRe ]
32.
Ge s ne , W.; Kemp e , R.; Leo an Hemmen, J.; Wagne , H. A neu onal lea ning ule o sub-millisecond
empo al coding. Le . Na . 1996,383, 76–78. [C ossRe ]
33.
Ma k am, H.; Lübke, J.; F o sche , M.; Sakmann, B. Regula ion o synap ic e icacy by coincidence o
pos synap ic APS and EPSPS. Science 1997,275, 213–215. [C ossRe ]
34.
Bi, G.; Poo, M. Synap ic modi ica ions in cul u ed hippocampal neu ons: Dependence on spike iming,
synap ic s eng h, and pos synap ic cell ype. J. Neu osci. 1998,18, 10464–10472. [C ossRe ]
35.
Bi, G.; Poo, M. Synap ic modi ica ion by co ela ed ac i i y: Hebb’s pos ula e e isi ed. Ann. Re . Neu osci.
2001,24, 139–166. [C ossRe ]
36.
Jacob, V.; B asie , D.J.; E cho a, I.; Feldman, D.; Shulz, D.E. Spike iming-dependen synap ic dep ession in
he in i o ba el co ex o he a . J. Neu osci. 2007,27, 1271–1284. [C ossRe ]
37. McCulloch, W.S.; Pi s, W. A logical calculus o he ideas immanen in ne ous ac i i y. Bull. Ma h. Biophys.
1943,5, 115–133. [C ossRe ]
38.
Rosenbla , F. The Pe cep on: A P obabilis ic Model o In o ma ion S o age and O ganiza ion in he B ain.
Psychol. Re . 1958,65, 386–408. [C ossRe ]
39. Hebb, D. The O ganiza ion o Beha io ; Wiley: New Yo k, NY, USA, 1949.
40. Minsky, M.L.; Pape , S.A. Pe cep ons; MIT P ess: Camb idge, MA, USA, 1969.
41.
Maass, W. Ne wo ks o spiking neu ons: The hi d gene a ion o neu al ne wo k models. Neu al Ne w.
1997
,
10, 1659–1671. [C ossRe ]
42. Ghosh-Das ida S.; Adeli H. Thi d Gene a ion Neu al Ne wo ks: Spiking Neu al Ne wo ks. In Ad ances in
Compu a ional In elligence. Ad ances in In elligen and So Compu ing; Yu, W., Sanchez, E.N., Eds.; Sp inge :
Be lin, Ge many, 2009.
43.
K izhe sky, A.; Su ske e , I.; Hin on, G.E. Imagene classi ica ion wi h deep con olu ional neu al ne wo ks.
In P oceedings o he ImageNe Classi ica ion wi h Deep Con olu ional Neu al Ne wo ks NIPS, Lake Tahoe,
CA, USA, 3–6 Decembe 2012.
44.
Wid ow, B. Adap i e “Adaline” Neu on Using Chemical “Memis o s”; Numbe Technical Repo 1553-2; S an o d
Elec on. Labs.: S an o d, CA, USA, 1960.
45.
Wid ow, B.; Leh , M.A. 30 yea s o Adap i e Neu al Ne wo ks: Pecep on, Madaline, and Backp opaga ion.
P oc. IEEE 1990,78, 1415–1442. [C ossRe ]
46.
We bos, P. Beyond Reg ession: New Tools o P edic ion and Analysis in he Beha io al Sciences. Ph.D.
Thesis, Ha a d Uni e si y, Camb idge, MA, USA, 1974.
47.
Pa ke , D. Lea ning-Logic; In en ion Repo 581-64, File 1; O ice o Technology Licensing, S an o d Uni .:
S an o d, CA, USA, 1982.
48.
Rumelha , D.E.; Hin on, G.E.; Williams, R.J. Lea ning ep esen a ions by back-p opaga ing e o s. Na u e
1986, 1476–4687. [C ossRe ]

Ma e ials 2019,12, 2745 22 o 28
49.
Ca pen e , G.A.; G ossbe g, S. A massi ely pa allel a chi ec u e o a sel -o ganizing neu al pa e n
ecogni ion machine. Compu . Vis. G . Image P ocess. 1983,37, 54–115. [C ossRe ]
50.
Kohonen, T. Sel -o ganized o ma ion o opologically co ec ea u e maps. Biolog. Cybe n.
1982
,43, 59–69.
[C ossRe ]
51.
Hop ield, J.J. Neu al ne wo ks and physical sys ems wi h eme gen collec i e compu a ional abili ies.
P oc. Na l. Acad. Sci. USA 1982,79, 2554–2558. [C ossRe ]
52. Bishop, C.M. Neu al Ne wo ks o Pa e n Recogni ion; Cla endon P ess: Ox o d, UK, 1995.
53. Bengio, Y. Lea ning Deep A chi ec u es o AI. Found. T ends Mach. Lea n. 2009,2, 1–127. [C ossRe ]
54.
Schmidhube , J. Deep Lea ning in Neu al Ne wo ks: An O e iew. Neu al Ne w.
2015
,61, 85–117. [C ossRe ]
55.
LeCun, Y.; Jackel, L.D.; Bose , B.; Denke , J.S.; G a , H.P.; Guyon, I.; Hende son, D.; Howa d, R.E.; Hubba d, W.
Handw i en digi ecogni ion: Applica ions o neu al ne wo k chips and au oma ic lea ning. IEEE Commun.
Mag. 1989,27, 41–46. [C ossRe ]
56.
Deng, L.; Li, J.; Huang, J.; Yao, K.; Yu, D.; Seide, F.; Sel ze , M.; Sweig, G.; He, X.; Williams, J.; e al.
Recen ad ances in deep lea ning o speech esea ch a Mic oso . In P oceedings o he 2013 IEEE In e na ional
Con e ence on Acous ics, Speech and Signal P ocessing, Vancou e , BC, Canada, 26–31 May 2013 .
57.
Tho pe, S.; Fize, D.; Ma lo , C. Speed o p ocessing in he human isual sys em. Na u e
1996
,381, 520–522.
[C ossRe ]
58.
Rolls, E.T.; To ee, M.J. P ocessing speed in he ce eb al co ex and he neu ophysiology o isual masking.
P oc. R. Soc. London. Se . B Biol. Sci. 1994,257, 9–15.
59.
Tho pe, S.; Delo me, A.; Van Rullen, R. Spike-based s a egies o apid p ocessing. Neu al Ne w.
2001
,14,
715–725. [C ossRe ]
60.
Huys, Q.; Zemel, R.; Na a ajan, R.; Dayan, P. Fas popula ion coding. Neu al Compu .
2007
,19, 404–441.
[C ossRe ]
61. Rullen, R.V.; Tho pe, S.J. Ra e coding e sus empo al o de coding: Wha he e inal ganglion cells ell he
isual co ex. Neu al Compu . 2001,13, 1255–1283. [C ossRe ]
62.
Indi e i, G.; Lina es-Ba anco, B.; Hamil on, T.J.; Schaik, A.; E ienne-Cummings, R.; Delb ück, T.; Liu, S.;
Dudek, P.; Hä lige , P.; Renaud, S.; e al. Neu omo phic silicon neu on ci cui s. F on . Neu osci.
2011
,5, 73.
[C ossRe ]
63.
Fukushima, K. Neocogni on: A hie a chical neu al ne wo k capable o isual pa e n ecogni ion.
Neu al Ne w. 1988,1, 119–130. [C ossRe ]
64.
Hubel, D.H.; Wiesel, T.N. Recep i e ields and unc ional a chi ec u e o monkey s ia e co ex. J. Physiol.
1968,195, 215–243. [C ossRe ]
65.
Fa abe , C.; Paz, R.; Pé ez-Ca asco, J.; Zama eño-Ramos, C.; Lina es-Ba anco, A.; Lecun, Y.; Culu ciello, E.;
Se ano-Go a edona, T.; Lina es-Ba anco, B. Compa ison be ween ame-cons ained ix-pixel- alue and
ame- ee spiking-dynamic-pixel Con Ne s o isual p ocessing. F on . Neu osci. 2012,6, 32. [C ossRe ]
66.
Pe ez-Ca asco, J.A.; Zhao, B.; Se ano, C.; Acha, B.; Se ano-Go a edona, T.; Chen, S.; Lina es-Ba anco,
B. Mapping om ame-d i en o ame- ee e en -d i en ision sys ems by low- a e a e coding and
coincidence p ocessing–applica ion o eed o wa d Con Ne s. IEEE T ans. Pa e n Anal. Mach. In ell.
2013
,
35, 2706–2719. [C ossRe ]
67.
Camunas-Mesa, L.; Acos a-Jiménez, A.; Zama eño-Ramos, C.; Se ano-Go a edona, T.; Lina es-Ba anco, B.
A 32x32 Pixel Con olu ion P ocesso Chip o Add ess E en Vision Senso s Wi h 155 ns E en La ency and
20 Meps Th oughpu . IEEE T ans. Ci cui s Sys . I Regul. Pap. 2011,58, 777–790. [C ossRe ]
68.
Sengup a, A.; Ye, Y.; Wang, R.; Liu, C.; Roy, K. Going Deepe in Spiking Neu al Ne wo ks: VGG and Residual
A chi ec u es. F on . Neu osci. 2019. [C ossRe ]
69.
Bou ie , M.; Valen ian, A.; Mesquida, T.; Rummens, F.; Reybox, M.; Vianello, E.; Biegne, E. Spiking Neu al
Ne wo ks Ha dwa e Implemen a ions and Challenges: A Su ey. ACM J. Eme g. Technlol. Compu . Sys .
2019,15, 1–35. [C ossRe ]
70.
Schmid, A. Neu omo phic mic oelec onics om de ices o ha dwa e sys ems and applica ions. Nonlinea
Theo y I s Appl. IEICE 2016,7, 468–498. [C ossRe ]
71.
Si ilo i, M. Wi ing Conside a ions in Analog VLSI Sys ems wi h Applica ion o Field-P og ammable
Ne wo ks. Ph.D. Thesis, Compu a ion and Neu al Sys ems, Cali o nia Ins . Technol., Pasadena, CA, USA, 1991.
72.
Se ano-Go a edona, T.; And eou, A.G.; Lina es-Ba anco, B. AER image il e ing a chi ec u e o
ision-p ocessing sys ems. IEEE T ans. Ci cui s Sys . I 1999,46, 1064–1071. [C ossRe ]
Ma e ials 2019,12, 2745 23 o 28
73.
Boahen, K. Poin - o-Poin connec i i y be ween neu omo phic chips using add ess e en s. IEEE T ans.
Ci cui s Sys . II 2000,47, 416–434. [C ossRe ]
74.
Boahen, K. A bu s -mode wo d-se ial add ess-e en link-I,II,III. IEEE T ans. Ci cui s Sys . I
2004
,51,
1269–1280. [C ossRe ]
75.
Lin, J.; Me olla, P.; A hu , J.; Boahen, K. P og ammable connec ions in neu omo phic g ids. In P oceedings
o he 2006 49 h IEEE In e na ional Midwes Symposium on Ci cui s and Sys ems, San Juan, Pue o Rico,
6–9 Augus 2006; pp. 80–84.
76.
Me olla, P.; A hu , J.; Shi, B.; Boahen, K. Expandable ne wo ks o neu omo phic chips. IEEE T ans. Ci cui s
Sys . I 2007,54, 301–311. [C ossRe ]
77.
Bam o d, S.A.; Mu ay, A.F.; Willshaw, D.J. La ge de eloping ecep i e ields using a dis ibu ed and locally
ep og ammable add ess-e en ecei e . IEEE T ans. Neu al Ne w. 2010,21, 286–304. [C ossRe ]
78.
Se ano-Go a edona, R.; Os e , M.; Lich s eine , P.; Lina es-Ba anco, A.; Paz-Vicen e, R.;
Gomez-Rod iguez, F.; Camuñas-Mesa, L.; Be ne , R.; Ri as-Pe ez, M.; Delb ück, T.; e al. CAVIAR: A 45k
neu on, 5M synapse, 12G connec s/s AER ha dwa e senso y-p ocessing-lea ning-ac ua ing sys em o
high-speed isual objec ecogni ion and acking. IEEE T ans. Neu al Ne w.
2009
,20, 1417–1438. [C ossRe ]
79.
Joshi, S.; Deiss, S.; A nold, M.; Pa k, J.; Yu, T.; Cauwenbe ghs, G. Scalable e en ou ing in hie a chical neu al
a ay a chi ec u e wi h global synap ic connec i i y. In P oceedings o he In e na ional Wo kshop Cellula
Nanoscale Ne wo ks and Thei Applica ions, Be keley, CA, USA, 3–5 Feb ua y 2010.
80.
Khan, M.; Les e , D.; Plana, L.; Ras , A.; Jin, X.; Paink as, E.; Fu be , S. SpiNNake : Mapping neu al ne wo ks
on o a massi ely-pa allel chip mul ip ocesso . In P oceedings o he 2008 IEEE In e na ional Join Con e ence
on Neu al Ne wo ks, Hong Kong, China, 1–8 June 2008; pp. 2849–2856.
81. Benini, L.; Micheli, G.D. Ne wo ks on chips: A new SoC pa adigm. IEEE Compu . 2002, 70–78. [C ossRe ]
82.
Zama eno-Ramos, C.; Lina es-Ba anco, A.; Se ano-Go a edona, T.; Lina es-Ba anco, B.
Mul icas ing mesh AER: A scalable assembly app oach o econ igu able neu omo phic s uc u ed AER
sys ems. Applica ion o Con Ne s. IEEE T ans. Biomed. Ci cui s Sys . 2013,7, 82–102. [C ossRe ]
83.
Fie es, J.; Schemmel, J.; Meie , K. Realizing biological spiking ne wo k models in a con igu able wa e -scale
ha dwa e sys em. In P oceedings o he 2008 IEEE In e na ional Join Con e ence on Neu al Ne wo ks,
Hong Kong, China, 1–8 June 2008; pp. 969–976.
84.
Scholze, S.; Schie e , S.; Pa zsch, J.; Ha mann, S.; May , C.; Höppne , S.; Eisen eich, H.; Henke , S.;
Vogginge , B.; Schü ny, R. VLSI implemen a ion o a 2.8 ge en /s packe based AER in e ace wi h ou ing
and e en so ing unc ionali y. F on . Neu osci. 2011,5, 117. [C ossRe ]
85.
Pa k, J.; Yu, T.; Joshi, S.; Maie , C.; Cauwenbe ghs, G. Hie a chical Add ess E en Rou ing o Recon igu able
La ge-Scale Neu omo phic Sys ems. IEEE T ans. Neu al Ne w. Lea n. Sys . 2017,28, 2408–2422. [C ossRe ]
86.
Mo adi, S.; Qiao, N.; S e anini, F.; Indi e i, G. A Scalable Mul ico e A chi ec u e wi h He e ogeneous
Memo y S uc u es o Dynamic Neu omo phic Asynch onous P ocesso s (DYNAPs). IEEE T ans. Biomed.
Ci cui s Sys . 2018,12, 106–122. [C ossRe ]
87.
Culu ciello, E.; E ienne-Cummings, R.; Boahen, K.A. A biomo phic digi al image senso . IEEE J.
Solid-S a e Ci cui s 2003,38, 281–294. [C ossRe ]
88.
Ruedi, P.F.; Heim, P.; Kaess, F.; G ene , E.; Hei ge , F.; Bu gi, P.; Gyge , S.; Nussbaum, P. A 128
×
128
pixel 120-dB dynamic- ange ision-senso chip o image con as and o ien a ion ex ac ion. IEEE J.
Solid-S a e Ci cui s 2003,1, 2325–2333. [C ossRe ]
89.
Ba ba o, M.; Bu gi, P.; Mo a a, R.; Nussbaum, P.; Hei ge , F. A 100
×
100 pixel silicon e ina o g adien
ex ac ion wi h s ee ing il e capabili ies and empo al ou pu coding. IEEE J. Solid-S a e Ci cui s
2002
,37,
160–172. [C ossRe ]
90.
Chen, S.; Be mak, A. A bi a ed ime- o- i s spike CMOS image senso wi h on-chip his og am equaliza ion.
IEEE T ans. Ve y La ge Scale In eg . Sys . 2007,15, 346–357.
91.
Qi, X.G.; Ha is, J. A ime- o- i s -spike CMOS image . In P oceedings o he 2004 IEEE In e na ional
Symposium on Ci cui s and Sys ems (IEEE Ca . No.04CH37512), Vancou e , BC, Canada, 23–26 May 2004;
pp. 824–827.
92.
Azadmeh , M.; Ab ahamsen, J.; Hä lige , P. A o ea ed AER image chip. In P oceedings o he IEEE
In e na ional Symposium on Ci cui s and Sys ems, Kobe, Japan, 23–26 May 2005; pp. 2751–2754.
Ma e ials 2019,12, 2745 24 o 28
93.
Vogels ein, R.J.; Mallik, U.; Culu ciello, E.; E ienne-Cummings, R.; Cauwenbe ghs, G. Spa ial acui y
modula ion o an add ess-e en image . In P oceedings o he IEEE ICECS, Tel A i , Is ael, Is ael,
15 Decembe 2004; pp. 207–210.
94.
Cos as-San os, J.; Se ano-Go a edona, T.; Se ano-Go a edona, R.; Lina es-Ba anco, B. A Spa ial
Con as Re ina wi h On-chip Calib a ion o Neu omo phic Spike-Based AER Vision Sys ems. IEEE T ans.
Ci cui s Sys . I 2007,54, 1444–1458. [C ossRe ]
95.
Leñe o-Ba dallo, J.A.; Se ano-Go a edona, T.; Lina es-Ba anco, B. A 5-Decade Dynamic Range
Ambien -Ligh -Independen Calib a ed Signed-Spa ial-Con as AER Re ina wi h 0.1ms La ency and
Op ional Time- o-Fi s -Spike Mode. IEEE T ans. Ci cui s Sys I 2010,57, 2632–2643. [C ossRe ]
96.
Zaghloul, K.A.; Boahen, K. Op ic ne e signals in a neu omo phic chip: Pa s 1 and 2. IEEE T ans. Biomed. Eng.
2004,51, 657–675. [C ossRe ]
97.
Leñe o-Ba dallo, J.A.; Se ano-Go a edona, T.; Lina es-Ba anco, B. A 3.6us Asynch onous F ame-F ee
E en -D i en Dynamic-Vision-Senso . IEEE J. Solid-S a e Ci cui s 2011,46, 1443–1455. [C ossRe ]
98.
K ame , J. An in eg a ed op ical ansien senso . IEEE T ans. Ci cui s Sys . II Analog Digi . Signal P ocess
2002,49, 612–628. [C ossRe ]
99.
Lich s eine , P.; Posch, C.; Delb ück, T. A 128
×
128 120 dB 15
µ
s la ency asynch onous empo al con as
ision senso . IEEE J. Solid-S a e Ci cui s 2008,43, 566–576. [C ossRe ]
100.
Se ano-Go a edona, T.; Lina es-Ba anco, B. A 128x128 1.5% Con as Sensi i i y 0.9% FPN 3us La ency
4mW Asynch onous F ame-F ee Dynamic Vision Senso Using T ansimpedance Ampli ie s. IEEE J.
Solid-S a e Ci cui s 2013, 827–838. [C ossRe ]
101.
B andli, C.; Be ne , R.; Yang, M.; Liu, S.; Delb ück, T. A 240
×
180 130 dB 3
µ
s La ency Global Shu e
Spa io empo al Vision Senso . IEEE J. Solid-S a e Ci cui s 2014, 2333–2341. [C ossRe ]
102.
Moeys, D.P.; Co adi, F.; Li, C.; Bam o d, S.; Longino i, L.; Voig , F.; Be y, S.; Ta e ni, G.; Helmchen, F.;
Delb ück, T. A Sensi i e Dynamic and Ac i e Pixel Vision Senso o Colo o Neu al Imaging Applica ions.
IEEE T ans. Biomed. Ci cui s Sys . 2018,12, 123–136. [C ossRe ]
103.
Posch, C.; Ma olin, D.; Wohlgenann , R. A QVGA 143 dB dynamic ange ame- ee PWM image senso wi h
lossless pixel-le el ideo comp ession and ime-domain CDS. IEEE J. Solid-S a e Ci cui s
2011
,46, 259–275.
[C ossRe ]
104.
Posch, C.; Se ano-Go a edona, T.; Lina es-Ba anco, B.; Delb ück, T. Re inomo phic E en -Based Vision
Senso s: Bioinspi ed Came as wi h Spiking Ou pu . P oc. IEEE 2014,102, 1470–1484. [C ossRe ]
105.
Son, B.; Suh, Y.; Kim, S.; Jung, H.; Kim, J.; Shin, C.; Pa k, K.; Lee, K.; Pa k, J.; Woo, J.; e al. A 640
×
480
dynamic ision senso wi h a 9um pixel and 300Meps add ess-e en ep esen a ion. IEEE In l. Solid-S a e
Ci cui s Con . 2017. [C ossRe ]
106.
Guo, M.; Huang, J.; Chen, S. Li e demons a ion: A 768
×
640 pixels 200 Meps dynamic ision senso .
In P oceedings
o he 2017 IEEE In e na ional Symposium on Ci cui s and Sys ems (ISCAS), Bal imo e, MD,
USA , 28–31 May 2017 .
107.
Lyon, R.F.; Mead, C. An analog elec onic cochlea. IEEE T ans. Acous . Speech Signal P ocess.
1988
,36,
1119–1134. [C ossRe ]
108.
Chan, V.; Liu, S.; an Schaik, A. AER EAR: A Ma ched Silicon Cochlea Pai wi h Add ess E en
Rep esen a ion In e ace. IEEE T ans. Ci cui s Sys . I Regul. Pap. 2007,54, 48–59. [C ossRe ]
109.
Wen, B.; Boahen, K. A Silicon Cochlea Wi h Ac i e Coupling. IEEE T ans. Biomed. Ci cui s Sys .
2009
,3,
444–455. [C ossRe ]
110.
Ca iglia, S.; Pinna, L.; Valle, M.; Ba olozzi, C. Spike-Based Readou o POSFET Tac ile Senso s. IEEE T ans.
Ci cui s Sys . I 2017,64, 1421–1431. [C ossRe ]
111.
Ros, P.M.; C epaldi, M.; Dema chi, D. A hyb id quasi-digi al/neu omo phic a chi ec u e o ac ile sensing
in humanoid obo s. In P oceedings o he In e na ional Wo kshop on Ad ances in Senso s and In e aces,
Gallipoli, I aly, 18–19 June 2015; pp. 126–130.
112.
Os e , M.; Douglas, R.; Liu, S.C. Compu a ion wi h Spikes in a Winne -Take-All Ne wo k. Neu al Compu .
2009,21, 2437–2465. [C ossRe ]
113.
Camuñas-Mesa, L.; Zama eño-Ramos, C.; Lina es-Ba anco, A.; Acos a-Jiménez, A.; Se ano-Go a edona, T.;
Lina es-Ba anco, B. An e en -d i en mul i-ke nel con olu ion p ocesso module o e en -d i en ision
senso s. IEEE J. Solid-S a e Ci cui s 2012,47, 504–517. [C ossRe ]
Ma e ials 2019,12, 2745 25 o 28
114.
Camuñas-Mesa, L.A.; Domínguez-Co de o, Y.L.; Lina es-Ba anco, A.; Se ano-Go a edona, T.;
Lina es-Ba anco, B. A Con igu able E en -D i en Con olu ional Node wi h Ra e Sa u a ion Mechanism o
Modula Con Ne Sys ems Implemen a ion. F on . Neu osci. 2018,12, 63. [C ossRe ]
115.
Camuñas-Mesa, L.A.; Se ano-Go a edona, T.; Lina es-Ba anco, B. E en -d i en sensing and p ocessing o
high-speed obo ic ision. In P oceedings o he IEEE Biomedical Ci cui s and Sys ems Con e ence (BioCAS)
P oceedings, Lausanne, Swi ze land, 22–24 Oc obe 2014; pp. 516–519.
116.
Indi e i, G. Modeling Selec i e A en ion Using a Neu omo phic Analog VLSI De ice. Neu al Compu .
2000
,
12, 2857–2880. [C ossRe ]
117.
Sch auwen, B.; D’Haene, M.; Ve s ae en, D.; Campenhou , J. Compac ha dwa e liquid s a e machines on
FPGA o eal- ime speech ecogni ion. Neu al Ne w. 2008,21, 511–523. [C ossRe ]
118.
Aloma , M.L.; Canals, V.; Mo o, A.; Oli e , A.; Rossello, J.L. S ochas ic ha dwa e implemen a ion o Liquid
S a e Machines. In P oceedings o he In e na ional Join Con e ence on Neu al Ne wo ks, Vancou e , BC,
Canada, 24–29 July 2016; pp. 1128–1133.
119.
Liu, S.C.; Delb uck, T.; Indi e i, G.; Wha ley, A.; Douglas, R. E en -Based Neu omo phic Sys ems; Wiley:
Hoboken, NJ, USA, 2015.
120.
Me olla, P.A.; A hu , J.V.; Al a ez-Icaza, R.; Cassidy, A.S.; Sawada, J.; Akopyan, F.; Jackson, B.L.; Imam, N.;
Guo, C.; Nakamu a, Y.; e al. A million spiking-neu on in eg a ed ci cui wi h a scalable communica ion
ne wo k and in e ace. Science 2014,345, 668–673. [C ossRe ]
121.
Benjamin, B.V.; Gao, P.; McQuinn, E.; Choudha y, S.; Chand aseka an, A.R.; Bussa , J.M.; Al a ez-Icaza, R.;
A hu , J.V.; Me olla, P.A.; Boahen, K. Neu og id: A mixed-analog-digi al mul ichip sys em o la ge-scale
neu al simua ions. P oc. IEEE 2014,102, 699–716. [C ossRe ]
122.
Necka , A.S. B aind op: A Mixed Signal Neu omo phic A chi ec u e wi h a Dynamical Sys ems-Based
P og amming Model. Ph.D. Thesis, S an o d Uni e si y, S an o d, CA, USA, 2018.
123.
Necka , A.; Fok, S.; Benjamin, B.; S ewa , T.; Oza, N.; Voelke , A.; Eliasmi h, C.; Manoha , R.; Boahen, K.
B aind op: A Mixed-Signal Neu omo phic A chi ec u e Wi h a Dynamical Sys ems-Based P og amming
Model. P oc. IEEE 2019,107, 144–164. [C ossRe ]
124.
Schemmel, J.; B iide le, D.; G iibl, A.; Hock, M.; Meie , K.; Millne , S. A wa e -scale neu omo phic ha dwa e
sys em o la ge-scale neu al modeling.
In P oceedings
o he 2010 IEEE In e na ional Symposium on
Ci cui s and Sys ems, Pa is, F ance, 30 May–2 June 2010; pp. 1947–1950.
125.
Fu be , S.B.; Galluppi, F.; Temple, S.; Plana, L.A. The SpiNNake p ojec . P oc. IEEE
2014
,102, 652–65.
[C ossRe ]
126.
Da ies, L.; S ini asa, N.; Lin, T.; Chinya, G.; Cao, Y.; Choday, S.; Dimou, G.; Joshi, P.; Imam, N.; Jain, S.; e al.
Loihi: A Neu omo phic Manyco e P ocesso wi h On-Chip Lea ning. IEEE Mic o
2018
,38, 82–99. [C ossRe ]
127.
Ma, D.; Shen, J.C.; Gu, Z.H.; Zhang, M.; Zhu, X.; Xu, X.; Xu, Q.; Shen, Y.; Pan, G. Da win: A neu omo phic
ha dwa e co-p ocesso based on Spiking Neu al Ne wo ks. Sci. China In . Sci. 2016,59, 023401. [C ossRe ]
128.
Qiao, N.; Mos a a, H.; Co adi, F.; Osswald, M.; S e anini, F.; Sumislawska, D.; Indi e i, G. A e-con igu able
on-line lea ning spiking neu omo phic p ocesso comp ising 256 neu ons and 128K synapses. F on . Neu osci.
2015,9, 141. [C ossRe ]
129.
F enkel, C.; Le eb e, M.; Lega , J.; Bol, D. A 0.086-mm
2
12.7-pJ/SOP 64k-Synapse 256-Neu on
Online-Lea ning Digi al Spiking Neu omo phic P ocesso in 28-nm CMOS. IEEE T ans. Biomed. Ci cui s Sys .
2019,13, 145–158.
130.
E yilmaz, S.B.; Joshi, S.; Ne ci, E.; Wan, W.; Cauwenbe ghs, G.; Wong, H.P. Neu omo phic a chi ec u es
wi h elec onic synapses. In P oceedings o he 17 h In e na ional Symposium on Quali y Elec onic Design
(ISQED), San a Cla a, CA, USA, 15–16 Ma ch 2016; pp. 118–123.
131.
Su i, M.; Bichle , O.; Que lioz, D.; Cue o, O.; Pe niola, L.; Sousa, V.; Vuillaume, D.; Gam a , C.; DeSal o, B.
Phase change memo y as synapse o ul a-dense neu omo phic sys ems: Applica ion o complex isual
pa e n ex ac ion. In P oceedings o he IEEE In e na ional Elec on De ices Mee ing (IEDM), Washing on,
DC, USA, 5–7 Decembe 2011.
132.
Valo , I.; Wase , R.; Jameson, J.; Kozicki, M. Elec ochemical me alliza ion memo ies- undamen als,
applica ions, p ospec s. Nano echnology 2011,22, 254003. [C ossRe ]
133.
Chan hbouala, A.; Ga cia, V.; Che i i, R.; Bouzehouane, K.; Fusil, S.; Moya, X.; Xa ie , S.; Yamada, H.;
De anlo , C.; Ma hu , N.; e al. A e oelec ic mem is o . Na . Ma e . 2012,11, 860–864. [C ossRe ]