STDP and STDP Variations with Memristors for Spiking Neuromorphic Learning Systems

FOCUSED REVIEW published: 18 February 2013 doi: 10.3389/fnins.2013.00002 STDP and STDP variations with memristors for spiking neuromorphic learning systems T. Serrano-Gotarredona1,T. Masquelier2,3,T. Prodromakis4,G. Indiveri5 and B. Linares-Barranco 1* 1Department of Analog and Mixed-Signal Design, Instituto de Microelectrónica de Sevilla, IMSE-CNM-CSIC, Sevilla, Spain 2Unit for Brain and Cognition, Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain 3Laboratory of Neurobiology of Adaptive Processes, UMR 7102, CNRS - University Pierre and Marie Curie, Paris, France 4Centre for Bio-inspired Technology, Institute of Biomedical Engineering, Imperial College London 5Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland In this paper we review several ways of realizing asynchronous Spike-Timing-DependentPlasticity (STDP) using memristors as synapses. Our focus is on how to use individual memristors to implement synaptic weight multiplications, in a way such that it is not necessary to (a) introduce global synchronization and (b) to separate memristor learning phases from memristor performing phases. In the approaches described, neurons fire spikes asynchronously when they wish and memristive synapses perform computation and learn at their own pace, as it happens in biological neural systems. We distinguish between two different memristor physics, depending on whether they respond to the original “moving wall” or to the “filament creation and annihilation” models. Independent of the memristor physics, we discuss two different types of STDP rules that can be implemented with memristors: either the pure timing-based rule that takes into account the arrival time of the spikes from the preand the post-synaptic neurons, or a hybrid rule that takes into account only the timing of pre-synaptic spikes and the membrane potential and other state variables of the post-synaptic neuron. We show how to implement these rules in cross-bar architectures that comprise massive arrays of memristors, and we discuss applications for artificial vision. Keywords: memristor/cmos, artificial-learning-synapses, spike-timing-dependent-plasticity, spiking-neuralnetworks Edited by: Gert Cauwenberghs, University of California, San Diego, USA Reviewed by: Emre O. Neftci, Institute of Neuroinformatics, Switzerland Siddharth Joshi, University of California, San Diego, USA *Correspondence: B. Linares-Barranco, BSc Physics 1986, MSc 1987, PhD 1990 from University of Seville, Spain, and 2nd PhD 1991 from Texas A&M University, USA. He is Full professor of Research at the Instituto de Microelectrónica de Sevilla (IMSE-CNM-CSIC). He has been involved with circuit design for telecommunication circuits, VLSI emulators of biological neurons, VLSI neural based pattern recognition systems, hearing aids, precision circuit design for instrumentation equipment, bio-inspired VLSI vision processing systems, AER, memristors and emerging nanodevices, and VLSI transistor mismatch parameters characterization. He is IEEE Fellow. [email protected] 1. INTRODUCTION For many years, the field of neuromorphic engineering has struggled to develop practical neuro-computing devices that mimicked the principles and operations of biological brains, by directly exploiting the physics of electronic devices in mixed analog/digital VLSI (Indiveri and Horiuchi, 2011). However, there always was aclamorforacompactanddistributednonvolatile memory, possibly tightly coupled to the signal processing components (neurons), so that the biological synapses counterparts could be properly emulated. The recent advent of nanoscale memristive-like devices (Wuttig and Yamada, 2007; Strukov et al., 2008; Yang et al., 2008; Jo et al., 2010; Govoreanu et al., 2011; Lee et al., 2011; Chanthbouala et al., 2012; Kuzum et al., 2012; Prodromakis et al., 2012a)opens the possibility of large-scale bio-inspired neural network implementations with minimal sizerequirements for those elements in the circuit that are most numerous and therefore most Frontiers in Neuroscience www.frontiersin.org February 2013 | Volume 7 | Article 2 |1 Serrano-Gotarredona et al. STDP and STDP variations with memristors space-intense: plastic synaptic connections. The strength of a synaptic link between two neighboring neurons depends on its history and more explicitly by the overall amount of neurotransmitters that has been propagated through it after a relevant neural spike. In similar fashion, the strength of a memristor, i.e., its memristance Memristor Two terminal electronic device which operates similar to a resistor, but whose resistance changes dynamically as the device is being used. Spike-Timing-Dependent-Plasticity (STDP) One type of learning rule for artificial synapses in spiking neural networks, where the synaptic update depends on the timing characteristics of individual spikes at the synapse terminals. (or instantaneous resistance) is dictated by the amount of charge qthat has flown through it or the accumulated voltage flux φ. Additionally, the intrinsic non-linear nature of practical solidstate memristors resembles the behavior of neural synapses. On the other hand, and independently of the new nanoscale devices availability, the neuromorphic engineering field evolved naturally toward circuits and systems exploiting spiking signal encoding, as in biology. For example, a large collection of spike-driven vision sensors have been reported, such as sensors for luminance (Culurciello et al., 2003; Chen et al., 2011), temporal contrast (Barbaro et al., 2002; Mallik et al., 2005; Chan et al., 2007a; Lichtsteiner et al., 2008; Leñero-Bardallo et al., 2011; Posch et al., 2011; Serrano-Gotarredona et al., 2013), motion (Kramer, 1996; Sarpeshkar et al., 1996; Ozalevli and Higgins, 2005), and spatial contrast (Ruedi et al., 2003; Zaghloul and Boahen, 2004; Costas-Santos et al., 2007; Massari et al., 2008; Leñero-Bardallo et al., 2010). Spike-driven principles have also been used for auditory systems (Sarpeshkar et al., 2005; Wen and Boahen, 2006, 2009; Chan et al., 2007b), competition and Winner-TakeAll networks (Indiveri, 2000; Chicca et al., 2007; Oster et al., 2008), learning (Mill et al., 2011), classification (Mitra et al., 2009), fall detection (Fu et al., 2008), and systems distributed over wireless sensor networks (Teixeira et al., 2005; Massari et al., 2008). Apart from realtime sensing, spike-driven processing systems can produce extremely fast responses. Examples of spike-driven processing modules (chips) are those that, emulating biological neocortical structures, perform spatio-temporal feature extraction such as fixed-kernel (Venier et al., 1997; Choi et al., 2005) or programmable kernel (Serrano-Gotarredona et al., 2006; CamuñasMesa et al., 2011, 2012) 2D convolutions, and generic massive neural processing (Vogelstein et al., 2007; Fieres et al., 2008; Khan et al., 2008; Serrano-Gotarredona et al., 2009; ZamarrenoRamos, 2012). Unavoidably, the learning capability is one key characteristic that is required for building cognitive artificial neural systems. Recently proposed artificial neural processing systems spend great resources for this task: the multi-million European initiative FACETS/BrainScales (Fieres et al., 2008) is developing a 200.000 neuron wafer1where most of the silicon area is used for implementing Spike-Timing-DependentPlasticity (STDP) learning mechanisms in the synapses. The UK initiative SpiNNaker (Khan et al., 2008) based on multi-processors ARM technology has to use hybrid packaging technology in order to encapsulate two separate Silicon chips into each chip package: one chip is being the genuine SpiNNaker chip with 18 ARM 2CPUs, and the second chip being a commercial 128MB DRAM chip for local synaptic storage. Both the learning mechanisms and the storage of learned parameters require substantial silicon real-estate in traditional silicon-based chip technology. However, the advent of new nanoscale technologies has shed new expectations, giving hopes for the development of ultra-compact, fast and efficient learning and storage mechanisms that may result in affordable, low power, compact, large scale, artificial neural systems (Wuttig and Yamada, 2007; Strukov et al., 2008; Yang et al., 2008; Jo et al., 2010; Govoreanu et al., 2011; Lee et al., 2011; Chanthbouala et al., 2012; Kuzum et al., 2012; Prodromakis et al., 2012a). A very promising new class of nanoscale devices is the one that comprises the so called memristors (Chua, 1971; Chua and Kang, 1976; Strukov et al., 2008; Borghetti et al., 2009; Jo et al., 2009, 2010), whose distinct characteristic is that they have memory while they operate like variable two-terminal resistors.Itwasrecentlypostulated that such tiny nanoscale devices, when driven by appropriately shaped voltage pulses, could be embedded within traditional CMOS3 microchips, resulting in truly asynchronous4 artificial learning neural “tissue” equipped 1Wafer: microchips are fabricated on silicon wafers (with diameter ranging from 1 to about 18), which can hold several hundreds or thousands of individual chips that are later on cut and encapsulated into chip packages. The FACETS/BrainScaleS project is a wafer-scale design, meaning that the wafer is not cut into individual chips, but the whole wafer is used as a unit circuit. 2ARM stands for “Acorn Risk Microprocessor” and is the name of a company providing embedded microprocessors for a variety of more complex chips, like cellular phones, usb-sticks, etc. 3CMOS stands for “Complementary Metal Oxide Semiconductor” and refers to the most standard technology used for microchip fabrication. 4In asynchronous systems no clock is required, as opposed to conventional digital computing systems. Consequently, there is no centralized time keeper that enforces actions to happen in lock-step with each other. Frontiers in Neuroscience www.frontiersin.org February 2013 | Volume 7 | Article 2 |2 Serrano-Gotarredona et al. STDP and STDP variations with memristors with STDP (Linares-Barranco and SerranoGotarredona, 2009b,a; Zamarreño-Ramos et al., 2011). Although this still needs to be proven experimentally and all practical limitations are yet to be identified, while memristors are continuously being improved and optimized over many labs worldwide, the potential of building very dense hybrid memristive-CMOS learning systems is there. The resulting implementations can be extremely compact STDP-equipped systems, which contrast with pure CMOS-based attempts that either have resulted in physical STDP synapses consuming significant chip realestate (Fieres et al., 2008)orcomplexcomputational work arounds in more algorithmic solutions (Rast et al., 2010; Davies et al., 2012). In this paper we quickly review the basic principles behind exploiting memristance for asynchronous STDP and extend the original findings to other types of STDP. In the next section we quickly review the memristor concept as well as some of the postulated physical mechanisms responsible for its operation. After this, section 3summarizes STDP and some variation of it, as well as additive, multiplicative and quadratic STDP. Sections 4and 5review how to combine memristors with specific CMOS neurons to result in different types of STDP. Section 6mentions an application in the context of artificial vision, and section 7provides conclusions. 2. MEMRISTORS Memristance was postulated in 1971 by Chua (1971) as the fourth missing canonical circuit element through his famous symmetry argument, illustrated in Figure 1. According to circuit theoretical fundamentals, there are four basic electrical quantities (Chua et al., 1987): (1) voltage difference between two terminals “v,” (2) current flowing through into a device terminal “i,” (3) charge flowing through a device terminal or integral of current q=i(τ)dτ,and(4) flux or integral of voltage φ=v(τ)dτ.Atwoterminal device is said to be canonical (Chua et al., 1987) if either two of the four basic electrical quantities are related by a static5relationship, as shown in Figure 1. A resistor has a static relationship between terminal voltage vand device current i,asshowninFigure 1B. A capacitor shows a static relationship between charge qand voltage v,asshowninFigure 1C.Aninductor has a static relationship between its current i and flux φ,asshowninFigure 1D.Thesethree devices have been very well known since the origins of Electronics and Electricity. However, there are other possibilities for combining the 5By “static” we mean it is not altered by changes of the above electrical quantities, or by their history, integrals, derivatives, etc. These “static” curves can, however, be time-varying if the change is caused by an external agent. For example, a motor driven potentiometer would have a “static” i/vcurve that is time varying. FIGURE 1 | Four variables of circuit theory linked by six mathematical relations consisting of the functional relationships of the four passive circuit elements, Faradays law of induction and the definition of electric current. (A) Chua’s symmetry argument and (B–E) descriptions of the four canonical two-terminal devices. (B) A resistor is defined by a static relationship between a device’s voltage and current. (C) A capacitor is defined by a static relationship between a device’s charge and voltage. (D) An inductor is defined by a static relationship between a device’s current and flux. (E) And a memristor is defined by a static relationship between a device’s charge and flux. Frontiers in Neuroscience www.frontiersin.org February 2013 | Volume 7 | Article 2 |3 Serrano-Gotarredona et al. STDP and STDP variations with memristors four basic electrical quantities: (q,i),(v,φ),and (q,φ). Ignoring the combinations of a quantity with its own time derivative leaves us with one single additional possibility: (q,φ).This reasoning led Chua to postulate the existence of a fourth basic two-terminal element, which he called the Memristor.Memristors behave as resistances in which the resistance changes through some of the basic electrical quantities, and is somehow memorized. The memristor would show a static relationship between charge qand flux φ,asshowninFigure 1E.If the qvs. φrelationship is linear, the memristor degenerates into a linear resistor. Although none of the so-far reported memristors can be described by a static constitutive relationship in the (q,φ)plane (and thus, strictly speaking, the 1971 fourth canonical element is still missing), they all fall within Chua’s 1976 generalization of Memristive Systems (Chua and Kang, 1976). From here on we will use the term memristor for Chua’s 1976 definition of memristive system. Consequently, the simple concept of memristance as defined in Figure 1D can be extended to refer to any device exhibiting resistive behavior (its i/vcurves cross the origin) whose resistance can change through some of the four basic electrical quantities (or a combination of them, or their time derivatives or integrals, etc.), while at the same time exhibiting memory for that resistance. In that case, more elaborate mathematical descriptions are required (Chua and Kang, 1976). Memristance has recently been demonstrated (with extraordinary impact among the research community) in nanoscale twoterminal devices, such as certain titaniumdioxide (Strukov et al., 2008; Borghetti et al., 2009; Prodromakis et al., 2011, 2012a)and amorphous Silicon (Jo et al., 2009) crosspoint switches. However, memristive devices were reported earlier by other groups (Argall, 1968; Prodromakis et al., 2012b). Memristance arises naturally in nanoscale devices because small voltages can yield enormous electric fields that produce the motion of charged atomic or molecular species, changing structural properties of a device (such as its doping profile) while it operates. Its functional characteristic has been a pinched hysteresis loop in the i–v domain (Figures 2C,D); a signature that has been observed in various dissipative devices (Prodromakis et al., 2012b). Particularly nowadays various emerging resistive random-access memory (ReRAM) nano-devices (Chua, 2011), with one scaling extreme being the atomic switch (Terabe et al., 2005), are classified as being memristors, and show attributes that resemble biological synapses (Ohno, 2011)providing exciting prospects for demonstrating neuromorphic applications (Avizienis et al., 2012). Hysteresis is typically noticed in systems/devices that possess certain inertia, causing the value of a physical property to lag behind changes in the mechanism causing it; manifesting memory (Pershin and Di Ventra, 2011). Particularly in the case of nanoscale memristors, this inertia has been ascribed to Joule heating (Fursina et al., 2009), the electrochemical migration of oxygen ions (Nian et al., 2007) and vacancies (Yang et al., 2008), the lowering of Schottky barrier heights by trapped charge carriers at interfacial states (Hur et al., 2010), the phase-change (Wuttig and Yamada, 2007), the formation/rupture of conductive filaments (Kwon et al., 2010), Yang et al. (2012) in a device’s core, or even to some extent a combination of the aforementioned switching mechanisms. Clearly, the impact of memristors is foreseen to be realized through their nanometric dimensions (see Figure 2B which is a cross section of one of the structures in Figure 2A), their capacity to store multiple bits or a continuum of information per element (Figure 2E) and the minuscule energy required to write distinct states, resulting in high spatialand high storage-density well beyond the current state-of-the-art (Govoreanu et al., 2011). Nonetheless, the fact that the functional properties of such elements are associated with rate-limiting (frequency-dependent) electroor thermo-dynamic changes that are contingent on both the present as well as the past environment, presents us with opportunities in exploiting them as novel computation elements. By definition, memristors can be either voltage/flux driven or current/charge driven. Depending on the polarity of the set and reset potentials required to change resistive states (RS), the devices can be classified as unipolar (URS) or bipolar (BRS) (Schindler et al., 2007) and consequently, their circuit symbol must indicate somehow their polarity, as depicted in Figure 3A. Voltage/flux driven memristors can be described by (Chua and Kang, 1976) iMR =G(w,vMR,t)vMR (1) ˙w=f(w,vMR,t)(2) while current/charge driven memristors would be described as (Chua and Kang, 1976) vMR =R(w,iMR,t)iMR (3) ˙w=f(w,iMR,t)(4) Frontiers in Neuroscience www.frontiersin.org February 2013 | Volume 7 | Article 2 |4 Serrano-Gotarredona et al. STDP and STDP variations with memristors FIGURE 2 | Solid-state TiO2-based memristors fabricated at Imperial College London. (A) Microphotograph of a memristor cross-bar array, with a close-up SEM illustration of a single cell appearing in the inset of (A).(B) CHEMI-STEM map of a lamella cross-section of one of the devices shown in (A): blue denotes Pt (top and bottom electrodes) while green and red correspond to Ti and O2species (Prodromakis et al., 2012a). (C) Simulated and measured pinched hysteresis I-V characteristics (absolute memristor current |I|vs. signed memristor voltage V) (Prodromakis et al., 2011) in log scale or (D) linear scale, and (E) multi-state programming of a TiO2-based memristor: read pulses are positive and small amplitude (1V) that do not alter the resistance (memristance) of the memristor, while successive set pulses have negative high amplitude (3V) and do progressively alter the resistance (memristance) of the memristor. Here wrepresents some structural property parameter of the memristor. For example, in the 2008 HP paper (Strukov et al., 2008)the operation of the reported memristor was postulated as described by the moving wall model depicted in Figure 3B. In this simplified model a memristor of height L, sandwiched between two electrodes, has a low resistance region of height wand a high resistance region of height L−w. The memristor is considered to be divided into two regions. Both regions are separated by a boundary wall at position w,whichmovesup Frontiers in Neuroscience www.frontiersin.org February 2013 | Volume 7 | Article 2 |5 Serrano-Gotarredona et al. STDP and STDP variations with memristors FIGURE 3 | (A) Memristor asymmetric symbols. (B) Illustration of moving wall model describing memristor operation as two variable resistors in series. (C) Illustration of filament formation/annihilation model describing memristor operation as two variable resistances in parallel. (D) Experimentally measured STDP function ξ(T)on biological synapses (data from Bi and Poo, 1998, 2001). (E) Ideal STDP update function used in computational models of STDP synaptic learning. (F) Anti-STDP learning function for inhibitory STDP synapses. (G) Shape of memristor weight update function f(vMR),(H) spike-shape waveform. and down with the amount of charge that has flown through the memristor (in the case of being current/charge driven) or the accumulated flux (in case of being voltage/flux driven). The memristor would behave as two variable resistors in series. The total effective resistance of the memristor would be described by R=RON w L+ROFF 1−w L(5) This moving wall model can approximate phenomena like migration of oxygen ions (Nian et al., 2007) and vacancies (Yang et al., 2008), the lowering of Schottky barrier heights by trapped charge carriers at interfacial states (Hur et al., 2010), and the phase-change in some PCM (phase change materials) devices (Wuttig and Yamada, 2007). However, resistive switching effects in dielectric-based devices have normally been assumed to be caused by conducting filament formation across the electrodes, although the understanding and modeling of these phenomena remains controversial. As a matter of fact, some researchers are observing the formation and annihilation of nanoscale width conducting filaments in memristors (Kwon et al., 2010; Yang et al., 2012). Precise modeling of this phenomenon is still under Frontiers in Neuroscience www.frontiersin.org February 2013 | Volume 7 | Article 2 |6 Serrano-Gotarredona et al. STDP and STDP variations with memristors research (Shihong et al., 2012). However, let us here propose the following very simplified view to approximate this physical mechanism. Figure 3C illustrates schematically a memristor with several conducting filaments between the two electrodes. The number of filaments or their cross-sectional area would increase or decrease with memristor operation. Let us call now wthe total cross sectional area of the effective conducting filaments at a given instant in time, and Sthe total cross section area of the memristor. The filaments present high conductivity (low resistivity), while the bulk presents much lower conductivity (high resistivity). All formed parallel filaments behave as one effective resistance of low resistance, whiletherestofthebulkbehavesasanother higher resistivity resistor. Therefore, now the memristor behaves as two variable resistors in parallel. Consequently, its total conductance (inverse of resistance) could be described as G=GON w S+GOFF 1−w S(6) where GON is the conductance per effective cross section area of the filaments, and GOFF is the conductance per effective cross section area of the filament-less bulk material. Parameter w would change from 0 to wmax, the maximum possible effective cross section area of total conductingfilaments(wmax ≤S). This changing cross section description not only approximates filament formation/annihilation phenomena, but also some other gradual cross section area variations observed in some phase-change or ferroelectricdomains-based materials (Chanthbouala et al., 2012). As we will highlight later in sections 4and Artificial learning synapses Artificially manufactured device that behaves similar to a biological synapse, e.g., it’s communication strength (or synaptic weight) changes as the device is used according to some learning rule. Nanoscale artificial synapse This is an artificial synapse made using some device whose dimensions are below the micron (10−6m). Tunable STDP STDP learning rule whose mathematical description can be made to change in time. 5, whether a memristor is better described by the moving wall model or the filament formation/annihilation model, impacts severely on the resulting type of STDP learning mechanism. The latter yields an additive type of STDP, while the former results in a quadratic type STDP. Note that a memristor can be either voltage/flux or current/charge driven, independently of whether it is a “wall” or a “filament” memristor. 3. SPIKE-TIMING-DEPENDENT-PLASTICITY STDP is the ability of natural or artificial synapses to change their strength according to the precise timing of individual preand/or post-synaptic spikes (Gerstner et al., 1993, 1996; Markram et al., 1997; Bi and Poo, 1998, 2001; Zhang et al., 1998; Feldman, 2000; Mu and Poo, 2006; Cassenaer and Laurent, 2007; Jacob et al., 2007; Young, 2007; Finelli et al., 2008; Masquelier et al., 2008, 2009). A nice overview of STDP and its history can be found elsewhere (Sjöström and Gerstner, 2010). STDP learning in biology is inherently asynchronous and on-line, meaning that synaptic incremental update occurs while neurons and synapses transmit spikes and perform computations. This contrasts to more traditional learning rules, like backpropagation (Rojas, 1996), where first neurons and synapses perform signal aggregation and neural state update (we call this here “performing phase”) and then synaptic updates are computed and applied (we call this here “weight update phase”) alternating these two phases during training. Even early proposals for memristor-based STDP learning implementations used artificial time-multiplexing to alternate continuously and synchronously between “performing” and “weight update” phases (Snider, 2008), thus requiring global systemwide synchronization. This can become a severe handicap when scaling up systems to arbitrary size. Here we show a fully asynchronous implementation for memristor-based STDP where “performing” and “weight update” phases happen simultaneously in a natural manner, as in biology (Linares-Barranco and SerranoGotarredona, 2009b,a; Zamarreño-Ramos et al., 2011), where there is no need for any global synchronization. Other researchers have proposed variations around these ideas (Bichler et al., 2012a; Kuzum et al., 2012). Figure 3D shows the change of synaptic strength (in percent) measured experimentally from biological synapses as function of relative timing T=tpos −tpre between the arrival time tpre of a pre-synaptic spike and the time tpos of generation of a post-synaptic spike. Although the data shows stochasticity, we can infer an underlying interpolated function ξ(T)as shown in Figure 3E. ξ(T)=a+e−T/τ+if T>0 −a−eT/τ−if T<0(7) For a causal pre to post spike timing relation (T>0) the strength of the synapse is increased, while for an anti-causal relation (T<0) it is decreased. In the case of synapses with negative synaptic strength (as in some artificial realizations), the reversed version shown in Figure 3F can be used. Microchip CMOS circuit implementations of STDP rules that follow the description of Equation (7) have been reported (Indiveri et al., 2006), which result in about 30 Frontiers in Neuroscience www.frontiersin.org February 2013 | Volume 7 | Article 2 |7 Serrano-Gotarredona et al. STDP and STDP variations with memristors transistors per plastic synapse, thus demonstrating the very high cost of their hardware realization. Let us call this double-spike STDP,sincethe weight will be updated after the arrival of the second spike (either preor post-synaptic). Alternative variations of STDP have been proposed that do not require the intervention of both preand post-synaptic spikes (Brader et al., 2007), resulting in slightly less complex circuit implementations (Mitra et al., 2009). Let us call this single-spike STDP,sincetheweightwillbe updated after the arrival of pre-synaptic spikes only.Thissingle-spikeSTDPruleupdatesthe synaptic weight depending on the value of two local neural soma state variables. The first one is the membrane voltage V(t)and the second one is an auxiliary state variable C(t)proportional to the neuron’s firing rate and equivalent to the biological neuron’s Calcium concentration, which has the following dynamics ˙ C=− C(t) τC +JC i δ(t−ti)(8) where JCrepresents the contribution of one single post-synaptic spike and the time constant τCis comparable to the STDP learning window T. The synaptic weight variable ξis updated only when a pre-synaptic spike occurs at time tpre. The synaptic strength is increased or decreased by fixed size steps |a±|depending on the instantaneous values of V(tpre)and C(tpre) with respect to a given set of global thresholds {θv,θl up,θh up,θl down,θh down},as: ξ(tpre)=⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ a+if V(tpre)>θvand θl up <C(tpre)<θh up −a−if V(tpre)<θvand θl down <C(tpre)<θh down (9) Additionally, in this model the synaptic strength drifts slowly toward its upper or lower bound depending on whether it is above or below an intermediate threshold. Both types of STDP rules, double-spike and single-spike, are very expensive to implement in conventional CMOS microchips (Indiveri et al., 2006; Fieres et al., 2008; Khan et al., 2008; Mitra et al., 2009). However, as we will see in the next section, both can be implemented with just one memristor per synapse if appropriate peripheral signal conditioning neurons are used in hybrid CMOS/memristor realizations. Hybrid nano/CMOS neural system Artificial neural network system built using conventional microchip technology (CMOS) combined with presently emerging nanoscale devices. Independently on whether STDP is either double-spike or single-spike, it is said to be either additive, multiplicative or quadratic if it additionally depends or not on the actual synaptic strength. If the STDP update is independent of the actual synaptic strength, it is said to be additive. Additive STDP requires the weight values to be bounded to an interval because weights will stabilize at one of their boundary values (van Rossum et al., 2000; Rubin et al., 2001). If the synaptic update is proportional to actual synaptic strength, it is called multiplicative STDP and weights may stabilize to values intermediate to their boundaries (van Rossum et al., 2000; Rubin et al., 2001; Gütig et al., 2003). If the synaptic weight update is proportional to the square of actual synaptic strength, we call it quadratic STDP (Zamarreño-Ramos et al., 2011). 4. MEMRISTORS AND CMOS NEURONS FOR DOUBLE-SPIKE STDP The more traditional double-spike STDP learning rule [as modeled by Equation (7)] can, in theory, be implemented by (Zamarreño-Ramos et al., 2011) (a) using a particular type of voltage/flux driven memristor (Jo et al., 2010) whose operation might be approximated by Equation (2)with(seeFigure 3G) f(vMR)=⎧ ⎨ ⎩ Iosign(vMR)e|vMR|/vo−evth/vo if |vMR|>vth 0otherwise (10) and bounded synaptic strength w∈ [wmin,wmax], while (b) providing appropriately shaped preand post-synaptic spikes available at both synapse (memristor) electrodes (Zamarreño-Ramos et al., 2011). For example, consider a pair of identical preand post-synaptic spikes with a shape resembling that of biological spikes, with an on-set duration |t+ ail|and a tail of duration |t− ail|,asshownin Figure 3H, spk(t)=⎧ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎩ A+ mp et/τ+−e−t+ ail/τ+ 1−e−t+ ail/τ+if −t+ ail <t<0 −A− mp e−t/τ−−e−t− ail/τ− 1−e−t− ail/τ−if 0 <t<t− ail 0otherwise (11) Under these circumstances, memristor voltage is vMR(t,t)=αposspk(t)−αprespk(t+t)and from Equations (2,10) synaptic strength update can be computed as w(T)=f(vMR(t,T))dt =ξ(T) (12) Frontiers in Neuroscience www.frontiersin.org February 2013 | Volume 7 | Article 2 |8 Serrano-Gotarredona et al. STDP and STDP variations with memristors which has been shown to result in the same shape illustrated in Figure 3E (ZamarreñoRamos et al., 2011). Furthermore, by reshaping thespikewaveformonecanfinetuneorcompletely alter the STDP learning function ξ(t), as illustrated in Figure 4 (Linares-Barranco and Serrano-Gotarredona, 2009a). This way, by building neurons with a given degree of shape programmability, it is possible to change the STDP learning function at will, depending on the application, or make it evolve in time as learning progresses. Figure 5A shows a way of interconnecting memristors and CMOS neurons for STDP learning. Triangles represent the neuron soma, being the flat side its input (dendrites) and the sharp side the output (axon). Dark rectangles are memristors, representing each one synaptic junction. Each neuron controls the voltage at its input (Vpost in Figure 5B)andoutput (Vpre in Figure 5B) nodes. When the neuron is not spiking it forces a constant voltage at both nodes, while collecting through its input node the sum of input synaptic spike currents coming from the memristors, which contribute to changing the neuron internal state. When the neuron spikes, it sets a one-spike waveform at both input and output nodes. This way, they send their output spikes forward as pre-synaptic spikes for the destination synaptic memristors, but also backward to preceding synaptic memristors as post-synaptic spikes. Zamarreño et al. showed extensive simulations on these concepts, and how one can change from STDP to anti-STDP by switching polarities of spikes or memristors (Zamarreño-Ramos et al., 2011). FIGURE 4 | Illustration of influence of action potential shapes on the resulting STDP memristor weight update function ξ(T). Memristor upper and lower thresholds are normalized to amplitudes ±1.0. From (A1,A2) to (E1,E2) the same spike waveform travels forward and backward. In (F1,F2) the forward and backward waveforms are the same but have opposite polarity. In (G1,G2) to (H1,H2) the forward and backward waveforms are different. In (G1,G2), the positive pulse of the backward waveform exceeds amplitude +1.0, thus producing negative STDP update whenever there is a post-synaptic spike alone (G2); otherwise if preand post-synaptic spikes happen within a given time window, there will be positive STDP update. Frontiers in Neuroscience www.frontiersin.org February 2013 | Volume 7 | Article 2 |9