A Current Attenuator for Efficient Memristive Crossbars Read-Out

A Current Attenuator for Efficient Memristive Crossbars Read-Out Charanraj Mohan, José M. de la Rosa, Elisa Vianello, Luca Perniola, Carlo Reita, Bernabé Linares-Barranco and Teresa Serrano-Gotarredona Abstract—This paper presents a new current attenuator circuit to scale down the inference currents in memristor based crossbars that drive integrate-and-fire neurons, which subsequently allows to reduce the size of integrating capacitors by several orders of magnitude, making IC integration possible. The proposed circuit uses a linear switch to divide the inference current and scale it down by a factor of about 104. The proposed attenuator has been designed in 130nm CMOS technology. Simulation results considering noise, process and temperature variations are shown to validate the presented approach. I. INTRODUCTION Memristors prompted an increasing interest among the neuromorphic community since these devices were successfully demonstrated in HP labs [1] as an experimental evidence of the circuit element predicted by Chua in 1971 [2], [3]. Since then, there have been a number of memristive devices developed based on diverse materials and technologies. Among others, Oxide-based Random-Access Memory (OxRAM) structures are one of the most promising approaches to implement synaptic elements in neuromorphic computing systems, because of their low-switching energy and high endurance [4]–[7]. Although OxRAMs have also been demonstrated to operate as analog memories [8]–[10], here we focus on their exploitation as binary (1-bit) memory cells. The OxRAM devices to be used in this work are optimized for binary memory applications [11]. This binary OxRAM (as highlighted in Fig. 1) has to be first formed and then switched between two states: SET or Low Resistance State (LRS)– typically in the order of kΩ– and RESET or High Resistance State (HRS)– in the order of few hundreds of kΩto MΩ. The resistance states are chosen by applying a control voltage to a selector MOSFET connected in series with the OxRAM in order to avoid sneak path currents, giving rise to a device often referred to as 1T1R, as shown in Fig. 1 [12]. In order to form the filament of the OxRAM, the selector MOSFET is biased with a high positive voltage (≈4V) at the top terminal, the bottom terminal is biased with 0V and the gate terminal voltage (1.5 V to 2.5 V) is adjusted such that the recommended compliance forming current of about 1µA is set. For a RESET operation, the top terminal is biased with 0V, the bottom terminal is biased with a positive voltage (≈4V) and the gate terminal is kept fully ON. For a SET operation the top terminal is biased with a positive voltage (≈2V), the bottom terminal is biased with 0V and the gate is set to limit the current to about 100µA. Fig. 1. 1T1R memristive synapse structure For a READ operation, the top terminal is biased with a small positive voltage (0.2 or 0.3V), the bottom terminal is biased with 0V and the gate is kept fully ON. Here we consider a memristor-based fully connected feedforward neural network, which mainly comprises the presynaptic neurons, the memristive crossbar and the postsynaptic neurons [13]. As the LRS current across a crossbar line is high due to the typical low resistance during an inference read operation (after a SET operation), an extremely large integrating capacitor would be needed (larger than nFs) at the post-synaptic neuron for reasonable integration speed, making IC integration impossible. Hence, a current attenuator is needed to scale down the read current. Several current attenuating strategies exist such as the Gilbert’s current normalizer circuit [14], MOS-ladder [15] and the Winner Take All (WTA) [16] based current attenuator [17]. This paper considers Gilbert’s current normalizer circuit as reference, on which modifications are done to ease attenuation of an inference current. The idea is based on creating a splitting of the inference current by a factor of about two by using a MOS biased in ohmic region. This paper proposes a Modified Current Normalizer (MCN) circuit for attenuating a crossbar read-out line current. Simulation results taking into account the effect of PVT variations are shown to validate the 978-1-5386-5108-7/17/$31.00 ©2018 IEEE Fig. 2. Scheme of a 4×41T1R crossbar with pre-synaptic neurons, current attenuators and post-synaptic neurons. proposed circuit technique. II. 1T1R CROSSBAR WITH PRE-SYNAPTIC NEURONS, CURRENT ATTENUATORS AND POST-SYNAPTIC NEURONS Fig. 2 shows a fully connected 4×4feed-forward neural network with pre-synaptic neurons, current attenuators and post-synaptic neurons. Circuit elements labeled as O1,2,3,....,16 comprise the 1T1R based memristor-selector synaptic devices. Each row has a pre-synaptic neuron, which is made up of a pulse-shaping digital block and an opamp that is finely calibrated [18]. Each column is connected to a current attenuator circuit, followed by a post-synaptic neuron. The post-synaptic neuron comprises a CMOS integrate-and-fire neuron, which integrates the inference current and generates an output spike when a threshold is reached. The OxRAM devices are initially formed to bring them from the Pristine Resistance State (PRS). Later, they can be switched in binary mode, i.e. either SET or RESET [19]. For reading the memristance of the OxRAMs, small read voltages of 0.2 or 0.3 V are applied across the rows, so that the aggregated inference read currents across columns are scaled down and integrated. Fig. 3. Details of MCN circuit schematics III. PROPOSED MCN CIRCUIT We propose an MCN based attenuator, which mainly includes having an MOS-resistor and a current-mirror at the load on the Gilbert’s two-input current normalizer circuit [14]. Fig. 3 shows the proposed MCN circuit connected between a column of a the crossbar and a post-synaptic neuron. It works by inserting an ohmic biased MOS (M6) which creates a current splitting of the crossbar column current Iin by factor of about two. This MOS-resistor creates a small resistance RM6 which is given by RM6=1 µn·Cox ·(VGS −VT)·LM6 WM6 (1) where µnis the charge-carrier effective mobility, Cox is the gate oxide capacitance, VGS is the gate-to-source voltage, VTis the threshold voltage, LM6and WM6are the length and width of the MOS-resistor. From Fig. 3, and knowing transistors M1and M4will be biased in strong inversion because of the large input current Iin ∆Vd=vd2−vd1=I1RM6(2) Ii=µn·Cox ·W1,4 2L1,4 (vdi −Vb−VT)2(3) which results in I1 I2 = (1 −∆Vd vd2−Vb−VT )2= (1 −qµn·Cox·W1,4 2L1,4I1RM6 √I2 )2(4) I1and I2are the input currents shown in fig.3. MOSFETs M1and M4have same size. If we can assume that RM6 √I2/(qµn·Cox·W1,4 2L1,4I1), then I1≃I2≃Iin 2with RM61 qµn·Cox·W1,4 2L1,4 r2 Iin (5) From eq. (2) we obtain ∆Vd≃IinRM6 2(6) The differential pair M2,M3in Fig. 3 will be biased in weak inversion, because ibis intentionally made very small. Consequently, Iouti =2n·µn·Cox ·U2 T·W2,3 L2,3·e vdi−vc nUT(7) ib=Iout1+Iout2(8) where nis the subthreshold slope factor and UTis thermal voltage. From here Iout2 Iout1 =e ∆Vd nUT=ex(9) where, from eq. (6) x= (IinRM6)/(2nUT). Straight forward calculations yield Iout2−Iout1=ib ex−1 ex+ 1 (10) If we can assume that x1(which is equivalent to RM6 2nUT/Iin), then Iout =Iout2−Iout1≃ib x 2=ibRM6 4nUT Iin (11) Therefore, the MCN circuit output current is, under some assumptions for RM6, proportional to Iin, and the proportionality factor can be controlled by bias current ib. This will allow us to scale down the input current by four orders of magnitude. Fig. 4. Minimum to maximum crossbar column inference current Vs Input current to neuron. Fig. 5. Comparison of average and standard deviation of output current for different attenuators: MCN circuit, MOS-ladder circuit and WTA circuit considering process and mismatch variations with 100 Monte Carlo runs. Transistor M11 is an optional switch, useful to isolate the post-synaptic neuron for test purposes. Differential MOSFET groups M2,21,22,23 and M3,31,32,33 are splitted along the length dimension in order to keep them in square shape and exploit inter-digitated layout to minimize mismatch due to gradients. IV. SIMULATION RESULTS OF THE 1T1R CROSSBAR This Section shows various simulation results of the proposed approach in comparison with other approaches (MOS- ladder and WTA type circuit) considering mismatch-and- process variations, temperature and input referred noise. The MCN circuit is implemented using a 130nm CMOS technology and its sizing and biasing is shown in Table I. Fig. 4 shows how the output current of the MCN circuit depends on the input current Iin. For the 1T1R memristive devices TABLE I SIZING AND BIASING OF PROPOSED CIRCUIT,DESIGNED FOR 130NM CMOS Parameter Value Supply voltage V DD =4.8 V Size of M1,4(W L)1,4= ( 10µm 1µm) Size of M2,3(W L)2,3= ( 1µm 1µm) Size of M21,22,23 (W L)21,22,23 = ( 1µm 1µm) Size of M31,32,33 (W L)31,32,33 = ( 1µm 1µm) Size of M5,7,8(W L)5,7,8= ( 1µm 1µm) Size of M6(W L)6= ( 8µm 1µm) Size of M9,10,11 (W L)9,10,11 = ( 1µm 1µm) Vb0.5 V ib25 nA we use here and with a read voltage of 0.3V, typical LRS is 13.9 kΩwhose inference read current is 21.5 µA, whereas typical HRS is 1MΩwhose inference read current results in 333nA. Minimum column current is when all the OxRAMs in a column are in HRS and maximum column current is when all the OxRAMs in a column are in LRS. Fig. 4 shows that the MCN output current (input current to neuron) is linear when the inference current stays below about 300µA. An optimal design procedure is carried out on the MCN circuit to keep mismatch as low as possible. Fig. 5 shows the statistical output current results (both mean-µand sigma-σ) for 100 Monte Carlo runs of the MCN circuit, a MOS-ladder and a WTA type circuit considering both process and mismatch variations. Fig. 6 shows how different column inference currents (minimum to maximum) vary due to temperature. We can see that temperature variations have fairly small impact on the output currents (less than 10%). The input-referred noise for the frequency range 0.01 Hz to 100 MHz is 3.178 nA/√Hz for the MCN circuit, 2.06 nA/√Hz for the MOS-ladder and 1.02 nA/√Hz for the WTA type circuit. Comparison of both area and input referred noise for different attenuators is shown in Fig. 7. V. CONCLUSIONS A new current attenuation circuit for OxRAM based crossbar applications has been proposed. The proposed technique allows to attenuate inference currents by several orders of magnitude before sending them to the post-synaptic neurons. Simulation results are provided based on a 130nm CMOS technology, with a scale-down factor of about 104. VI. ACKNOWLEDGEMENTS This work has been supported in part by the EU H2020 grants 687299 NeuRAM3, 824164 HERMES, by the Spanish Ministry of Economy and Competitiveness (with support from Fig. 6. Temperature variations of output current of MCN circuit, MOS-ladder and WTA circuit for different values of output currents. Fig. 7. Area and input referred noise of MCN circuit, MOS-ladder and WTA circuit the European RDF) under contracts TEC2015-63884-C2-1-P (COGNET) and by Lloyds Register Foundation’s ICON. REFERENCES [1] D. B. Strukov, Gregory S. Snider, Duncan R. Stewart and R. Stanley William, "The missing memristor found", Nature, 453 (7191), pp. 80-83, 2008. [2] L.O. Chua, "Memristor- The missing circuit element", Transactions on Circuits theory IEEE, vol. CT-18, no. 5, pp. 507-509, Sept. 1971. [3] L. O. Chua and S. Kang, "Memristive devices and systems", Proceedings of the IEEE, vol. 64, no. 2, pp. 209-223, 1976. [4] J. J. Yang, M. D. Pickett, X. Li, D. A. A. Ohlberg, D. R. Stewart and R. S. Williams,"Memristive switching mechanism for metal/oxide/metal nanodevices", Nat. Nanotechnology.,3, 429, 2008. [5] R. Waser, R. Dittmann, G. Staikov and K. Szot,"Redox-based resistive switching memories-nanoionic mechanism, prospects, and challenges", Adv. Mater., 21 (25-26), pp. 2632-2663, July 2009. [6] S. Yu, X. Guan and H. P. Wong,"Conduction mechanism of TiN/HfOx/Pt resistive switching memory: a trap-assisted-tunneling model", Appl. Phys. Lett..,99, 063507-063507, 2011. [7] S. D. Ha and S. Ramanathan,"Adaptive oxide electronics: a review", J.Appl.Physics., 110, 071101, 2011. [8] J. Frascaroli, S. Brivio, E. Covi, and S. Spiga, "Evidence of soft bound behaviour in analogue memristive devices for neuromorphic computing," Scientific Report, vol. 8, Page 7178, 2018. [9] X. Guo, F. Merrikh-Bayat, L. Gao, B.D. Hoskins, F. Alibart, B. Linares- Barranco, L. Theogarajan, C. Teuscher, and D.B. Strukov, "Modeling and Experimental Demonstration of a Hopfield Network Analog-to- Digital Converter with Hybrid CMOS/Memristor Circuits," Frontiers in Neuromorphic Engineering. Front. Neurosci. 9:488, 2015. [10] S. Stathopoulos, A. Khiat, A. Serb, and T. Prodromakis, "Benchmarking Analogue Performance of Emerging Random Access Memory Technologies," Proc. of the 2018 IEEE Int. Symp. on Circ. and Syst. (ISCAS2018), Florence, Italy, 2018. [11] D. Garbin, E. Vianello, O. Bicher , Q. Rafhay, C. Gamrat, G. Ghibaudo, B. DeSalvo and L. Perniola,"HfO2- Based OxRAM Devices as Synapses for Convolutional Neural Networks", IEEE Transactions on Electron Devices.,vol. 62, no.8, pp. 2494-2501, August 2015. [12] Y. Cassuto, S. Kvatinsky and E. Yaakobi,"Sneak-path constraints in memristor crossbar arrays", Information Theory Proceedings (ISIT), 2013 IEEE International Symposium on.,10.1109/ISIT.2013.6620207, October 2013. [13] C. Zamarre˜no-Ramos, L. A. Camu˜nas-Mesa, J. A. P´erez-Carraso, T. Masquelier, T. Serrano-Gotarredona and B. Linares-Barranco,"On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex", Front Neurosci., 17 March 2011. [14] B. Gilbert,"Current-Mode circuits from a Translinear Viewpoint: A Tutorial", In: Analogue IC design: the current-mode approach, Ed. by C. Tomazou, F.J.Lidgey and D.G. Haigh. Stevenage, Herts., UK: Peregrinus, 1990. Chap. 2, pp. 11-91. [15] K. Bult and J.G.M. Geelen, "An inherently linear and compact MOST- only current division technique," IEEE J. Solid-State Circuits, vol. 27, pp. 1730-1735, Dec. 1992. [16] J. Lazzaro, S. Ryckebusch, M. A. Mahowald and C. A. Mead,"Winnertake-all networks of O(N) complexity", in Advances in Neural Information Processing Systems 1, Morgan Kaufmann Publishers, San Francisco, CA, 1989. [17] Manu V. Nair, Lorenz K. Muller, and Giacomo Indiveri,"A differential memristive synapse circuit for on-line learning in neuromorphic computing systems", Nano Futuresv, vol. 1, no. 3, 2017. [18] C. Mohan, L. Camu˜nas-Mesa, E. Vianello, L. Periniolla, C. Reita, J. M. de la Rosa, T. Serrano-Gotarredona and B. Linares-Barranco, "Calibration of offset via bulk for low-power HfO2based 1T1R memristive crossbar read-out system", Microelectronic Engineering, Elsevier, vol 198, pp 35-47, 15 October 2018. [19] D. Garbin et al., "Variability-tolerant convolutional neural network for pattern recognition applications based on OxRAM synapses", Proc. IEEE Int. Electron Devices Meeting (IEDM), pp 28.4.1-28.4.4, December 2014.