Experimental study of artificial neural networks using a digital memristor simulator

T Experimental Study of Artificial Neural Networks Using a Digital Memristor Simulator Vasileios Ntinas, Ioannis Vourkas, Member, IEEE, Angel Abusleme, Member, IEEE, Georgios Ch. Sirakoulis, Member, IEEE, and Antonio Rubio, Senior Member, IEEE Abstract—This paper presents a fully digital implementation of a memristor hardware simulator, as the core of an emulator, based on a behavioral model of voltage-controlled threshold-type bipolar memristors. Compared to other analog solutions, the proposed digital design is compact, easily reconfigurable, demonstrates very good matching with the mathematical model on which it is based, and complies with all the required features for memristor emulators. We validated its functionality using Altera Quartus II and ModelSim tools targeting low-cost yet powerful field programmable gate array (FPGA) families. We tested its suitability for complex memristive circuits as well as its synapse functioning in artificial neural networks (ANNs), implementing examples of associative memory and unsupervised learning of spatio-temporal correlations in parallel input streams using a simplified STDP. We provide the full circuit schematics of all our digital circuit designs and comment on the required hardware resources and their scaling trends, thus presenting a design framework for applications based on our hardware simulator. Index Terms—associative memory, computing, emulator, memristor, neural network, neuromorphic, resistive switching I. INTRODUCTION HE existence of the 4th fundamental circuit element was postulated by Chua in 1971 and was termed “memristor” (short for “memory resistor”) [1]. Today, the term “memristor” usually refers to any resistance-switching (RS) device that complies with a particular set of requirements known as memristor “fingerprints” [2], regardless of the fabrication details [3]. The hysteretic RS properties of oxides sandwiched between metal electrodes was well known though even from the 60s, as seen in relevant publications by Hickmott and Argall [4], [5]. Chua’s theory of the memristor was however connected with experimental devices only in 2008 by Hewlett-Packard Laboratories and their work on TiO2 RS devices [6]. Memristors are now considered a rapidly emerging technology [7] that creates opportunities to realize innovative circuits and systems with applications such as nonvolatile memory [8], [9], adaptive circuits [10], [11], signal processing [12], and logic/computing [13-15]. The memristor has also been proposed as the electronic analog of biological synapses [16], [17]. It is essentially a resistor with memory; it is nonvolatile (although volatile devices have been also reported [18] and the specific theory can be found in [19]), its response depends on its whole dynamical history, and it demonstrates a continuous set of resistance values, making it ideal for tuning synaptic weights of artificial neural networks (ANNs) [20-22]. An ANN is a data-processing model based on the biological nervous system, implemented as a parallel and distributed network of simple nonlinear processing units [23]. Hardware (HW) implementation of ANNs is an important step toward obtaining human-brain-like functionalities at circuit level. The basic components of ANNs are neurons and synapses, whose circuit realization should mainly be compact to make scaling up to approach the total biological device numbers (~1011 neurons and ~1015 synapses in human brain) feasible. As typically happens with all new electronic devices, modeling and simulation are the first steps to exploring memristors’ general attributes, verifying theoretical aspects, and understanding the effect of different model parameter values. In this context, several either behavioral or physicsbased (usually SPICE-compatible) models have been developed [24-27]. Lab experiments with fabricated memristors are the next step. However, memristor technology is still in progress and device fabrication implies considerable costs and difficulties. Furthermore, the memristors commercially available to date are quite expensive and still not very reliable [28]. Consequently, research and development have largely focused on various (mostly analog) HW emulators [29-31] that facilitate the experimental exploration of memristive behavior. According to [30], the required features for a memristor emulator are: i) a wide memristance range; ii) nonvolatility; iii) initial state configurability; iv) floating operation; v) operability for high-frequency and continuous input signals; and vi) support interconnection with other components. Taking these six electrical requirements into account, in this paper we build upon our previous work in [32] and develop a fully digital memristor hardware simulator based on a behavioral model of voltage-controlled threshold-type bipolar memristors [27]. The presented electronic module constitutes the core of a digital memristor emulator, which further requires interface circuitry to permit connection to external circuits as a two terminal element, such as in [27], [31]. We conducted all the required verification tests and validated its functionality using Altera Quartus II and ModelSim software (SW), targeting low-cost yet powerful field programmable gate array (FPGA) families. The FPGAs are reconfigurable electronic platforms well-suited to implement ANNs [33], [34] owing to their HW flexibility, which allows rapid prototyping of different ANN topologies and implementation strategies. In this context, we chose the FPGA as the target electronic platform and showed that our design is suitable for FPGAbased ANNs. Our motivation was to design and implement digital HW electronic synapses particularly based on memristive dynamics and prove their suitability and applicability to a variety of ANN-based applications. Compared to other emulation approaches from the recent literature, this digital design is compact, easily reconfigurable, demonstrates excellent matching with the memristor model on which it is based [27], and complies with all the aforementioned electrical requirements (i to vi). Moreover, we tested its suitability for anti-serial memristive interconnections [35] and proved its synapse functioning in single-layer perceptron, implementing examples of associative memory and a simplified variation of spike timing dependent plasticity (STDP) [36] unsupervised learning of spatio-temporal correlations in parallel input streams, following previous demonstrations in [31] and [37], respectively. We present the schematics of our digital circuit designs and comment on the required HW resource scaling, thereby providing a complete design framework for such memristor emulator-based ANNs. II. MEMRISTOR DIGITAL HARDWARE SIMULATOR A. The Behavioral Memristor Model Even though any mathematical model [25] could serve as a basis of our digital implementation, the developed digital simulator is based on (and meets all the characteristics of) the behavioral threshold-type bipolar memristor model proposed in [27], described by the following equations: Fig. 1. Qualitative graph for the memristance change rate in (2) as a function of the applied voltage. Fig. 2. Compact memristor hardware simulator block diagram. state-dependent Ohm’s Law, where i(t) is the flowing current and v(t) is the voltage drop on the memristor, whereas R is the memristance and, at the same time, the system’s only state variable. Equation (2) depends only on v(t) and R. As shown in Fig. 1, its value changes at different rates when the applied voltage is either higher or lower than the threshold voltage vT, limited by upper and lower boundaries, namely RON and ROFF (i.e., RMIN and RMAX). The latter is accomplished with the use of the step function θ() in (2), which indicates that R can change only between its limiting values. The resistance change rate of threshold-type switching memristors is very fast above (and negligibly slow below) the threshold vT, which here, for the purposes of simplicity, is considered symmetric for both the SET (ROFF→RON) and RESET (RON→ROFF) transitions. The constants α and b in (2) define this change rate when |v(t)| < vT or |v(t)| > vT, respectively, with α, b < 0 and |α|<|b|. Thus, the resistance decreases when the memristor is forward-biased and increases when it is reverse-biased. Hereinafter we will refer to a memristor being forward/reverse-biased when the voltage at the top/bottom terminal is higher than that on its bottom/top terminal; the bottom terminal is denoted by the thick black line in the circuit schematic (see Fig. 2). i(t ) = R−1 ⋅ v(t ) R & = b ⋅ v + 1 (a − b)⋅ (| v + v | − | v − v |) (1) B. Circuit Implementation Fig. 2 shows the compact block diagram of the memristor module. The input signals include: the top and bottom 2 T T (2) electrode voltage (VTE and V BE), these being the basic two ⋅θ (R − RON )⋅θ (ROFF − R). It is a behavioral model of a voltage-controlled timeinvariant memristor whose memristance change rate is given by the piece-wise linear equation (2). Equation (1) reflects the inputs of the block, the initial memristance value (RINIT), which is loaded when reset = ‘1’, and two 2-bit flag signals to denote whether its terminals are properly connected, i.e., whether the applied voltage is valid or has been left floating (the need for two bits instead of one is explained in Section Fig. 3. Detailed memristor hardware simulator block diagram. TABLE I FPGA IMPLEMENTATION LOGIC DATA Family / Device Cyclone II / EP2C70F672C6 Total Logic Elements 3266 (5%) Total Registers 32 (<1%) IV). Unlike in [32], in this version of the simulator design, the output signal in most cases concerns the memductance G = R-1 because using G instead of R enables a simplified circuit design, lower HW resource requirements, and greater computational precision, as shown in the following sections. The model-specific parameters α, b, vT, RMIN, and RMAX are defined as internal constants (stored in memory) since we assume they are device/module-specific. Preferring power-of2 values for such internal constants and for all the auxiliary variables used in multiplications/divisions significantly minimizes the required HW resources. Fig. 3 shows a more detailed implementation block diagram. The input voltage is used to compute the derivative of R as in (2) and multiply it by a properly selected integration time step Δt, which also defines the maximum supported input signal frequency. The result is added to the current R value and, once the out-of-bound and floating terminal controls have been performed (the number (2)10 = (10)2 shown at the bottom left of Fig. 3 corresponds to the case of a floating connection), is stored in the corresponding register. The result of such controls are selection bits in multiplexers, i.e. blocks that conditionally pass always one of the inputs to the output according to the selection bit. For data representation, we use up to 32-bit integers and thus guarantee a wide value range and adequate precision for all important parameters (e.g., a wide memristance range), while preventing under/overflow during computations through the appropriate selection of α, b, and Δt. In integer computations, the fractional part of any result is truncated. Therefore, we represent resistance values in mΩ instead of Ω (i.e., 5000 instead of 5), and voltages in µV instead of V, to create the necessary precision. Moreover, the use of an auxiliary variable (the max integer) is shown in the last block in Fig. 3. In fact, this is a necessary transformation to obtain a valid G value since inverting R in digital HW would simply give zero as a result. Therefore, in order to keep this information from the inversion, we shift the result via a Fig. 4. Comparison between the mathematical model (Matlab) and the hardware simulator response (VHDL). α = -2000 Ω/(V×s), b = -190000 Ω/(V×s), Δt = 0.0005s, vT = 1V, RMIN = 100Ω and RMAX = 10KΩ. multiplication with a very large number defined as power of two (i.e. if R = 104Ω, then it is 107mΩ and we compute G = (231-1)/107) = 214S). The result is of course not the correct G as the latter includes the shifting operation. However, this is not really a problem and the practical meaning of this transformation is further explained in Section IV. Generally the use of such auxiliary variables is omitted in the block diagrams for the sake of simplicity. Basic information about the HW resources of the memristor module for a specific FPGA, is given in Table I (% refers to the percentage of the total available resources in the FPGA that are being used). III. BEHAVIORAL VERIFICATION TESTS This section presents the series of functional verification tests carried out to prove the HW simulator’s proper functioning and matching with the mathematical model, its compliance with the characteristic fingerprints, the multi-level tuning property required for analog applications, and its suitability for complex interconnections. All the measurements (a) Fig. 5. (a) i-v and R-v plot concerning the response of the HW simulator for Fig. 6. Multi-state programming feature. Comparison between the mathematical base model (Matlab) and the HW simulator response (VHDL). α = -2000 Ω/(V×s), b = -190000 Ω/(V×s), Δt = 0.0001s, vT = 1V, RMIN = 100Ω various frequencies of the input voltage v(t) = 3sin(2πft) when R INIT = RMAX. and RMAX=10KΩ, whereas the applied voltage was v(t) = ±3sin2(100πt). (b) i-v plot for varying input amplitudes Vo of v(t) when f = 12 Hz and RINIT = RMAX. (c) i-v plot for varying initial conditions (RINIT) when Vo = 3V and f = 12 Hz. In all scenarios we used α = -2000 Ω/(V×s), b = -190000 Ω/(V×s), Δt = 0.0001s, vT = 1V, RMIN = 100Ω and RMAX = 10KΩ. for our design use the ModelSim HW simulation data for a target FPGA device and 100MHz clock frequency. These data were collected via proper Matlab scripts and compared with the reference model data. The ModelSim input files were prepared using the Matlab HDL coder. In all demonstrated measurements, the reset phase when the system was initialized was simply omitted. A. Match with the Mathematical Memristor Model It is important to guarantee an exact match between the HW simulator’s response and data from the memristor model on which it is based, regardless of the characteristics of the input voltage. Fig. 4 shows a relevant comparison concerning a triangular input voltage pulse. The HW simulator’s response matches very well that of the model. The available precision during computations with integer variables in our design guarantees infinitesimal error. Such precise matching is obtained for different input pulse types and frequencies, provided that the selected model parameter values do not cause under/overflow problems. B. Compliance with the Characteristic Fingerprints We proved that our HW simulator indeed behaves as a memristor by testing its compliance with memristor fingerprints [3]. Fig. 5(a) shows a set of current-voltage curves, along with the memristance-voltage curves, for different frequencies of the input sinusoidal voltage. All curves in the i-v plane are pinched at the origin (i, v) = (0, 0), i.e., there is no phase shift between the i(t) and v(t) (c) (b) waveforms. This is valid for all amplitudes and frequencies of the input signal (see Fig. 5(b)), and for any possible initial condition of memristor (see Fig. 5(c)), as seen in the R-v and iv planes. Moreover, the so-called single-valued function limiting phenomenon is confirmed with the collapsing hysteresis loops in the i-v graph of Fig. 5(a). As the sweep frequency f increases, the area of each lobe of the pinched hysteresis loop shrinks, such that the memristance function degenerates to a straight line (tends to a single value) as f increases towards infinity. C. Multi-State Tuning Capability One of the main reasons why memristors have been proposed as the electronic analog of biological synapses is because they demonstrate a continuous set of resistance values and are thus ideal for representing synaptic weights. Such multi-level tuning capability is crucial for analog applications. Therefore, we tested the multi-state tuning capability of our design, which is required to model synapse functioning. The relevant HW simulation results are shown in Fig. 6. Multiple continuous states are obtained via successive short voltage pulses of the same polarity. In our case, we show seven distinct memristance levels achieved with a ±Vosin2(100πt) pulse train (Vo = 3V). Since higher applied voltages cause faster switching, decreasing the pulse amplitude while ensuring it remains above the threshold results in much closer distinct memristance levels. Shortening the pulse duration while maintaining the same amplitude has a similar effect. As shown in Fig. 6, in all such cases the module’s multi-level switching response matches very well the reference model. (a) (b) Fig. 7. Anti-serial connected memristors. (a) Block diagram showing the interconnection of the electronic modules and the additional components for the computation of the voltage divider equation. The inset shows the equivalent circuit using the memristor symbol. (b) Simulation results showing the applied voltage VIN and the voltage drop on the two memristors, the composite i-v plot, and the HW simulators’ memristance evolution with time and with the input voltage, respectively. α = -2000 Ω/(V×s), b = -190000 Ω/(V×s), Δt = 0.0005s, vT = 1V, RMIN = 100Ω and RMAX = 10KΩ. D. Complex Device Interconnection In keeping with the electrical requirements for memristor emulators mentioned in the Introduction, this verification test checked the possibility of interconnection with other Fig. 8. Generic implementation scheme for a single-layer ANN with n input neurons Ni (i = 1,…, n) connected to an output neuron No via n memristors. Go is a small conductance used in the Kirchhoff’s Current Law computation. components, i.e., whether the developed HW simulator can be connected to other devices or emulators, which is essential for it to be used in more sophisticated circuit configurations. It is worth mentioning that unless an interface circuit is added, such as for example an ADC and a digital potentiometer, similar to that shown by Pershin and Di Ventra in [27] and [31], then our implementation cannot be electrically connected to an external circuit as a complete digital memristor emulator. To this end, we studied the suitability of the simulator for the complementary resistive switch (CRS) configuration [35]. A CRS consists of two memristors connected in series but with opposite polarities (anti-serially), hereinafter called the forward-polarized memristor (FPM) and the reverse-polarized memristor (RPM). The CRS is a comprehensive enough test, also easy to implement, which allows to confirm both the interconnection property and the polarity-dependent switching of multiple such modules combined together. Memristors with opposite polarities demonstrate reverse behavior to the applied signal; i.e., during one period of the AC input voltage, complementary devices reciprocally change their states. Fig. 7(a) shows the block diagram of the CRS configuration. For simplicity, we have defined the output of the simulators as the memristance R instead of the memductance G shown in Fig. 3. Apart from the two memristor modules, the system requires a combinational part to calculate the voltage VC on the common intermediate node of the memristors, i.e., to compute the voltage divider equation. The latter receives VIN as input and uses the current state of the two memristors (R1 and R2) to drive them with the corresponding VTE value. The sum of R1 and R2 is computed first and then the fraction, which is multiplied by the input voltage, as shown in the inset. A MUX is used to prevent invalid results by division with zero. The flag inputs “01” denote there are no floating electrodes. The two modules are set to the FPM/RPM = ROFF/RON state during initialization. The inset shows the equivalent circuit schematic for such connection, using the memristor symbol for clarity. Fig. 7(b) shows the HW simulation results. The positive (a) (c) (b) Fig. 9. (a) Compact and (b) detailed block diagram of the neuron module based on the model described in [31]; (c) HW simulation results for the neuron’s behavior for different amplitudes of the input voltage VIN. For the purposes of clarity, the inset in the output plot focuses on a specific excitation cycle. part of the triangular input VIN creates the necessary conditions first to change the state of the FPM (R2) from ROFF to RON and, later, that of the RPM (R1) from RON to ROFF, resulting in a flipped resistive configuration. The memristors then exhibit an ohmic behavior until the applied voltage exceeds the respective negative thresholds and forces them to successively switch to their initial states. In Fig. 7(b) we also show the perfectly symmetric composite i-v curve. Overall, the results confirm the reproduction of the CRS operation. IV. MEMRISTIVE ARTIFICIAL NEURAL NETWORKS A. Circuit Implementation In this section we use the developed hardware simulator as a synapse and present an implementation scheme for the perceptron topology shown in Fig. 8. All the additional electronic modules, like the previously presented for memristor, concern digital designs easily tuned and generically built to facilitate the development of singleor multi-layer ANNs on FPGA devices for several applications. Specifically in the single-layer ANN example shown in Fig. 8, the input neurons Ni (i = 1, …, n) are connected with an output neuron No via synapses (memristors), while the output signal OUT is determined by the applied input signals INi and the strength of the synaptic connections Gi, which weigh the potentials Vi. Biological neurons generally have receptor (synaptic) and action potentials. When the receptor potential at the input of an idle (not firing) neuron exceeds a given threshold, the neuron is excited and starts firing, i.e. starts emitting forward and backward fixed-amplitude action pulses. As we will show in the following ANN examples, the backpropagating pulses are responsible for synaptic tuning and, therefore, for continuous re-learning. For the purposes of our experiments, we developed a digital electronic version of a neuron model as described next and in more detail in [31]. Figs. 9(a) and 9(b) show the compact and detailed block diagrams of the developed neuron module, which receives two inputs. VIN is the receptor potential, which is constantly monitored; once it exceeds a threshold value VT (defined as an internal constant, i.e., stored in memory), forward and backward fixed-amplitude action pulses are generated. However, the computed pulse separation (the waiting/idle time) varies according to the strength of the receptor input stimulus and a random parameter, which is the module’s (a) (b) Fig. 10. (a) Compact and (b) detailed block diagram of the Kirchhoff’s Current Law (KCL) computation (weighted summation) module giving the voltage at node Vo of Fig. 8. We set Go = (231-1)/Ro, and assumed a large resistance Ro ≈ 1MΩ. second input. The output VOUT of the neuron is only indicating whether the neuron is excited or idle. There are three possible cases for the memristor terminals to be considered since, according to [31], when a neuron is idle (not firing), its output terminal (e.g., Vi for input neuron Ni in Fig. 8) becomes floating. However, while still in idle state, its input terminal (i.e. the common node of all the synapses, Vo for output neuron No in Fig. 8) may still be connected and receiving input stimuli caused by the pre-synaptic pulses in case one of the input neurons is firing. Therefore, defining Vo simply floating due to the neuron being idle, is incorrect. We overcame this requirement via local bit-processing in-between the VOUT of a neuron module and the Valid_VTE/BE flag of a memristor module, thereby driving two bits to the flag inputs, enough to model the three possible cases (connected, floating, and connected while neuron is idle, as explained in Table II). This enabled us to keep the implementation of neurons general and their complexity low. Moreover, the fixed amplitude of the action pulses is not defined inside the neurons but rather is set externally and thus applied directly to the corresponding terminal of every memristor, as shown below. Unlike in our previous work [32], here we included a second neuron output VSIGN indicating whether the back-propagating pulses have a negative sign. As we will show below in our ANN examples, unlike in [31], [32], this important added property makes it possible both to increase and decrease the synaptic weights through the simplified STDP scheme implemented here, previously proposed in [38] (although action potentials resembling more the true spike waveforms found in biological neural systems, as presented in [36], [39], [40], could be implemented as well by more HW resources). When both input and output neurons are firing, the resulting voltage drop (e.g., Vi - Vo in Fig. 8) on the memristors is of constant amplitude but different durations, depending on the timing of the voltage signals at the two sides of the synapses. On the other hand, when the output TABLE II POSSIBLE SITUATIONS FOR OUTPUT NEURON’S TERMINALS Neuron’s state Input terminal (Vo) Output terminal (OUT) Idle Connected (VOUT of KCL module) Floating Excited Connected (±VPULSE) Connected (VPULSE) neuron is not firing, then Vo varies depending on the state of input neurons and their synapses. More specifically, according to Fig. 9(b), the neuron operation is determined based on two counters defining the total excitation time steps and the pulse separation time, mentioned before. We arbitrarily set the excitation time to 100 clock steps. This duration is defined by the step counter (bottom left in the figure), which starts counting when the neuron becomes excited. Inside the neuron, the number of steps when the VSIGN is ‘1’ is also defined as VSIGN_TIME which is assumed to be 20 steps in Fig. 9(b), a value chosen based on trial and error to improve the results obtained in the application examples shown next; for the rest of the excitation cycle steps, VSIGN is ‘0’. The hold register is responsible for keeping the neuron excited (i.e. neuron’s output VOUT = ‘1’) for 100 steps. Moreover, when the step counter starts, the dt register stores the idle/waiting time, which is computed according to a formula proposed in [31] and converted to true simulation time via multiplication with the integration time step Δt. When the step counter reaches 100, the hold register becomes ‘0’ but the dt enabled becomes ‘1’, thereby activating the dt counter, which is responsible for keeping the neuron idle for a waiting time (refractory period) equal to dt. When dt counter = dt, then the dt enabled becomes ‘0’. A positive difference VIN-VT can activate the neuron provided that the dt enabled register is ‘0’, i.e., that the neuron is not in refractory period. Adjusting the VSIGN_TIME during module instantiation makes it possible to create the desired time ratio Fig. 11. Block-level circuit topology implementing the linear perceptron with three neurons and two synapses, according to [31], using the neuron, the memristor, and the weighted summation (KCL) electronic modules. The small gear driving the memristor Valid_VTE inputs denotes local bit-processing mentioned in text. of negative and positive back-propagating action pulses during excitation, as proposed in [38]. HW simulation results of the neuron’s behavior are shown in Fig. 9(c). We present the action potentials caused by different amplitudes of the input VIN, whereas for the rand input we use a series of randomly generated integers ∈[0, 10]. When VIN<VT there is no firing. Otherwise, the average pulse separation decreases as the input amplitude increases. The effect of the rand input on the dt computation is more evident at higher firing rates. Fig. 9(c) also shows the VSIGN output, which, in this simulation scenario, is ‘1’ for the first 20 of the 100 excitation time steps. In short, looking back at Fig. 8, we can conclude that when the input neurons are firing, the memristors receive positive action potentials Vi at their TE, whereas when they are idle, these terminals are assumed to be floating. On the other hand, when the output neuron is firing, the memristors receive either positive or negative action potentials Vo at their BE, but when the neuron is idle, the receptor potential Vo at its input terminal needs to be calculated (see Table II). We saw a similar problem before in the CRS example. Whenever there is a shared node among many interconnected memristors, an additional module responsible for computing the instant potential at that node is required. Therefore, for the purposes of our experiments, we developed an electronic version of the Kirchhoff’s Current Law (KCL) computation (weighted summation) shown in Fig. 10. As can be seen in Fig. 10(a), this module receives the current memductance of every memristor Gi (which includes the shifted operation as explained previously) and a series of bits Vi, which are the outputs of the input neurons (VOUT). The output (i.e., the Vo potential in Fig. 8) is updated using the KCL equation when the input VBACK is ‘0’, i.e., when the output neuron is not excited and there is thus no back-propagating pulses. Otherwise, i.e. when the output neuron is firing, the output becomes equal to the predefined amplitude ±VPULSE depending on the VSIGN input. VPULSE is the fixed amplitude assumed for both the forward and backward action pulses in our ANNs. As shown in Fig. 10(b), we built this module in a generic manner in order to receive an arbitrary number of inputs, provided that the internal KCL computations do not cause overflow. According to the KCL formula Vo = (V1G1 + V2G2 + … + VnGn) / (Go + G1 + G2 + … + Gn), first we sequentially compute the conditional sum of all Gi (depending on the corresponding Vi which serves as MUX selection bit) and Go. Then we compute the fraction and eventually multiply it by the fixed amplitude VPULSE. Since this formula has G both in the numerator and denominator, the previous shifting transformation is inherently removed and does not affect the result of this computation. The reason we prefer G instead of R is to simplify the implementation of the KCL block as several more divisions are required if R is used in this formula, while we also noted even a better precision. In the next section, we use all these modules and present two ANN example configurations for two different applications. B. ANN Example for Associative Memory Following [31], which demonstrates a neural network implementing the famous “Pavlov’s dog” experiment [41], here we present a similar ANN implementation as a proof of concept of associative memory. Fig. 11 presents the blocklevel circuit topology implementing the linear perceptron with two input neurons and one output neuron connected via two synapses. A few details about the Pavlov’s dog target experiment [31], [41]: Initially, a dog salivates only at the