Analog Sequential Hippocampal Memory Model for Trajectory Learning and Recalling: A Robustness Analysis Overview

Analog Sequential Hippocampal Memory Model for Trajectory Learning and Recalling: A Robustness Analysis Overview Daniel Casanueva-Morato,* Alvaro Ayuso-Martinez, Giacomo Indiveri, Juan P. Dominguez-Morales, and Gabriel Jimenez-Moreno 1. Introduction The expansion of information systems to all areas of society has generated the need for new computing systems that are more powerful, more efficient, and consume less power than current systems. [1–3] Different fields of engineering are exploring new alternatives to solve these problems. Among them, neuromorphic engineering stands out. This field focuses on proposing systems that are inspired by the functioning of the brain, imitating specific parts of it, both at the architecture level or at the functionality level. The brain is capable of solving complex problems efficiently in real time and with low energy consumption, neuromorphic engineering tries to incorporate these superior capabilities into current computing systems. [4–6] To mimic how the brain works, neuromorphic systems make use of spiking neural networks (SNN). These networks generate and transmit information by means of electrical pulses (spikes) between their different components, achieving asynchronous, and distributed operational capabilities. Since all information within SNNs relies on generating and transmitting spikes, no processing is D. Casanueva-Morato, A. Ayuso-Martinez, J. P. Dominguez-Morales, G. Jimenez-Moreno Robotics and Technology of Computers Lab. Universidad de Sevilla Sevilla, Spain E-mail: [email protected] D. Casanueva-Morato, A. Ayuso-Martinez, J. P. Dominguez-Morales, G. Jimenez-Moreno Escuela Técnica Superior de Ingeniería Informática (ETSII) Universidad de Sevilla Sevilla 41012, Spain The ORCID identification number(s) for the author(s) of this article can be found under https://doi.org/10.1002/aisy.202400282. © 2024 The Author(s). Advanced Intelligent Systems published by Wiley- VCH GmbH. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. DOI: 10.1002/aisy.202400282 D. Casanueva-Morato, A. Ayuso-Martinez, J. P. Dominguez-Morales, G. Jimenez-Moreno Escuela Politécnica Superior (EPS) Universidad de Sevilla Sevilla 41011, Spain G. Indiveri Institute of Neuroinformatics University of Zurich and ETH Zurich Zurich 8057, Switzerland J. P. Dominguez-Morales, G. Jimenez-Moreno Smart Computer Systems Research and Engineering Lab (SCORE) Research Institute of Computer Engineering (I3US) Universidad de Sevilla Sevilla, Spain The rapid expansion of information systems in all areas of society demands more powerful, efficient, and low-energy consumption computing systems. Neuromorphic engineering has emerged as a solution that attempts to mimic the brain to incorporate its capabilities to solve complex problems in a computationally and energy-efficient way in real time. Within neuromorphic computing, building systems to efficiently store the information is still a challenge. Among all the brain regions, the hippocampus stands out as a short-term memory capable of learning and recalling large amounts of information quickly and efficiently. Herein, a spike-based bio-inspired hippocampus sequential memory model is proposed that makes use of the benefits of analog computing and spiking neural networks (SNNs): noise robustness, improved real-time operation, and energy efficiency. This model is applied to robotic navigation to learn and recall trajectories that lead to a goal position within a known grid environment. The model is implemented on the special-purpose SNNs mixed-signal DYNAP-SE hardware platform. Through extensive experimentation together with an extensive analysis of the model’s behavior in the presence of external noise sources, its correct functioning is demonstrated, proving the robustness and consistency of the proposed neuromorphic sequential memory system. RESEARCH ARTICLE www.advintellsyst.com Adv. Intell. Syst. 2024, 2400282 2400282 (1 of 16) © 2024 The Author(s). Advanced Intelligent Systems published by Wiley-VCH GmbH performed when no spikes are generated. Due to this, SNNs are characterized by lower energy consumption, [7] greater resistance to noise, [8–10] and higher real-time efficiency compared to traditional systems. [11,12] From a biological point of view, the brain is closer to an analog computer. [13] Analog computing, by exploiting the physical variables (continuous time evolution of voltages, currents, and conductance changes), allows the dynamics of neurons and synapses to be emulated. [4,14,15] When moving to physical implementations, analog neuromorphic hardware has better performance, energy efficiency, and scalability than its digital counterparts. [8,13,15,16] On the contrary, the main disadvantage of neural analog systems is the noisy nature of analog circuits themselves and mismatch. [8,13] However, the brain itself directly and efficiently processes signals that are noisy. [17] Therefore, the design of neuromorphic systems would benefit from the use of analog computation coupled with a noise-tolerant designs. [8] In any computing system, memory plays an important role. However, within neuromorphic engineering, memory systems are in early stages of development. Of all the regions of the brain involved in memory, the hippocampus is the most prominent. On the one hand, it is a short-term memory capable of storing large amounts of information from various cortical sources in the form of memories by associating the different components that make up this input information. Moreover, from a memory fragment, it can sequentially recall the complete memory. On the other hand, due to the exploitation of its characteristic as a self-associative sequential memory and the functional diversity of the neurons that compose it, it is capable of performing a certain amount of computation within the memory itself. Consequently, the hippocampus is involved in navigation, making it possible to map the environment and retrieve trajectories to goal positions. [18,19] The hippocampus achieves these characteristics due to its structure and the existence of special neurons called place cells (PC). On the one hand, the structure of the hippocampus is composed of three main brain regions (Figure 1a): dentate gyrus (DG), cornu ammonis 3 (CA3), and cornu ammonis 1 (CA1). The spiking input information reaches DG, which, due to its competitive network structure, increases the dimensionality, i.e., it carries out a dispersion of the input information. This information then reaches CA3, which consists of a recurrent collateral network (Figure 1b). Thanks to oscillations of the input activity, this region is responsible for learning and storing the information in the collateral connections themselves and, at the same time, recalling it completely from a fragment. Finally, the memory arrives at CA1, where the original format of the input information is recovered before leaving the hippocampus. [18,19] Furthermore, the main input and output pathway of the hippocampus is the entorhinal cortex (EC), which allows a part of the output information to return to the input, allowing the hippocampus to act as a sequential memory. On the other hand, PC neurons are activated when the subject is within a certain position in the environment and, therefore, enable the creation of a spike representation of the environment in the memory itself that can be recalled at any moment. [18,20] Bio-inspired neuromorphic models of the hippocampus and their application to memory systems or robotic navigation systems have been previously addressed in the literature. From the point of view of memory models, several works propose spike-based sequential memory models bio-inspired by the hippocampus. [21,22] Other articles propose spike-based bioinspired hippocampal memory models but not sequential models. [23–26] If the spectrum is broadened, different studies [27–31] present bio-inspired hippocampal memory models but not purely spike-based, or [32–35] spike-based but not bio-inspired memory models are proposed. From the point of view of the spiking systems involved in navigation, Oess et al. [36] proposed a spike-based navigation system where the memory system is static and bio-inspired by the hippocampus. Different studies, such as refs. [37–41], present models focusing on bio-inspired environment mapping based on the hippocampal PC neurons, and others, such as refs. [42–45], propose path-planning techniques. In the brain, the robustness of the system and its tolerance to noise are fundamental characteristics. Moreover, computational systems often have to deal with noise, either external noise that adds to the signal to be analyzed or internal noise produced by the technology itself (an intrinsic feature of analog computing). Despite this, in neuromorphic engineering, hippocampal Figure 1. a) Architecture of the biological model of the hippocampus and b) CA3. www.advancedsciencenews.com www.advintellsyst.com Adv. Intell. Syst. 2024, 2400282 2400282 (2 of 16) © 2024 The Author(s). Advanced Intelligent Systems published by Wiley-VCH GmbH 26404567, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/aisy.202400282 by Readcube (Labtiva Inc.), Wiley Online Library on [09/10/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License bio-inspired works dealing with memory systems and robotic navigation systems are scarce, have limitations, and assume ideal scenarios, i.e., they do not analyze the behavior of the proposed system under noisy conditions. In view of this, this article proposes an analog spike-based sequential memory system bioinspired by the hippocampus. This system is applied to robotic navigation to sequentially recall the trajectory to a goal position within a known environment. Finally, the noise tolerance of the proposed system is analyzed through detailed experimentation, in which the system is subjected to variable noise inputs. The main contributions of this work include the following: 1) A spike-based bio-inspired hippocampal analog sequential memory model capable of learning, recalling from a fragment, and forgetting sequences of memories. In addition, it can work with both orthogonal and nonorthogonal patterns. 2) Application of the model to robotic navigation. The system is able to learn the different trajectories from any initial position to a goal position within a known environment. In addition, the system is able to recall sequentially, from each initial position, the different positions of the trajectory toward the goal position. 3) A detailed analysis of the system was carried out, proving the high robustness and noise tolerance of the system in this type of application. 4) This is an implementation of a spike-based analog bio-inspired hippocampal sequential memory model on a special-purpose analog hardware platform for SNN systems. 5) The source code is publicly available, together with the documentation, including all the necessary details regarding the SNN architectures. The rest of the article is structured as follows: Section 2 presents the materials used in this work. In Section 3, the proposed model is detailed. The experiments performed to evaluate the functionality, performance, robustness, and noise tolerance of the proposed model are explained in Section 4, along with the results obtained. Then, in Section 5, the results of the experiments are discussed. Finally, the conclusions of the article are presented in Section 6. 2. Materials 2.1. SNNs Hardware implementations of neuromorphic systems typically belong to the third generation of neural networks, known as SNNs. [46] These networks consist of clusters of neurons that dynamically process incoming signals and generate action potentials (spikes) when the combined inputs surpass a certain threshold. They transmit their spikes to their target neurons instantaneously, via synapses. SNNs offer significant computational efficiency by transmitting spikes only when they occur, and they can be programmed to perform complex computations by utilizing sparse activity in both spatial and temporal dimensions. [47] Various computational models can be used to implement each component of the SNN, aiming to approximate the biological behavior observed in nature. Among these models, the Leaky Integrate-and-Fire (LIF) model has gained significant popularity. [48,49] In the LIF model, the neuron’s membrane potential is driven by the sum of input currents, which are generated by synapses stimulated by incoming spikes. If the total input current exceeds the neuron’s“leak”current, the membrane potential increases until it reaches a threshold. Conversely, if the total input current is below the leak current, the membrane potential returns to the neuron’s resting state. Once the threshold is reached, the neuron generates a brief pulse, known as a spike, and resets its membrane potential. Following the generation of a spike, the neuron remains in a reset state for a specific period of time, known as the refractory period, before it begins integrating its input current again. [50] The characteristics that define connections between synapses are direction, delay, and weight. The delay attribute represents the time it takes for a spike to travel from the presynaptic neuron to the postsynaptic neuron. The weight attribute indicates the magnitude of the change in the postsynaptic neuron’s membrane potential. Lastly, the direction attribute determines whether the change is positive (for excitatory synapses) or negative (for inhibitory synapses). The learning rules that govern the process of acquiring and storing information play a crucial role in neural networks. While there has been extensive research on spike-timing- dependent plasticity (STDP) learning rules in SNNs, [51–53] recent studies have discovered spike-based synaptic plasticity mechanisms that take into account additional factors, such as the neuron’s membrane potential or its recent firing activity, which have proven to be more effective. For a comprehensive overview of these rules, which are also compatible with neuromorphic hardware, please refer to ref. [54]. The STDP triplet rule, which is an expansion of the plain STDP rule, provides a more precise replication of experimental data obtained from actual synapses. [55] In contrast to the basic STDP rule, which only considers pairs of presynaptic and postsynaptic spikes to calculate changes in synaptic weight (increasing the weight if the postsynaptic spike follows the presynaptic one, and decreasing it otherwise), the STDP triplet rule takes into account additional scenarios. These scenarios include a presynaptic spike followed by a postsynaptic spike and another presynaptic spike, as well as a postsynaptic spike followed by a presynaptic spike and another postsynaptic spike. In the former scenario, the synaptic weight decreases, while in the latter scenario, it increases. This learning rule can accurately reproduce the precise spike-timing behavior of the basic STDP rule at low frequencies of input/output spikes. Furthermore, it can explain and replicate Hebbian-type rules based on rates or correlations in high-frequency regimes. [56] 2.2. The DYNAP-SE Chip While there is a wide range of dedicated digital neuromorphic processors that can implement SNNs, [57–59] the research on mixed-signal analog/digital implementations is still ongoing. Currently, only proof-of-concept prototypes of such implementations are available. In this study, we make use of one of these prototypes called the “Dynamic neuromorphic asynchronous processor - scalable”(DYNAP-SE). [60] DYNAP-SE is a hardware platform that utilizes a scalable multicore architecture with various memory structures to support dynamic asynchronous event-based processing. This platform combines analog circuits, which implement synaptic and neural www.advancedsciencenews.com www.advintellsyst.com Adv. Intell. Syst. 2024, 2400282 2400282 (3 of 16) © 2024 The Author(s). Advanced Intelligent Systems published by Wiley-VCH GmbH 26404567, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/aisy.202400282 by Readcube (Labtiva Inc.), Wiley Online Library on [09/10/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License dynamics, with digital circuits that handle network connectivity and spike routing among firing neurons. Each DYNAP-SE chip consists of four cores, with each core accommodating 256 neurons. These neurons have a fan-in of 64 synapses and a fan-out of 4000 synapses. The chips were manufactured using 0.18 μm 1P6M CMOS technology and incorporated hierarchical asynchronous routers, as well as distributed integrated asynchronous SRAM and CAM memory cells across the cores. This study utilized a board consisting of 4 DYNAP-SE chips. The circuits within each core, which include parameters such as neuron leakage and refractory period, have the same nominal value. However, due to device mismatch, the actual value of each circuit may differ. The DYNAP-SE circuits have a typical coefficient of variation of around 20%. [8] As a result, there can be a notable difference between the specified values and the actual values of these parameters. The neuron circuits in the DYNAP-SE implement a model equivalent to the Adaptive-Exponential Integrate and Fire (AdExp-I&F), [61,62] whose parameters can be configured to behave like LIF neurons. Synapses and biophysically realistic synapse dynamics are implemented using the current-mode differential pair integrator (DPI) log-domain filter, [63] which can be configured to give rise to 4 possible synapse types: AMPA (fast, excitatory), NMDA (slow, excitatory), GABA B (subtractive inhibitory), and GABA A (shunting inhibitory). 3. Sequential Analog Hippocampus Computational Memory Model 3.1. Architecture This work is based on and extends the model proposed in ref. [26]. The architecture of the spike-based hippocampal bioinspired analog model of a sequential memory is presented in Figure 2. The model works with real information encoded in trains of 5 to 20 spikes within time windows of 5 to 10 milliseconds (ms). Its architecture consists of 4 main neural structures: DG, CA3, CA1, and EC, where the novelty with respect to previous work lies in EC. DG performs the dispersion of the content of the input memory. This dispersion only applies to a fragment of the entire memory. The dispersed part of the memory will be referred to as the cue and the remaining part of the memory that remains unchanged is referred to as the content. Concretely, the maximum possible degree of sparsity is applied, i.e., one-hot encoding. To achieve this functionality, DG presents a multilayer cascaded filter structure that acts as a competitive Winner- Take-All network. [26] For each layer of the filter, the activity corresponding to the cue of the input memory will be propagated and the activity of a set of combinations of input neurons will be computed to determine which output neuron should be activated. If any of the output neurons of that layer is activated, the activity of this neuron will represent the cue of the sparse memory and will propagate through each of the remaining layers of the filter until it reaches the output. At the same time, this activity will inhibit the remaining output neurons in consecutive layers of the filter to prevent more than one output neuron from being activated at the same time. CA3 is responsible for the learning, storage, and subsequent recall of individual memories that reach the model. For this purpose, it has a recurrent collateral network structure that can be divided into 2 subpopulations of neurons: the population that receives cue-related activity (CA3cue) and the population that receives content-related activity (CA3cont). [26] CA3cue neurons have lateral all-to-all connections with CA3cont neurons, where the triplet STDP learning mechanism is located. The activation of neurons in CA3 as a consequence of the arrival of a complete memory will result in the learning of the cue-content pair and the storage of this information in the synapses themselves. At the same time, the activation of neurons in CA3 as a consequence of the arrival of a fragment of the memory, corresponding to the cue, will trigger the recall of the complete memory. CA1 is in charge of recovering the original format of the memory before it exits the model. To attain this, it has a structure similar to DG, but with a single layer. [26] All those DG neurons that were activated by input neurons iwill now participate in the activation of output neurons iof CA1. In this way, the activity of the dispersed cue returns to its original format while the activity of the content remains unchanged. Finally, there is EC. This is a chain of neuron populations that propagate the output activity of a part of the memory back to DG and where the delay of the propagation is regulated by the number of layers in the chain. Specifically, the activity of a subset of neurons belonging to the memory content in CA1 will be Figure 2. Architecture of the sequential analog hippocampus computational memory model proposed. The model divides its architecture into 4 blocks: DG, CA3, CA1, and EC. The word Triplet STDP marks those synapses that exhibit the triplet implementation of the STDP learning mechanism. www.advancedsciencenews.com www.advintellsyst.com Adv. Intell. Syst. 2024, 2400282 2400282 (4 of 16) © 2024 The Author(s). Advanced Intelligent Systems published by Wiley-VCH GmbH 26404567, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/aisy.202400282 by Readcube (Labtiva Inc.), Wiley Online Library on [09/10/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License propagated and connected to the neurons that will encode the cue at the DG input. This population endows the network with the ability to recall sequences of linked memories. In a generic way, for a memory of size Min a memory with a capacity of Nmemories and a delay factor (number of populations to reach a given delay from the base delay of the synapses) of D, the first log2ðNþ1Þ de neurons would correspond to the cue (cueSize) and the remaining McueSize would correspond to the content (contSize). Inside the content neurons, the activity of the last cueSize neurons can be distinguished as content identifying the next memory in the sequence and the first M2cueSize neurons as content with information belonging to the current memory. Taking these variables into account, the model would have a consumption of 3 Mþ2NþcueSize2 ðcueSize 2ÞþDcueSize neurons, 4 Mþ2NþDþ cueSize ðcueSize32cueSize2þ3cueSize 6þDÞstatic synapses, and NcontSize dynamic synapses with the triplet STDP learning mechanism. 3.2. Operating Principle The spatio-temporal coding of the model’s spiking activity is based on the concept that the activation of neurons from the same population at a particular time instant or window is associated with the same memory, while the activation of these neurons but at different time instants or windows is linked to different memories. The proposed model is capable of learning, recalling from a fragment, and forgetting individual memories, as well as sequential recall of memories. Learning starts with the input of a complete memory through DG. The part corresponding to the cue will be dispersed, while the content will pass through unchanged. This activity will reach the CA3cue and CA3cont subpopulations of CA3, respectively, triggering the activation of the triplet STDP learning mechanism and, thus, the learning and storage of the memory. For the triplet STDP learning mechanism to work with spike trains, it was configured in such a way that each pair and triplet of spikes trigger small synaptic variations. [26] Specifically, the input of 3 separate spike trains is necessary to ensure the correct learning of the complete memory which, after leaving CA3, will reach CA1, where it will recover the original format and leave the network. At the same time, this configuration prevents independent spikes produced by punctual noise from causing enough synaptic change to carry out a correct learning. Taking this into account, for each spike train, a temporal separation of 100 ms is required to prevent the learning mechanism from mixing the synaptic weight variations of one train with those of the next, with the learning operation having a duration of 300 ms. The recall operation for a memory starts with the input of neural activity corresponding to only the cue of the memory to be recalled. After being dispersed by DG, it will reach the CA3cue subpopulation in CA3, activating the corresponding neuron. This activation will propagate to CA3cont, activating the subset of content neurons that were associated with the memory by previous learning. In this way, from a fragment of the memory, the complete memory is obtained, which will pass through CA1 to recover its original format before leaving the network. The time it takes for the network to recall the complete memory from the time its cue is introduced until the complete memory is obtained is 25 ms. This would be the case of recall operations for a single memory. If the activity of the memory that is passed back to DG, through EC, contains some kind of information concerning another memory (namely, the cue), this information would enter the network again to start its recall. In short, thanks to EC in conjunction with the use of a subset of neurons from the content of one memory to contain the information concerning the cue of the next memory in a sequence, sequential recall is achieved. The propagation delay activity of the neurons that return the information to the input to achieve this sequentiality was regulated to allow time for the input spike train to be processed by the network before starting the next memory in the sequence. Finally, there is the forgetting memory operation. This operation does not occur explicitly in the network, but occurs indirectly by attempting to learn a memory whose cue is common to another previously learned memory. At the start of the learning operation, the new memory will pass through DG which, while dispersing the activity of the cue, creates a time shift in the spatial encoding of the memory. That is, the content of the memory will leave DG and thus enter CA3 before the cue. This cue, being common to another previously learned memory, will trigger the recall of this previous memory. In addition, CA3cont neurons are configured to reduce the frequency of the input spike train. Putting these factors together, the following sequence of neuron activation occurs. First, the neurons encoding the content of the new memory are activated, followed by the activation of the neurons encoding the cue, and finally, the activation of the neurons encoding the content of the old memory. This sequentiality is exploited by the learning mechanism and results in a decrease in the weight of the synapses storing the old memory, causing it to be forgotten, and an increase in the weight of the synapses storing the new memory, causing it to be learned. This operation requires the same execution time as a normal learning operation, i.e., 300 ms. 3.3. Application to Sequential Trajectory Recall The sequential hippocampus memory model was applied to robotic navigation. The purpose of this application is for the memory model to be able to learn and recall different trajectories to a goal position within a grid environment. This information must be sufficient, in order for the robot to be able to reach a goal position from any position within the environment. To this end, the content of the memory will be interpreted as a map of the environment where each position contains information about the next position the robot has to move to to reach the goal position. Therefore, each memory will consist of a unique position in the environment as the cue and the next neighboring position to move to as the content. In this case, the content of the memory will be constituted only by the information of the next memory. After learning the complete map information, by starting a recall operation on an initial position in the environment, the hippocampal memory model can return the sequence of positions that the robot has to pass through to reach the goal position. In other words, the hippocampal model is able to tell the robot, www.advancedsciencenews.com www.advintellsyst.com Adv. Intell. Syst. 2024, 2400282 2400282 (5 of 16) © 2024 The Author(s). Advanced Intelligent Systems published by Wiley-VCH GmbH 26404567, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/aisy.202400282 by Readcube (Labtiva Inc.), Wiley Online Library on [09/10/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License for each position in the environment, which trajectory it must follow to reach the goal position. 4. Experimentation and Results The proposed bio-inspired sequential hippocampus memory model was implemented on the DYNAP-SE hardware platform. This hardware implementation of the model presents a capacity to learn and store up to 15 different memories at the same time (N), where each memory is defined by the activity of 8 neurons (M) and with a delay factor of 24 (D). Of the 8 neurons that define memory, the first 4 represent the cue and the remaining 4 the content. For the specific experiments carried out, we focused on testing the sequentiality of the memory. Therefore, the 4 content neurons are used to identify the next memory to be recalled in the sequence, i.e., the next cue in the sequence. For these characteristics, taking into account the equations given in Section 3, the network defining the model will be constituted by a total of 182 neurons (59 from DG, 19 from CA3, 8 from CA1, and 96 from EC), 334 static excitatory and inhibitory synapses (24 from IN-DG, 144 from DG-DG, 19 from DG-CA3, 19 from CA3-CA1, 8 from CA1-OUT, and 120 from EC-EC, CA1-EC, and EC-DG), and 60 dynamic synapses with the triplet STDP learning mechanism (from CA3cue-CA3cont). All inhibitory synapses used are GABA B type and, although this particular implementation was used, the model was also tested for other network sizes of smaller and larger capacity in terms of both number of memories and memory size. Regarding the hardware implementation of the model, a set of experiments was developed to verify its correct functioning. These experiments consisted in stimulating the model by connecting its input to a neural activity generator (contained in the hardware platform itself ) to observe its behavior in response to the input of information in the form of memories. Specifically, all experiments were carried out in the context of applying the model to sequential trajectory recall with and without noise. Each memory consists of the position it represents in the grid environment as the cue and the next position in the sequence as the content. Experiments to verify the basic functionalities can be found in ref. [26]. In these experiments, the grid map of the known environment to learn and recall trajectories is presented in Figure 3. It has a total of 16 positions. Position 2 is the goal position and position 16 is not accessible. None of these special positions has any additional information. The remaining 14 positions mark the next position to reach the goal position. The grid map supports 8 positions adjacent to the current one, although, for the initial case, only the 4 main positions (north, south, east, and west) are used. Given an initial position, it is guaranteed that there is a sequence of positions that allows the goal position to be reached. This sequence of positions is obtained by following the arrows indicated in each position. For example, starting from position 11, the next position would be 7, 6, and 2, respectively. In addition, each experiment includes a raster plot that gives a summary of the network’s spiking behavior from the input activity to the network’s output. The x-axis represents the progression of time in milliseconds, while the y-axis represents each neuron in the network, identified by its population and internal ID. Each point on the plot represents a spike fired by the neuron specified on the y-axis at the corresponding time on the x-axis. 4.1. Operational Test: Learning and Sequential Recalling without Noise The first experiment attempts to demonstrate the sequential operation of the model with the incorporation of EC. To this end, this experiment has a first phase of learning the trajectory map of the known environment, and a second phase of recalling the trajectories from each possible initial position to the goal position. In the learning phase, a total of 14 learning operations are performed, one for each of the positions with information about the next position in the sequence. This phase begins with the learning of position 1 at ms 0 and ends with the learning of position 15 at ms 13 000. The activity of the network during this phase of the experiment, along with each of these learning operations marked, are presented in Figure 4a. As an example of learning operation, the learning of position 3 was taken. At position 3, the memory is formed by position 3 as cue and position 2 as content, which corresponds to the activation of the input neurons of the network 0 and 1 for cue and 5 for content, considering a spatial binary encoding of these values. This activity is generated in the form of spike trains and fed into the network a total of three times starting at ms 1000. Each time, the cue will pass through DG to be dispersed, leading to the activation of neuron 2 in CA3cue, while the content will pass unchanged through DG activating neuron 1 in CA3cont. At that instant, the triplet STDP learning mechanism in CA3 will be activated, leading to an increase of the weight at those synapses connecting the activated CA3cue and CA3cont neurons and, with it, the learning of memory occurs. The dispersed memory then arrives at CA1 where it regains its original format, i.e., activation of neurons 0, 1, and 5, before leaving the hippocampus at ms 1750. The same is true for the remaining learning operations with their respective encoding of information. In the recall phase, a total of 14 recall operations are carried out, one for each of the possible positions in which the robot could find itself (ignoring the goal position itself ). The activity Figure 3. Grid map of the known environment over which the robot will navigate. For each position, the position identifier is marked in the upper left corner and, in the center, the next position through which to advance (marked by an arrow) to reach the goal position (green circle). The red cross marks a position that is not accessible due to the presence of an obstacle. www.advancedsciencenews.com www.advintellsyst.com Adv. Intell. Syst. 2024, 2400282 2400282 (6 of 16) © 2024 The Author(s). Advanced Intelligent Systems published by Wiley-VCH GmbH 26404567, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/aisy.202400282 by Readcube (Labtiva Inc.), Wiley Online Library on [09/10/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License of the network during this phase of the experiment, along with each of these recall operations marked, can be observed in Figure 4b. As an example of recall operation, recall of position 9 was taken. To start the recall operation for position 9, the network receives the activation of input neurons 0 and 3 as a cue via spike trains at ms 1500. After passing through DG, the cue is dispersed, leading to the activation of neuron 8 in CA3cue. The activity train generated by this neuron in CA3cue is transmitted to CA3cont where it will activate those neurons which it was associated with during previous learning (neurons 1 and 3 of CA3cont). These neurons represent position 10, i.e., the next position to which the robot must move to reach its goal position. After passing through CA1 and recovering its original format, the memory will leave the network. At the same time, the activity of these neurons encoding position 10 will return to the network as the cue of a memory through EC (neurons 1 and 3 of IN), triggering the start of a new recall operation immediately. This second recall operation is performed automatically and, as a result, the next position to 10 (neuron 9 of CA3cue) in the sequence is position 6 (neurons 1 and 2 of CA3cont). This activity will return to the network again through EC as the cue of another memory (neurons 1 and 2 of IN) and will start a new recall operation. In this third recall operation, it is obtained that, if the robot is at position 6 (neuron 5 of CA3cue), the next position toward the goal is position 2 (neuron 1 of CA3cont). Finally, this activity returns to the network input (neuron 1 of IN) to start the fourth and last recall operation for this trajectory. As the current position on the map corresponds to the goal position, there is no next position, i.e., the sequence of memories and recall operations end. This same sequential process occurs for the remaining recall operations, each with its own path length to the goal position and, thus, its own memory sequences. 4.2. Noise Analysis Stress Test This experiment has the same basis as the previous experiment, learning the information from the map and recalling the trajectories from each possible position in the environment. Unlike the previous experiment, it introduces noise at the input of the network to analyze the behavior of the model in the presence of external noise sources. To generate the noise, each input neuron of the network is connected to a Poisson generator available on the DYNAP-SE1 hardware platform. [60,64] For the presented implementation, a total of 8 Poisson generators are used. Each Poisson generator is able to generate spikes following a Poisson distribution. [65] Thus, given a period, the Poisson generator will generate a spike train in which each spike will be shifted by a random probabilistic error within its period. [64,65] In other words, it will generate a spike at a random Figure 4. a) Raster plot of the spiking activity of the network during the learning of the whole map and b) the recalling of the trajectories from all possible initial positions. Each operation is marked together with the identifier of the map position involved. www.advancedsciencenews.com www.advintellsyst.com Adv. Intell. Syst. 2024, 2400282 2400282 (7 of 16) © 2024 The Author(s). Advanced Intelligent Systems published by Wiley-VCH GmbH 26404567, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/aisy.202400282 by Readcube (Labtiva Inc.), Wiley Online Library on [09/10/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License instant within each time interval defined by its period. Specifically, the period of operation of the generators is defined by their frequency. This experimentation considers different variations or test cases of noise application at the network input: 1) The amount of noise applied. This quantity is defined by the frequency in Hz of each of the Poisson generators. The values explored are 0.5, 1, 2, 3, 4, and 10 Hz. 2) The phase of the process that is affected by the noise. A distinction is made between applying noise only to the learning phase, only to the recall phase, and to both phases. 3) The memory fragment which the noise is applied to. A distinction is made between applying the noise only to the cue of the memory, only to the content of the memory, and to the whole memory. Each of these cases covered in the experimentation was repeated a total of 5 times for each of the 14 possible initial positions of the map. This leaves a total of 756 contemplated cases repeated 5 times or 3780 tests based on the introduction of noise carried out on the model. In addition, for each test, a learning phase and a recall phase were carried out. Due to the large number of test cases in the experiment, there are a large number of figures reflecting the behavior and outcome of the network during these tests. Therefore, the most significant figures are shown, and the remaining figures are added as additional material and are also available in the repository of this work (see Section 6). Figure 5 shows the spiking activity of the network as a result of the first test case of the experiment, i.e., the application of 0.5 Hz Poisson noise to the input only during the learning phase and only to the cue of the memory. As the cue of the memory is formed by the activity of 4 neurons, only 4 Poisson generators are used at the input of the network. The noise introduced into the network by these generators (pink dots in Figure 5a) is mixed with the input activity and propagates through the network. It is necessary to look at the output of the network during the recall phase, Figure 5b, to see what effect this noise has had during the learning process. As this is one of the cases with the least amount of noise introduced into the network, all operations were performed correctly except for one, the recall of the trajectory starting at position 8. In the test without noise, the trajectory should start at position 8 and progress through positions 4, 3, and 2, whereas, in this iteration of the test case, the trajectory is defined by positions 8, 15, 14, 10, 6, and 2. SNRðdBÞ¼10 log10 finput fnoise  (1) Figure 5. a) Raster plot of the spiking activity of the network during the noise analysis stress test for the learning of the whole map and b) the recalling of the trajectories from all possible initial positions. In this test case, a Poisson noise of 0.5 Hz for each generator is introduced into the network only in the learning phase and only applied to the cue of the memory. In (b), a recall operation affected by noise during learning is marked. www.advancedsciencenews.com www.advintellsyst.com Adv. Intell. Syst. 2024, 2400282 2400282 (8 of 16) © 2024 The Author(s). Advanced Intelligent Systems published by Wiley-VCH GmbH 26404567, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/aisy.202400282 by Readcube (Labtiva Inc.), Wiley Online Library on [09/10/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License In this work, two types of noise can be distinguished: the “synthetic”noise introduced artificially into the model to show the performance of the network in the presence of an external noise source, and the noise introduced by the network and the platform itself derived from this input noise. To measure the impact of the input noise on the network, the signal-to-noise ratio (SNR) was calculated from Equation (1). This indicator allows analyzing the amount of noise with respect to the input information from the frequency of both. On the one hand, finput is the input frequency of useful information external to the network, it is defined by the input frequency of the spike trains of the different operations multiplied by the average of spikes that define these spike trains. For the case of the implementation used, the spike trains present, on average, 10 spikes and have a frequency of 7 Hz, giving a finput equal to 70 hz. On the other hand, fnoise is the external noise input frequency and is defined by the frequency of the Poisson generators times the number of Poisson generators acting. The SNR measured in decibels (dB) for the 5 frequencies explored in the 2 possible cases (depending on which part of the memory is affected) is shown in Table 1. We also considered the internal noise factor (INF) at the internal level of the network, not only at the input, to measure the noise induced by the network and the platform derived from the input noise. This factor is calculated from Equation (2), and is the result of dividing the amount of activity generated by the noise in the network by the amount of activity belonging to the useful information in the network. The average INF of this experimentation as a function of the frequency values of the Poisson noise generators is shown for the learning phase in Figure 6a and for the recall phase in Figure 6b. INF ¼number of spikes of noise in the network number of spikes of useful information in the network (2) To analyze the performance of the network, the hit rate was measured for each of the individual recall operations (recall hit rate) and for each recall of the entire path (path hit rate). In the recall hit rate, a hit is considered if the recall operation gets the next expected position, the one that would be obtained if the network were free of noise. In the path hit rate, it is considered a hit if, after finishing the recall sequence for an initial position, it is able to reach the goal position, regardless of whether it took the expected or an alternative path. The mean and standard deviation of the recall hit rate (Figure 7a and Table 2) and of the path hit rate (Figure 7b and Table 3) were calculated as a function of the frequency of the noise generators and the SNR values for each of the 9 sets of possible test cases. These sets of test cases arise from the combination of the possible phases and the possible fragments of the memory to which the input noise can be applied. Finally, to check the impact of noise on the network at the operation level, the average number of recall operations as a function of the frequency of the noise generators was obtained for each of the 9 sets of possible test cases. The results of these Table 1. SNR of the experiment as a function of the different frequencies of the Poisson noise generator for the set of test cases where the noise is only applied to the cue or only applied to the content of the memory (A) and for the set of test cases where the noise is applied to the complete memory (B). Frequency [Hz] 0.5 1 2 3 4 10 SNR (dB) case A 15.44 12.43 9.42 7.66 6.41 2.43 SNR (dB) case B 12.43 9.42 6.41 4.65 3.4 0 Figure 6. a) Mean rate of noise activity in the network versus useful information activity in the network during the noise analysis stress test for the different frequencies of the Poisson noise generators for the learning phase and b) the recalling phase. www.advancedsciencenews.com www.advintellsyst.com Adv. Intell. Syst. 2024, 2400282 2400282 (9 of 16) © 2024 The Author(s). Advanced Intelligent Systems published by Wiley-VCH GmbH 26404567, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/aisy.202400282 by Readcube (Labtiva Inc.), Wiley Online Library on [09/10/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License [52] L. F. Abbott, S. B. Nelson, Nat. Neurosci. 2000,3, 1178. [53] J. Sjöström, W. Gerstner, Spike-Timing Depend. Plast. 2010, 35, 574. [54] L. Khacef, P. Klein, M. Cartiglia, A. Rubino, G. Indiveri, E. Chicca, Neuromorphic Comput. Eng. 2023,3, 042001. [55] J.-P. Pfister, W. Gerstner, J. Neurosci. 2006,26, 9673. [56] R. Kempter, W. Gerstner, J. L. Van Hemmen, Phys. Rev. E 1999,59, 4498. [57] S. B. Furber, F. Galluppi, S. Temple, L. A. Plana, Proc. IEEE 2014,102, 652. [58] M. Davies, N. Srinivasa, T.-H. Lin, G. Chinya, Y. Cao, S. H. Choday, G. Dimou, P. Joshi, N. Imam, S. Jain, Y. Liao, C.-K. Lin, A. Lines, R. Liu, D. Mathaikutty, S. Mccoy, A. Paul, J. Tse, G. Venkataramanan, Y.-H. Weng, A. Wild, Y. Yang, H. Wang, IEEE Micro 2018,38, 82. [59] P. A. Merolla, J. V. Arthur, R. Alvarez-Icaza, A. S. Cassidy, J. Sawada, F. Akopyan, B. L. Jackson, N. Imam, C. Guo, Y. Nakamura, B. Brezzo, I. Vo, S. K. Esser, R. Appuswamy, B. Taba, A. Amir, M. D. Flickner, W. P. Risk, R. Manohar, D. S. Modha, Alvarez-Icaza, Science 2014, 345, 668. [60] S. Moradi, N. Qiao, F. Stefanini, G. Indiveri, IEEE Trans. Biomed. Circuits Syst. 2018,12, 106. [61] N. Qiao, H. Mostafa, F. Corradi, M. Osswald, F. Stefanini, D. Sumislawska, G. Indiveri, Front. Neurosci. 2015,9, 141. [62] R. Brette, W. Gerstner, J. Neurophysiol. 2005,94, 3637. [63] C. Bartolozzi, G. Indiveri, Neural Comput. 2007,19, 2581. [64] M. Nilsson, Monte carlo optimization of neuromorphic cricket auditory feature detection circuits in the dynap-se processor, 2018. [65] D. Heeger, Poisson Model of Spike Generation, Handout, Vol. 5, University of Standford, Stanford, CA 2000, p. 76. www.advancedsciencenews.com www.advintellsyst.com Adv. Intell. Syst. 2024, 2400282 2400282 (16 of 16) © 2024 The Author(s). Advanced Intelligent Systems published by Wiley-VCH GmbH 26404567, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/aisy.202400282 by Readcube (Labtiva Inc.), Wiley Online Library on [09/10/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License