AER tools for Communications and Debugging

AER tools for Communications and Debugging F. Gomez-Rodriguez, R. Paz, A. Linares-Barranco, M. Rivas, L. Miro, S. Vicente, G. Jimenez, A. Civit. Departamento de Arquitectura y Tecnologia de Computadores. Universidad de Sevilla. Seville, SPAIN gomezroz(atc.us.es AbstractAddress-Event-Representation (AER) is a the same pulse stream. The receiver cell integrates the communications protocol for transferring spikes between biopulses and reconstructs the original low frequency inspired chips. Such systems may consist of a hierarchical continuous-time waveform. structure with several chips that transmit spikes among them Transmitting the pixel addresses allows performing extra in real time, while performing some processing. To develop operations on the images while they travel from one chip to and test AER based systems it is convenient to have a set of another. For example, inserting memories (e.g. BEPROM) instruments that would allow to: generate AER streams, allows transformations of images. monitor the output produced by neural chips and modify the spike stream produced by an emitting chip to adapt it to the b iThere is a growing community of AER protocol users for requirements of the receiving elements. In this paper we bio-inspired applications in vision and audiion systems, as present a set of tools that implement these functions developed demonstrated by the success in the last years of the AER in the CAVIAR EU project. group at the Neuromorphic Engineering Workshop series [2]. The goal of this community is to build large multi-chip hierarchically structured systems capable of performing I. INTRODUCTION complicated array data processing in real time. The Address-Event-Representation (AER) was proposed in CAVIAR EU project has the objective to demonstrate this 1991 by Sivilotti [1] for transferring the state of an array of technology by targeting and following a moving ball. The neurons from one chip to another. It uses mixed analog and planned AER system under CAVIAR uses the following digital principles and exploits pulse density modulation for AER chips: one Retina, four Convolutions, four Winnercoding information. The state of the neurons is a continuous Take-All (Object) and one Learning chip. time varying analog signal. To make possible the right communication of these chips Figure 1. explains the principle behind the AER. The and for debugging purposes it is essential to have a set of emitter chip contains an array of cells (like, e.g., an imager instruments that would allow to: or artificial retina chip) where each pixel shows a state that - Sequence: Produce synthetic AER event streams that can changes with a slow time constant (in the order of be used as controlled inputs while testing and adjusting a milliseconds). Each pixel includes an oscillator that chip or set of chips. generates pulses of minimum width (a few nanoseconds). - Monitor: Observe the output of any element in the Each time a pixel generates a pulse (called "event"), it system. communicates with the periphery and its address is placed - Map: Alter the stream produced by an emitter and send on the external digital bus (the AER bus). Handshaking the modified stream to a receiver lines (Acknowledge and Request) are used for completing For these purposes we have designed and implemented the communication. two different instruments: a PCI-based board capable of =HPI CHEP2 H fl = _ _ = C2sequencing and monitoring events at a rate of over /'+~~~~ FLLLL AST / 15MAevents/s and a versatile USB-based board that can be l DIGITALJ used for sequencing, monitoring and mapping. This last board can be used either in a stand alone mode or connected BS mto an external computer through a USB bus. A possible D3 K D3 02 111111 1 Xscenario for these tools is shown in Figure 2. where a computer with a PCI-AER [9] board produces output for Figure 1. AER inter-chip communication scheme. AER chip 1. The output from this chip is remapped by an USB-AER board and fetched to AER chip 2. The stream In the receiver chip the pulses are directed to the pixels or produced by chip 2 is monitored by another USB-AER cells whose address was on the bus. This way, pixels with board which can send its output directly to a VGA monitor the same address in the emitter and receiver chips will "see" or to a computer through USB bus. 0-7803-9390-2/06/$20.00 ©2006 IEEE 3253 ISCAS 2006 AER Chip 2 used also as a sequencer or a monitor. Due to the bandwidth l l(Tcsting PCB With limitations of full speed USB 1.1 (I2Mbit/s) hardware based AER in and out AERb -. __ connectors event to frame conversion is essential in this board for high, or even moderate, event rates. / -~~~~~~~S AF arasThe USB-AER board is based around a Spartan-II 200 Xilinx FPGA, with a 512K*32 12ns SRAM memory bank. USB~~AER board as __ The board uses a Silicon Laboratories C805I1F320 US-__rAvER oard framegrabbr microcontroller to implement the USB and the MMC/SD /rl0ni 7 AER_ Chip 1interface. The USB interface allows up to 6 Mbit/s. A (TestingaPDwt simple VGA monitor interface is also provided to allow the AER in and out PCLAER board as connedurs) board to act as a monitor (frame-grabber). sequencer of AER The board will act as a different device according to the e(fen based)or module that is loaded in the FPGA either through a event based) MMC/SD card or from the USB bus. Currently the following modules are implemented: Figure 2. AER tools usage scenario A. Mapper Mapping functionality consists in a change of the events To be useful for debugging an AER tool should be able to adrse.Tsschnscnb: receive and also send a long sequence of events interfering Onest one: This man i ne - One to one: This mapping iS used in one to one as little as possible with the system under test. communication, when the addresses of the sender As neurons have the information coded in the frequency device are different from the addresses of the (or timing) of their spikes, the pixels that transmit their receiver device. address through an AER bus also have their information reeiverevic. coded in the frequency of appearance of those addresses in One to one relicated: This mapping replicates, for the bus. Therefore, inter-spike-intervals (ISIs) are critical for each input event, up to 8 times the same output this communication mechanism. Thus, a well designed tool event. shouldn't modify the ISIs of the AER. - One to several: This mapping is used in one to Sections II presents the USB solutions and their several communication. In this case one input event applications in AER testing. Section III presents a Switchin sent to several output addressed. AER. Section IV present a reduced version of the USB- - No event: this mapping no emits event. AER board with lower capabilities and performance, but Different mapping types can be specified for each input very simple to use and better for low traffic applications. address. And finally in section V we conclude with one example of B. Monitor connectivity. This functionality allows to record the traffic in an AER II. USB-AER INTERFACE bus and to represent it in a VGA or in a computer. Monitor The CAVIAR PCI-AER board [9] can perform Address types are: Event sequencing and monitoring functions but has no - Frame-grabber: The USB-AER board is inserted in hardware mapping capabilities. Although software based the AER bus, and it captures the events during a mapping is feasible, if we want to build AER systems that time; then reconstructs and image and sends it to the can operate without requiring a computer, we need a computer using the USB port. specific device for this purpose. The USB-AER board - Sniffer: The USB-AER board is connected to the allows standalone operation and has several functionalities, AER bus without interfering in the traffic. In this like hardware mapping and more. case the sniffer must be faster than the other AER This standalone operating mode requires to be able to devices connected to the AER bus. The USB-AER load the FPGA and the mapping RAM from some type of board captures the events and stores it with a non volatile storage that can be easily modified by the users. timestamps; when all internal RAM is full, it sends MMC/SD cards used in digital cameras are a very attractive the information to the computer using the USB port. possibility. However in the development stage users prefer - Logger: The USB-AER board is inserted in the AER to load the board directly from a computer and, for this bus, and captures the events with timestamps. In this purpose USB seems the most suitable solution. case the logger interferes in the AER traffic. Many AER researchers would like to demonstrate their - VGA: the USB-AER board captures the events and systems using instruments that could be easily interfaced to converts it in a B&W image and sends it using the a laptop computer. This requirement can also be supported VGA connector. A Gray VGA is possible using a with the USB-AER board as it includes a relatively large VGA-DAC board connected to the out-AER FPGA that can be loaded from MMC/SD or USB, a large connector of the board. SRAM bank and two AER ports. Thus the board can be 3254 C. Sequencer monitor. Furthermore an API for C++ and for MATLAB is In this case the USB-AER board is used as events also available. generator, there is two types of AER traffic generation: - Frame-based generation: The USB-AER board A Linux driver for the USB-AER is currently under test. generates events using a digital image stored in a With this driver the USB-AER board can be easily computer. Computer sends to the board a digital integrated with several MATLAB applications developed at image the USB-AER board converts it in an events INI [8]. stream that can be transmitted through AER bus. To III. AER-SWITCH convert the image to events stream Random or Exhaustive methods [4][5][6] are used. This board allows to connect several AER devices to the - Player: The USB-AER board generates events using same AER BUS. The connection possibilities are: 4 to 1 or timestamp information. This information is sent to I to 4. This board allows: the board using the USB port, or this information can - The connection of more complex AER systems. be the information captured using the board as a - An easier debugging by inserting PCI-AER or USBlogger o as a sniffer. AER board without modifying the structure of the global system to be tested. The functionality sniffer, logger and player are implemented in the same module. The USB-AER can This board has a CPLD as a communication centre that capture sequences of up to 512K events with timestamps manages the different modes and controls the protocol lines. and play these sequences back, without reprogramming it, It can work in two modes: only sending the correct command. These modules are very - Merger: 4 input and 1 output interesting when a researcher wants to use the output stream - Splitter: 1 input and 4 oututt mode. produced by a chip from another researcher as input to his Both can be unicast (selecting one output) or broadcast. This or her chip. functionality should be configured by jumpers. There are 5 different AER ports; one of them works always as an This new board was interfaced in Telluride 04 [7] to the output, and another as an input. The others three are current version of the CAVIAR retina and to an imager bidirectional and work as input or output depending on the developed at JHU. Later in the CAVIAR meeting in mode of operation. Figure 4 shows the current version of September 04 it was interfaced to the remaining project this board. chips. Later in May 05 it was used successfully during the CAVIAR project workshop. The USB-AER board is shown in Figure 3. _ ~~~Figure3. Us BAE BoaXrd A simple interface to control this board is available underFiue4AE-wthBad windows. It allows to load modules into the FPGA, gue4AE-wthBad uploading or downloading data to or from the board, and showing the received images when the board acts as a 3255 IV. MiNI USB-AER USB-AER board AER bUS (jATA-100) wilth AER Synthetic For those tests or applications where it is not needed high Generator speed performance, a reduced version of the USB-AER (sequencer) I board is available. This one doesn't have FPGA and MMC/SD card. This board can be connected to the PC B USSWAER board as through the USB bus, the functionality (Monitor or Sequencer) has to be programmed into the microcontroller under C code. Figure 5. shows the current version of this board. PCKAER board at monitor of AER (aevnt brased) USB~AER board aER Sic or AER generator {2 inpUt, output) Figure5. Mini AER-USB Board. (sequencer) ER board as Figure m lrElEs =apper hi I V. CONCLUSIONS , zZ AER e eind out A set of tools has been developed that allow efficient con hectors testing and demonstration of address event based systems. rdwitdboar Two demonstration scenarios are shown in Figure 6. In the u~ilPt 2 ut) us(ATA-100) USB~AER board as........ first case a USB-AER board is generating a stream of frarnegtabber events, using a hardware synthetic method, that correspond (monitor) to the image shown in Figure 6. on the laptop's screen, Figure 6. Two demonstration Scenarios. which is rotated by a second USB-AER board used as a mapper to produce an AER events stream, which is captured REFERENCES by a PCI-AER board used as a monitor to produce the [1] M. Sivilotti, Wiring Considerations in analog VLSI Systems with output shown in the desktop PC's screen. Application to Field-Programmable Networks, Ph.D. Thesis, In the second scenario an USB-AER is working as a California Institute of Technology, Pasadena CA, 1991. frame to AER sequencer to feed a AER chip. This chip [2] A. Cohen, R. Douglas, C. Koch, T. Sejnowski, S. Shamma, T. receives also the transformed output of another AER chip Horiuchi, and G. Indiveri, Report to the National Science Foundation: using the AER-Switch. The output of the second chip can be Workshop on Neuromorphic Engineering, Telluride, Colorado, USA, viewed on the laptop's screen, using another USB-AER as a June-July 2001. [www.ini.unizh.ch/telluride] monitor. [3] A. Wilen, J.Schade, R. Thornburg. "Introduction to PCI Express", In this scenario only the presented tools are shown. In Ine rss 03 Inal thisd cascenario onlyt thehpres tools are shee o wo Iune [4] A. Linares-Barranco. Estudio y evaluaci6n de interfaces para la real world cases the tools are used to evaluate or to tune conexi6n de sistemas neurom6rficos mediante AddressEventneural chips. In the CAVIAR project the chips have been Representation. Ph.D. Thesis, University of Seville, Spain, 2003 interfaced to two different retinas, a convolution chip, a [5] A. Linares-Barranco, G. Jimenez-Moreno, A. Civit-Ballcels, and B. winner take-all (object) chip and a learning chip. Linares-Barranco. "On Algorithmic Rate-Coded AER Generation". Accepted for publication on IEEE Transaction on ACKNOWLEDGMENT Neural Networks. [6] Linares-Barranco, A.; Jimenez-Moreno, G.; Civit-Ballcels, A.; This work was in part supported by EU project ISTLinares-Barranco, B.; On synthetic AER generation. ISCAS '04. 2001-34124 (CAVIAR), and Spanish project TIC-2003Proceedings of the IEEE 2004 May 2004 Pages:V-784 - V-787 Vol.5 08164-C03-02 (SAMANTA). [7] Avis Cohen, et.al., Report on the 2004 Workshop On Neuromorphic Engineering, Telluride, CO. June - July , 2004 [www.ini.unizh.ch/telluride/previous/reportO4.pdf] [8] M. Oster, Serverbased Software Architecture for AER systems [http://www.ini.unizh.ch/-mao/AerSoftware/SoftwareOverview.pdfl [9] R. Paz-Vicente, A. Linares-Barranco, D. Cascado, S. Vicente, G. Jimenez, A. Civit. PCI-AER interface for Neuro-inspired Spiking Systems. Submitted to ISCAS 2006. Kos, Greece. 3256