Live Demonstration: Neuromorphic Row-by-Row Multi-convolution FPGA Processor-SpiNNaker architecture for Dynamic-Vision Feature Extraction

Live Demonstration: Neuromorphic Row-by-Row Multi-convolution FPGA Processor-SpiNNaker architecture for Dynamic-Vision Feature Extraction R. Tapiador-Morales, Juan P. Dominguez-Morales, D. Gutierrez-Galan, A. Rios-Navarro, A. Jimenez-Fernandez and A. Linares-Barranco Robotic and Tech of Computers Lab. University of Seville. Seville, Spain 41012 Email: [email protected] http://www.rtc.us.es/ Abstract— In this demonstration a spiking neural network architecture for vision recognition using an FPGA spiking convolution processor, based on leaky integrate and fire neurons (LIF) and a SpiNNaker board is presented. The network has been trained with Poker-DVS dataset in order to classify the four different card symbols. The spiking convolution processor extracts features from images in form of spikes, computes by one layer of 64 convolutions. These features are sent to an OKAERtool board that converts from AER to 2-7 protocol to be classified by a spiking neural network deployed on a SpiNNaker platform. Index Terms— Address-Event-Representation, Spiking Neural Networks, FPGA, Neuromorphic engineering, SpiNNaker. I. INTRODUCTION This document describes the live demonstration of a classification system using a spiking convolution processor implemented on FPGA and a spiking classifier running on SpiNNaker. The convolution processor convolves the visual input stimulus from a card image, and it fires those events that reach a threshold value after the convolution. Output events are processed by a spiking neural network (SNN) model deployed in SpiNNaker, which classifies the corresponding card symbol (spade, diamond, club, heart). In previous work [1], the convolution processor was used to perform multiple images convolution over the Poker-DVS [2] dataset under FPGA post-implementation simulation. Currently, the design has been implemented in a Zynq- 7100 MMP platform. This design is able to perform 64 convolutions of 128x128 image size with kernel sizes from 1x1 to 7x7 in 1.3µs and 9.01µs. SpiNNaker is a massively parallel multicore processor consisting of several chips, each one being powered by eighteen ARM9 cores. In this work we have used a SpiNN-3 machine that has 4 SpiNNaker chips and is able to simulate up to 1024 LIF neurons. This board also has two spinnaker links which allow real-time input/output communication. II. DEMONSTRATION SETUP This demonstration uses the Zynq-7100 MMP platform implementing the LIF convolution processor connected to a DVS retina and OKAER tool [3] , which communicates with the SpiNNaker board. Output AER events will be sent to the SpiNNaker board through one of the spinnaker links and it is used as input for the SNN. The DVS retina will be capturing events and the classification results of SpiNNaker will be sent to a computer to be visualized, using the Ethernet connector as is shown in Fig.1. As shown in Fig. 2, the card symbol can be played from a USBAERmini2 board, or use this board as a passthrough monitor of the DVS retina. III. VISITOR EXPERIENCE Visitors will be able to test the classification system moving a chosen poker card in front of a DVS retina. The result of each convolution module will be shown in real time in jAER software with the result of the SpiNNaker classification. Fig. 1: Card image events are processed by the LIF FPGA Convolution processor, OKAER Tool board and Spinnaker. Results are shown in a PC running jAER software and a Graphical user interface(GUI). Fig. 2: Hardware set-up. A) DVS Retina. B) MMP with LIF convolution processor. C) OKAERtool board that receives events from B, translate and sends them to D. D) SpiNN-3 board running a SNN. REFERENCES [1] R. Tapiador-Morales et al., “Event-based row-by-row multi-convolution engine for dynamic-vision feature extraction on fpga,” in 2018 International Joint Conference on Neural Networks (IJCNN), July 2018, pp. 1–7. [2] T. Serrano-Gotarredona et al., “Poker-DVS and MNIST-DVS. Their History, How They Were Made, and Other Details,” Frontiers in Neuroscience, vol. 9, p. 481, 2015. [3] A. Rios-Navarro et al., “A 20mevps/32mev event-based usb framework for neuromorphic systems debugging,” in 2016 Second International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP), June 2016, pp. 1–6.