CNN technology in action

2000 6w IEEE International Workshop on Cellular Neural Networks and neir Applications proceedings CNN Technology in Action Demo Session Akos Zarindy*, Servando Espejo', Pdter F61desyf*, Lbsz16 Kdk*, GusztAvo Linin', Csaba Rekeczky', Angel Rodriguez-Vhquez', Tam& Roska', IstvAn Szatmki', Tam@ Szirhyi', P6ter Szolgay' 'Analogical and Neural Computing Laboratory of the Computer and Automation Research Institute of the Hungarian Academy of Science email: [email protected] + Institute of Microelectronics of Seville of the Spanish National Microelectronics Center email: [email protected] ABSTRACT: Two CNN-LIMprototypes are demonstrated in action. TheJrst one is the latest 4096 cell-processor, analog I/O, analogic CNN Visual Microprocessor, on which on-line video image processing will be pe$ormed The second one, the 20x22 binary input-binary output CNN-UMchip is introduced as an ultra high-speed focal plane array processor. In the live demonstration it captures and classifes IO, OOOframes in a second. 1. Introduction In the last few years the analog VLSI CNN chips [1,2,3,4] and their embedding softwarehardware environment [5] reached a high sophistication level, which makes the CNN technology ready to be applied in industry or in commercial products. This session is devoted to demonstrate the advanced features of the new CNN chip based systems. The demonstrated analogic CNN visual microprocessors were designed in the Institufe of Microelectronics of Seville of the Spanish National Microelectronics Center. Using the CNN-UM chips, the first prototypes of a visual computer (called Aladdin) was designed and built in the Analogical and Neural Computation Laboratory of the Computer and Automation Research Institufe of the Hungarian Academy of Science. The first demonstration (Section 2) introduces the 64x64 CNN-UM chip [3] as an on-line video flow processor, while the second one (Section 3) applies the 20x22 chip [4] as an ultra high-speed focal plane array. 2. On-line video flow processing Motivations: In the last few years particle detection and classification in fluid-flows have received considerable interest among the applications requiring image processing at ultra high 6ame rates. For instance, a sensory module capable of identifying the density of debris particles in the oilflow of various engines would enable a cost-effective on-line monitoring of these systems (e.g. condition based monitoring ofjet engines). Figure 1. The experimental setup of the marble-bubble detection system. Cellular neural/nonIinear network (CNh') technology offers a parallel and analogic (combined analog and logic) approach to these problems. The c" chip can be used either as a focal-plane array processor or a video-fow processing visual microprocessor. In the latter case recent feasibility studies 0-7803-6344-2/00/$10.00 02000 IEEE 79 and experiments indicate that in a demo prototyping system detection and classification of the particles can be performed on-line on a 64x64 CNN chip [3]. Tusk specification and the demo system: Figure 1 shows the experimental setup of the on-line video-flow demonstration. In a water tank containing bubbles and marbles, a fast turbulent flow is generated. The task is to detect and separate the marbles from air-bubbles in each acquired image. The demonstration aims to prove that a morphology based complex algorithm can be executed during an on-line vide-flow processing in a CNN system. -75 pec - 150 pec / on the 64x64 Figure 2. The flow-chart ofthe marble-bubble detection algorithm. The time requirement of each steps is also indicated. General idea of the solution: (i) adaptive thresholding: all objects are detected in the image field through a spatially adaptive thresholding, (ii) morphologicalpre-filtering: objects are compared to prototype objects to filter out single bubbles and bubble-groups, furthermore to classify the remaining objects into different particle groups, (iii) object classifcation: in the last stage objects are classified based on their size and morphology (and a simple statistics is calculated for different object groups). The on-chip time performance of major subroutines of the algorithm is summarized in Figure 2 (no transfer and display time included): 3. Ultra high frame-rate image capturing and processing Motivations: Ultra high frame-rate (exceeding 10 000 frame/sec) image processing is an unsolved problem with current digital systems of affordable price and size. Both the limited computational power and the I/O bottleneck (when the image is transferred from the sensor to the processor) represent major obstacles in digital systems. Cellular neuraUnonlinear network (CNN) technology offers a parallel and analogic (combined analog and logic) approach to these problems. If a CNN chip is used as a focal-plane array, the zero computational load requirement is satisfied immediately. This chip [4] acts as a focal-plane visual microprocessor: acquires image frames parallel through the optical input, transfers them to the processor elements and performs the analysis also in parallel. In 20 pee, approximately 5 analog operations (CNN templates) and 10 local logic operations can be completed. This makes it possible that even a complex morphological decision can be performed within two subsequent frames at a 50 000 frameshec operational speed. Figure 3. The experimental setup of the ultra high speed, focal plane array processor system 80 Task specification and the demo system: The experimental setup is shown in Figure 3. The CNN platform which carries the chip is mounted on the back panel of a camera (only the optics is used, no shutter is required). On a rotating disk different images are posted and during the experiment these images are projected to the chip through the lens system of the camera. The demonstration proves that the system is able to classify six different flying objects (hot-air balloons and airplanes) based on their silhouettes’ low resolution projections on the chip’s optical sensors at a speed of approximately 10 000 framedsec. In Figure 4 the major subroutines of the algorithm are shown along with their measured onchip time performance (no transfer and display time included). Detailed description of this experiment can be found in [6]. I Objects as captured by the chip’s optical input. __-____-____-__ Each captured object is analyzed by a set of CNN operations (shape, size, orientation). ___--__-_-__-__ Finally, each object is Figure 4. The flow-chart of the image classification algorithm. 4. Conclusion Analogic visual computers, based on CNN technology are demonstrated in action. It is proved that the CNN technology is ready to be applied in industrial vision application or in commercial products. References [I] [2] [3] L.O. Chua and L. Yang, “Cellular Neural Networks: Theory and Applications”, IEEE Transactions on Circuits andSystems, vol. 35, no. IO, October 1988, pp. 1257-1290 T. Roska and L.O. Chua, “The CNN Universal Machine: An Analogic Array Computer”, IEEE Transactions on Circuits andsystems - II, vol. 40, March 1993, pp. 163-173, 1993 S. Espejo, R. Dominguez-Castro, G. LiRbn, and A. Rodriguez-Vbzquez, “A 64x64 CNN universal chip with analog and digital U0 Proc. 5~ Int. Conf. on Electroonics, Circuits and Systems (ICECS-98), Lisbon, Portugal, pp. 203-206 1998. S. Espejo, A.Rodriguez-VAzquez, R. A. Carmona, P. Foldesy, A. Zarandy, P. Szolgay, T. Szirhnyi, and T. Roska, “0.8Fm CMOS Two Dimensional Programmable Mixed-Signal Focal- Plane Array Processor with On-Chip Binary Imaging and Instruction Storage”, IEEE Journal on Solid State Circuits, Vol. 32. No. 7. pp.1013-1026. July, 1997. T. Roska, A. Zarandy, S. Ziild, P. Foldesy and P. Smlgay, ”The Computational Infrastructure of Analogic CNN Computing - Pari 1: The CNN-UM Chip Prototyping System“, IEEE Trans. on Circuits and Systems I: Special Issue on Bio-Inspired Processors and Cellular Neural Networks for Vision, (CAS-I Special Issue), Vol. 46, No.2, pp. 261-268, 1999 P. A. Zarandy, M. Csapodi, T. Roska, ”20 pec focal plane image processing”, in this proceedings. [4] [5] [6] 81