Enabling Real-Time Multispectral Imagery Processing And Object Detection With Edge Computer On Unmanned Aerial Vehicles.

XXX-X-XXXX-XXXX-X/XX/$XX.00 © 20XX IEEE Enabling Real-Time Multispectral Imagery Processing And Object Detection With Edge Computer On Unmanned Aerial Vehicles. Hsin-Hung Tseng Department of Civil Engineering National Chung Hsing University Taichung City, Taiwan [email protected] Ming-Der Yang Department of Civil Engineering National Chung Hsing University Taichung City, Taiwan [email protected] Abstract—Unmanned aerial vehicles (UAVs) have become increasingly popular in recent years due to the maturity of UAV systems and the decreased hardware costs. In the remote sensing domain, many UAV-based remote sensing technologies and hardware have been developed, making high spatial-temporal resolution imaging more affordable and convenient. However, detailed image data can only be obtained after the UAV returns to base. If the data quality does not meet the requirements, the mission must be replanned, which will incur additional manpower and time costs. If the mission has a time window, the valid collection time will be missed and the mission will fail, e.g., crop growth or disaster surveys, which have high spatial-temporal varying characteristics. This study proposed a complete procedure of real-time multispectral imagery processing and anomaly detection on UAVs. The imagery process includes quality verification, distortion correction, radiometric calibration, band alignment, and data normalization. Especially, the band alignment process requires a complex vision-based calculation. The study applies a convolutional neural network with multispectral imagery on real-time broccoli flower detection as the experiment. The concept is to collect the maturity of broccoli for the optimized harvest timing. The study adopts the model - EDANet and the modified one that trains with a dataset that comprises 955 training, 284 validation, and 128 test samples. The model modification adds two latent connections from the two dense connection blocks to the cascade upsampling layers, which adds detailed features to enhance object edge segmenting. The preliminary result shows a promising detection rate that applies to the field. Also, this study adopts the model optimization to improve the inference throughputs and latency to achieve real-time detection on the UAV with edge devices. The study applies the inference to two types of edge hardware, general-purposed mini-PC, and AIoriented edge computers, and compares the performance, price, compatibility, etc. Keywords—multispectral, deep learning, CNN, agriculture, edge computing