Autonomous wildfire geolocation system based on thermographic and synthetic vision techniques

Autonomous Wildfire Geolocation System based on Thermographic and Synthetic Vision Techniques Javier Pérez Mato Sistema de Geolocalización Autónoma de Incendios Forestales basado en Técnicas de Visión Sintética y Termográfica Instituto Universitario de Ciencias y Tecnologías Cibernéticas Las Palmas de Gran Canaria, Abril de 2017 T E S I S D O C T O R A L UNIVERSIDAD DE LAS PALMAS DE GRAN CANARIA INSTITUTO UNIVERSITARIO DE CIENCIAS Y TECNOLOGÍAS CIBERNÉTICAS Doctorado en Cibernética y Telecomunicación TESIS DOCTORAL AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES SISTEMA DE GEOLOCALIZACIÓN AUTÓNOMA DE INCENDIOS FORESTALES BASADO EN TÉCNICAS DE VISIÓN SINTÉTICA Y TERMOGRÁFICA Autor: Director: Director: JAVIER Dr. VÍCTOR ALEXIS Dr. FRANCISCO JOSÉ PÉREZ MATO ARAÑA PULIDO CABRERA ALMEIDA Las Palmas de Gran Canaria, Abril de 2017 UNIVERSIDAD DE LAS PALMAS DE GRAN CANARIA INSTITUTO UNIVERSITARIO DE CIENCIAS Y TECNOLOGÍAS CIBERNÉTICAS Doctorado en Cibernética y Telecomunicación TESIS DOCTORAL AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES SISTEMA DE GEOLOCALIZACIÓN AUTÓNOMA DE INCENDIOS FORESTALES BASADO EN TÉCNICAS DE VISIÓN SINTÉTICA Y TERMOGRÁFICA Autor: JAVIER PÉREZ MATO Directores: Dr. VÍCTOR ALEXIS ARAÑA PULIDO Dr. FRANCISCO JOSÉ CABRERA ALMEIDA Las Palmas de Gran Canaria, Abril de 2017 Abstract Each year thousands of wildfires devastate large areas of forest worldwide. The detection and tracking of wildfires is a complicated task, especially affected by abrupt terrain characteristics and meteorological conditions. One of the most widely used systems for wildfire detection is based on thermographic cameras installed on strategic observation points and can often be complemented by either airborne or satellite solutions. The use of thermographic cameras provides some very important advantages, such as being capable of remotely detecting hot spots that could potentially turn into a wildfire if the appropriate conditions are met. For instance, they can serve as a key preventive method especially when the 30-30-30 rule is met. This rule describes a situation where there’s an ambient temperature higher than 30◦C, a relative humidity lower than 30% and wind speed higher than 30 km/h. Under these circumstances the likelihood of wildfire outburst is quite high and its effects can be catastrophic due to the high speed winds and dry conditions. If this sort of scenario actually occurs, every possible technological advantage shall be used by firefighting teams to enable them to rapidly and efficiently coordinate their response teams and be able to control the wildfire following a safe and well-planned strategy. However, most of early detection methods for wildfires, such as the aforementioned thermographic cameras lack of a sufficient level of automation and mostly rely on human interaction, posing them to high degrees of subjectivity and latency when a high volume of data is required in real-time to correctly support decision-making scenarios during the wildfire suppression. The present thesis addresses this situation by designing and implementing a fully autonomous wildfire detection and tracking system which contains a fully automated wildfire geolocation subsystem based on synthetic vision technology. Such a tool would provide firefighting teams with a solution capable of continuously surveilling a particular area and completely autonomously identifying and providing georeferenced information of current or potential wildfires in real-time. IX To my family. List of Figures 1.1 Front cover of the February 2016 issue of the IEEE Aerospace and Electronic Systems Magazine. Source: IEEE........................................................................................... 7 2.1 Thermographic camera installed on a watchtower. Source: Sistema Bosque, Universitat Politecnica de Valencia (UPV).. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2 NASA’s Ikhana (MQ-9 Reaper) Source: NASA................................................................. 12 2.3 California 2012 wildfires detected by NASA´s Terra (EOS AM-1) satellite. Source: NASA (EOS AM-1)........................................................................................................ 14 2.4 FARSITE fire spread prediction overlaid on Google-Earth. Source: RedfishGroup.................................................................................................................................... 16 3.1 Radiation curves of a black-body radiator......................................................................... 20 3.2 Absorptivity, reflectivity and transmissivity of thermal energy. ............................... 23 3.3 Block diagram of a thermographic camera....................................................................... 24 3.4 Sources of thermal radiation and atmospheric effects. ............................................... 27 3.5 Atmospheric transmissivity at 5 km with RH varying from 10% to 70%. .......... 28 3.6 Atmospheric transmissivity simulation up to 10 km with Tamb =30◦C and RH =60%. ....................................................................................................................................... 31 4.1 Example of a Digital Elevation Model (DEM). ................................................................ 33 4.2 Synthetic terrain reconstruction on an aircraft´s glass-cockpit display............... 34 4.3 Example of ray tracing. .............................................................................................................. 35 4.4 Line-of-Sight (LOS) image generation.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.5 Contents of a GeoTIFF file........................................................................................................ 38 4.6 GeoTIFF UTM pixel coordinate determination based on the geospatial reference matrix RGEO................................................................................................................. 39 4.7 Relation between cartesian and spherical coordinate systems................................ 41 4.8 Sign convention used for the correction of magnetic declination.......................... 42 4.9 Synthetic image generation using ray-tracing techniques......................................... 43 4.10 Thermographic and synthetic image comparison.......................................................... 44 4.11 FLIR A615 thermographic camera........................................................................................ 44 4.12 Absence of hard iron and soft iron distortions................................................................ 48 4.13 Presence of hard iron distortions........................................................................................... 49 4.14 Presence of both hard iron and soft iron distortions.................................................... 49 5.1 VHF data modem.......................................................................................................................... 52 5.2 Iridium satellite. ............................................................................................................................ 53 XIX 5.3 Iridium orbital planes. ................................................................................................................ 54 5.4 Example of a typical TETRA Network Architecture...................................................... 55 5.5 Overview of the emergency data relay dispatcher......................................................... 58 5.6 Iridium SBD system architecture........................................................................................... 60 5.7 Iridium 9603 SBD transceiver. ................................................................................................ 60 5.8 SBD MO data segmentation for packets larger than 340 bytes............................... 64 5.9 Structure of a TETRA TSI identifier...................................................................................... 64 5.10 Sepura SRG3900 TETRA terminal........................................................................................ 66 5.11 Motorola TOM100 TETRA terminal..................................................................................... 67 5.12 SIMCom SIM5218 HSPA modem. ......................................................................................... 69 5.13 Development board for the SIMCom SIM5218 HSPA modem................................. 69 5.14 PPP network interface created on Linux for direct connection to a UMTS/GPRS APN......................................................................................................................... 71 6.1 Block diagram of the Rapidly Deployable Mobile Unit (RDMU)............................ 74 6.2 RDMU custom backpack containing all control, power and communication subsystems. ...................................................................................................................................... 74 6.3 RDMU backpack connected to the external equipment.............................................. 75 6.4 Visible channel and thermographic cameras. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 6.5 Pan (yaw) and tilt (pitch) servos attached to torque-multiplying gearboxes. . . 76 6.6 PWM signal used to control the servo’s angular position. ......................................... 77 6.7 Inertial Measurement Unit (IMU)......................................................................................... 78 6.8 High-precision GPS receiver..................................................................................................... 78 6.9 Example of a ($GPGGA) NMEA 0183 GPS sentence.................................................... 78 6.10 Flowchart describing the execution of the DPGU software running inside the RDMU......................................................................................................................................... 81 6.11 Power distribution board........................................................................................................... 83 6.12 FLIR Quark 640 thermographic camera............................................................................. 84 6.13 New generation of the DPGU.................................................................................................. 84 7.1 Synthetic image alignment test with known landmarks............................................. 86 7.2 Thermographic image equivalent to the synthetic image after attitude compensation.................................................................................................................................. 87 7.3 Synthetic image with large disalignment with respect to the thermographic Image.................................................................................................................................................. 88 7.4 Computed sky-terrain interface using the synthetic image....................................... 88 7.5 Binarized temperature differences image generation based on the thermographic image.................................................................................................................. 89 7.6 Pitch, roll and yaw offset determination based on sky-terrain interface. ........... 90 7.7 Final result and comparison of the thermographic and synthetic images after performing a terrain profile alignment.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 7.8 Error due to the presence of vegetation on the sky-terrain interface................... 90 7.9 Controlled burn carried out by professional firefighters. ........................................... 91 7.10 Set of images produced by the RDMU during nominal operation......................... 92 XX 7.11 Effectiveness of the terrain profile alignment process................................................. 92 7.12 Automated geolocation of the wildfire on Google-Earth............................................ 92 7.13 Final result of the RDMU in full autonomous operation. ........................................... 93 A.1 Portada de la edición de febrero de 2016 de la revista IEEE Aerospace and Electronic Systems Magazine. Fuente: IEEE......................................................................104 A.2 Cámara termográfica instalada en una torreta de observación. Fuente: Sistema Bosque, Universitat Politecnica de Valencia (UPV). .........................................106 A.3 UAV Ikhana (MQ-9 Reaper) Fuente: NASA.......................................................................108 A.4 Incendios de California en 2012 detectados por el satélite Terra (EOS AM-1) de la NASA. Fuente: NASA (EOS AM-1).. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 A.5 Predicción de la propagación de un incendio forestal utilizando FARSITE. Fuente: RedfishGroup...................................................................................................................112 A.6 Curvas de radiación de un cuerpo negro.. . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 A.7 Absortividad, reflectividad y transmisividad de la energía térmica. .....................115 A.8 Fuentes de radiación térmica y efectos atmosféricos...................................................116 A.9 Ejemplo de un Modelo Digital de Elevación del Terreno (DEM)............................118 A.10 Generación de la imagen sintética utilizando trazado de rayos (ray tracing)..119 A.11 Comparación entre imagen termográfica y sintética....................................................119 A.12 Cámara termográfica FLIR A615. ..........................................................................................120 A.13 Módem de datos VHF. .................................................................................................................122 A.14 Satélite Iridium..............................................................................................................................124 A.15 Planos orbitales de Iridium. .....................................................................................................124 A.16 Ejemplo de la arquitectura de red típica en TETRA. ....................................................126 A.17 Descripción del gestor de comunicaciones multimodo. ..............................................129 A.18 Diagrama de bloques de la RDMU........................................................................................131 A.19 Mochila modificada para implementar la RDMU. .........................................................131 A.20 Mochila RDMU conectada a los equipos externos.........................................................132 A.21 Cámaras de canal visible y termográfica.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 A.22 Servos utilizados para el movimiento del cabezal de cámaras................................134 A.23 Señal PWM utilizada para controlar la posición angular de los servos...............134 A.24 Unidad de Medición Inercial (IMU).. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 A.25 Receptor GPS de alta precisión. .............................................................................................135 A.26 Ejemplo de una trama ($GPGGA) NMEA 0183 de GPS..............................................136 A.27 Diagrama de flujo del sistema de control implementado en el DPGU. ................138 A.28 Placa de distribución de alimentación. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 A.29 Cámara termográfica FLIR Quark 640.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 141 A.30 Segunda generación de la RDMU. ........................................................................................141 A.31 Alienación de imagen sintética con referencias topográficas conocidas. ............143 A.32 Equivalencia entre imágenes sintética y termográfica después del ajuste de actitud................................................................................................................................................143 A.33 Imagen sintética con desalineación prominente. ...........................................................144 A.34 Interfaz cielo-tierra calculado en la imagen sintética. .................................................145 XXI A.35 Generación de imagen binarizada de diferencias de temperatura a partir de la imagen termográfica. .............................................................................................................146 A.36 Determinación del offset de pitch, roll y aaw a partir del interfaz cielo-tierra.146 A.37 Resultado final después de una alineación del perfil del terreno...........................147 A.38 Error debido a la presencia de vegetación en el interfaz cielo-tierra. ..................147 A.39 Quema controlada supervisada por bomberos profesionales...................................148 A.40 Conjunto de imágenes generadas por la RDMU.............................................................148 A.41 Proceso de alineación del perfil del terreno. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 A.42 Geolocalización autónoma del incendio forestal en Google-Earth. .......................149 A.43 Resultado final proporcionado por la RDMU operando de forma autónoma.. . 150 XXII List of Tables 3.1 Infrared spectrum classification............................................................................................. 21 3.2 Examples of emissivity values based on materials and surface finish.................. 22 3.3 Influence of atmospheric transmissivity on temperature determination (Tamb =30◦C, RH =60%)........................................................................................................ 31 4.1 Optical characteristics of the FLIR A615 thermographic camera. .......................... 45 5.1 SBD message structure............................................................................................................... 60 5.2 AT control messages used on the Iridium 9603 SBD transceiver............................ 61 5.3 SBD message structure............................................................................................................... 65 5.4 AT control messages used on the Sepura SRG3900 TETRA terminal................... 65 5.5 AT control messages used on the SIMCom SIM5218 HSPA modem. .................... 70 6.1 Field decoding of a $GPGGA NMEA 0183 sentence ..................................................... 79 6.2 Power consumption of the RDMU components.............................................................. 82 A.1 Influencia de la transmisividad atmosférica en la determinación de la temperatura (Tamb =30◦C, RH=60%).. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 A.2 Características ópticas de la cámara termográfica FLIR A615.................................120 A.3 Decodificación de una trama $GPGGA NMEA 0183. ...................................................136 A.4 Consumo de potencia de los elementos de la RDMU...................................................139 XXIII Glossary A AHRS Attitude and Heading Reference System APN Access Point Name ASSI Alias Short Subscriber Identity AT ATtention Command B BS Base Station C COTS Commercial Off-The-Shelf D DEM Digital Elevation Model DGPS Differential Global Positioning System DMR Digital Mobile Radio DPGU Data Processing and Georeferencing Unit DSL Digital Subscriber Line DXT TETRA Digital Exchange E EOS Earth Observation Satellite ESA European Space Agency ETSI European Telecommunications Standards Institute F FIR Far-Infrared FOV Field Of View FPA Focal Plane Array FSK Frequency-Shift Keying FTP File Transfer Protocol G GEO Geostationary Earth Orbit GIS Geographical Information System GPRS General Packet Radio Service GPS Global Positioning System GPU Graphics Processing Unit GSM Global System for Mobile Communications GSSI Group Short Subscriber Identity H HFOV Horizontal Field of View HSPA High Speed Packet Access HTTP Hypertext Transfer Protocol I IDeTIC Instituto para el Desarrollo Tecnológico y la Innovación en Comunicaciones IE Information Element IEEE Institute of Electrical and Electronics Engineers IFOV Instantaneous Field Of View IMEI International Mobile Equipment Identity IMU Inertial Measurement Unit IP Internet Protocol IR Infrared Radiation ISSI Individual Short Subscriber Identity L LEO Low Earth Orbit LIDAR Light Detection and Ranging LOS Line of Sight LS Link Service LWIR Long-Wavelength Infrared LZW Lempel-Ziv-Welch M XXV MCC Mobile Country Code MNC Mobile Network Code MO Mobile Originated MSL Mean Sea Level MT Mobile Terminated MWIR Medium-Wavelength Infrared N NASA National Aeronautics and Space Administration NIR Near-Infrared NMEA National Marine Electronics Association P PAMR Personal Access Mobile Radio PDP Packet Data Protocol PIN Personal Identification Number PMR Personal Mobile Radio PPP Point to Point Protocol PWM Pulse Width Modulation Q QWIP Quantum Well Infrared Photon R RDMU Rapidly-Deployable Mobile Unit RESCAN Red de Emergencias y Seguridad de Canarias RF Radio Frequency RH Relative Humidity RLE Run Length Encoding S SAR Synthetic Aperture Radar SBD Short Burst Data SDS Short Data Service SIM Subscriber Identity Module SMS Short Message Service SS Subscriber Station SSD Solid-State Drive SSH Secure Shell SSI Short Subscriber Identity SWIR Short-Wavelength Infrared T TC Telecommand TCP Transmission Control Protocol TDMA Time Division Multiple Access TETRA Terrestrial Trunked Radio TIFF Tagged Image File Format TL Transport Layer TM Telemetry TSI TETRA Subscriber Identity U UART Universal Asynchronous Receiver-Transmitter UAV Unmanned Aerial Vehicle UDP User Datagram Protocol ULPGC Universidad de Las Palmas de Gran Canaria UMTS Universal Mobile Telecommunications System USB Universal Serial Bus UTC Coordinated Universal Time UTM Universal Transverse Mercator V VDC Volts Direct Current VFOV Vertical Field of View VHF Very High Frequency VSAT Very Small Aperture Terminal W WGS World Geodetic System WMM World Magnetic Model WSN Wireless Sensor Network XXVI INTRODUCTION Chapter 1 Every year, thousands of wildfires devastate large extensions of forest around the world, causing the loss of human life and incurring costs of several hundred million Euros only in the last ten years (Anual Fire Reports, Forest Fires in Europe, Middle East and North Africa, 2015). This disaster also has a direct consequence in the reduction of the forest biomass, causing a severe damage to the Earth’s forest ecosystem. Being able to detect and track the behaviour of wildfires is vital in order to rapidly and efficiently organise the available resources to control and extinguish them as fast as possible. This task can be severely affected in areas of abrupt terrain, which not only difficult the visual detection and surveillance of wildfire fronts or hot spots, but also account for some very localised winds and meteorological conditions affecting the prediction of the wildfire evolution (Sanjuan, Brun, Margalef, & Cortes, 2014). A good example of this type of terrain are the Canary Islands in Spain, which is where the research project behind this thesis has been conducted (SeLiF: Development of New Techniques for the Detection and Tracking of Wildfires based upon Rapidly-Deployable Mobile Thermal Sensors, 2013). There are several key technologies and methods commonly used for wildfire detection, surveillance and prediction. Some of these methods include fixed and mobile ground based solutions, aerial platforms and satellite imaging or sensing. However, most of them present specific limitations that can affect both their performance and the usefulness of the generated information. Moreover, algorithms used to predict wildfire spread tend to rely on accurate near real-time input data to maximise their reliability, but unfortunately the majority of the previously described technologies are not able to provide this data either fast or accurately enough (Duff & Tolhurst, 2015). Another important issue relates to the geolocation of the wildfire itself. Most of the ground and airborne solutions only provide images of the fire, captured either by thermographic or visual channel cameras and leave the wildfire geolocation task to the 1 WILDFIRE DETECTION AND PREDICTION SYSTEMS Chapter 2 This chapter provides an overview of the most widely used wildfire detection systems, which can also be employed as tracking tools in case a wildfire has already started. Several of their main advantages and limitations are discussed in each case, as well as finally presenting a short description on how they can be combined with wildfire behaviour prediction techniques. 2.1 Fixed and Mobile Optical Ground Detection Most of ground based detection techniques are either based on direct observations made on the field by wildfire surveillance staff or by optical systems, either in visible channel or using thermographic cameras. Visible channel systems usually rely on conventional high-resolution cameras which try to detect the early occurrence of smoke or flames. There are several algorithms (Labati, Genovese, Piuri, & Scotti, 2013; Dimitropoulos, Barmpoutis, & Grammalidis, 2015) which are used over digitised visible channel images in order to detect a wildfire during its early stages of development. On the other hand, solutions based thermographic cameras usually scan a specific area of forest and try to detect hot spots laying above a specific and predefined temperature threshold in order to determine the possible occurrence of a wildfire. This second type of wildfire optical detection solutions will be the main focus of this thesis. Fixed ground detection systems are usually composed of cameras fixed on a pole attached to watchtowers or other similar infrastructures (Sistema Bosque, 2011), which provide the necessary power and communication interfaces. These cameras are usually mounted on gimbal bases allowing the camera to pan and tilt over a certain range. 9 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES The most common and useful type of camera used is a thermographic imaging camera in the near-infrared spectrum (NIR) as it’s able to detect subtle changes in temperature to precisely determine the presence of fire. With the appropriate combination of infrared-transparent lenses and sensors, mainly consisting of microbolometers, these cameras are able to detect fire on ranges of up to several kilometres, even through dense smoke or during night time. In order be able to surveil large extensions of forest, several of these observation points need to be installed (FireWatch: Early Detection of Forest Fires for Large Areas, 2014), ideally accounting for the field of view of each individual camera and considering their individual pan and tilt ranges, as well as the presence of any surrounding obstacles, such as valleys or hills. An example of watchtower with a thermographic camera can be seen in Figure 2.1. FIGURE 2.1. Thermographic camera installed on a watchtower. Source: Sistema Bosque, Universitat Politecnica de Valencia (UPV). This type of ground-based detection presents some specific limitations (Ahmad & Alkhalib, 2014) described hereafter: • Cameras are permanently exposed to inclement weather conditions, including high levels of humidity, rain and dust. This demands a very regular maintenance in order to keep the cameras in good working order. Also they are prone to vandalism for the fact of being exposed 24/7 all year round, even when there’s no risk of fire. • Each of the observation points has a limited field of view. This demands a large number of observation points in order to cover a large area. Furthermore, because of being fixed points attached to infrastructures they cannot be relocated dynamically if the fire falls out of the field of view, leaving some observation points useless. • The georeferencing of the detected fire is performed manually by firefighting staff by comparing the thermographic image with specific cartography of the area. This 10 2 Wildfire Detection and Prediction Systems tasks are usually complemented by the use of Geographical Information Systems (GIS), but they are exposed to human error and a certain degree of subjectivity. • The costs of maintaining each observation point is usually quite high and requires several teams of skilled staff to perform maintenance tasks. A complementary technique for ground based detection relies on the use of vehicles with thermographic cameras installed on extensible poles. This increases the mobility and reconfigurability of the observation points, as well as reducing the possibility of damage caused by both weather and vandalism. However, they still suffer from the same subjectivity with regards to wildfire georeferencing and poses a high risk for the vehicle operators. Two of the most widely implemented commercial fixed ground detection systems are FireWatch (FireWatch: Early Detection of Forest Fires for Large Areas, 2014) and ForestWatch (ForestWatch Wildfire Detection, 2015). Both systems have gone through extensive development phases and are currently being used in many countries around the world. However, both of these systems have been subject to important limitations with respect to their performance as described in (Ahmad & Alkhalib, 2014). The detection performance of both systems was determined to be partially a function of the size and distance between the wildfire and the camera, where large fires were detected at long distances of 70 km or more, but moderate fires at distances of 10 or 20 km where missed. They also present some localisation error problems, where the system is unable to use topography maps of the terrain as an input for wildfire geolocation purposes. This is particularly one of the main issues that will be addressed within this thesis. Optical sensors in general, especially the visible channel ones, are also prone to a high level of false alarms due to the various dynamic effects they witness such as tree movement due to wind, cloud shadows, reflections and human activity. These difficulties in processing forest imagery are mainly derived from the varying nature of forest landscape and the dynamic events that may occur under different environmental conditions, such as weather, distance, sun angle, interfering objects and many more. Some proposals have been made to implement either mobile or fixed sensor networks to detect the occurrence of wildfires (Kremens, Faulring, Gallagher, Seema, & Vodacek, 2003; Fernandez-Berni, Carmona-Galan, Martinez-Carmona, & Rodriguez-Vazquez, 2012), but they usually lack a certain degree of dynamic reconfigurability and easiness of use. 2.2 Aerial Detection Aerial detection might seem like a plausible alternative to overcome some of the limitations observed in ground based detection. Manned helicopters and aircraft are often used during wildfires for both firefighting and monitoring tasks, as they have both the capability of spraying water over the fire and also serve as high altitude observation points. 11 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES Most of these helicopters and aircraft are equipped with advanced electronics and communications subsystems, used mainly for the supervision and tactical information relay of observed data to firefighters. This can include GPS units, multi-spectral cameras and cellular data connectivity among other. Aircraft flying over a wildfire area provide a much larger field of view and can be dynamically moved from one place to another as the wildfire evolves. This makes them much more versatile and efficient than ground based solutions, but they are still prone to some important limitations. On the other hand, the crew of the aircraft is exposed to significant risk due to the proximity to the active fire. Convective currents formed in these situations and bad weather may also generate local turbulence and severely affect the aircraft’s stability. These circumstances can be life-threatening and have been a direct consequence of many accidents in the past (Aerial Accidents Report in Wildland Firefighting in Spain, 2015). As an alternative, Unmanned Aerial Vehicles (UAVs) are starting to be widely used for monitoring tasks under riskful situations (Beard, McLain, Nelson, Kingston, & Johanson, 2006). By being remotely or autonomously operated they eliminate the need of having a crew on board. Figure 2.2 shows NASA’s Ikhana (MQ-9 Reaper) UAV platform, which is currently being used as a tool for early wildfire detection and tracking in remote locations (Ambrosia & Hinkley, 2008). This UAV is equipped with thermographic imaging cameras and can fly at 15,000 m for more than 20 hours. FIGURE 2.2. NASA’s Ikhana (MQ-9 Reaper) Source: NASA. However, there is concern about how to address a loss of control if they are sharing the same airspace with manned aircraft. In case a UAV looses control it could crash against another nearby aircraft and cause a disastrous situation. Currently regulations with respect to non-segregated airspace have not been completely deployed and this kind of situations are not allowed, although there is progress being made to implement this type of scenario (Ramalingam, Kalawsky, & Noonan, 2011). Other typical limitations of commercially available UAVs are their limited autonomy, low payload capacity and their limited ability to withstand strong winds or turbulence. 12 2 Wildfire Detection and Prediction Systems 2.3 Non-Optical Wireless Sensor Networks Detection Instead of using images to detect the early occurrence of wildfires a new opportunity is currently emerging based on the use of Wireless Sensor Networks (WSNs), such as the approaches described in (Manatakis & Manolakos, 2011) and (Yoon et al., 2012). These are individual units with both sensing and communication subsystems embedded on the same device and can measure physical magnitudes like temperature, pressure and humidity, as well as chemical parameters like carbon dioxide and nitrogen dioxide levels. Most of these sensors or nodes can operate autonomously in a self-organising environment (Gasull, Larios, Barbancho, León, & Obaidat, 2011). One of the key technologies used for WSNs is ZigBee (802.15.4 IEEE Standard for Low Data Rate Wireless Networks, 2005), which is based on the IEEE 802.15.4 standard (802.15.4 IEEE Standard for Low Data Rate Wireless Networks, 2005). This technology uses low cost battery powered devices and in most cases can even be powered by small solar panels. They provide a low data rate communication to a centralised server capable of processing the measured variables and performing some further computations to trigger alarms under predefined situations. This solution has the potential to be applied in many different scenarios. Each of the nodes needs to be sufficiently inexpensive so that a large number of them can be deployed over extense areas. This is linked to the fact that their operating principle relies on the close observation of a particular spot instead of perceiving an overall vision of a large extension of wildland, as it is usually the case with optical ground detection systems. It is also quite possible that some of the nodes can get damaged by the wildfire itself, so a loss of sensing elements must be accounted for and used to calculate worst case scenarios. In order to be able to use this type of sensors as an early wildfire detection mechanism they must also be able to provide high levels of reactivity, reliability and robustness. To achieve this characteristics, both the unit itself and the telecommunication infrastructure behind it must be well designed and leave room for both escalability and redundancy. This also means that each sensor is not fully autonomous by itself, but it rather relies on a central node acting as an information relay or on a dedicated telecommunications infrastructure. Their easiness of replication and therefore extensive deployment permits having many units installed on extensive areas of wildland. However, this adds a high complexity to their maintenance tasks. Most of the sensors will be located in remote or poorly-accessible locations, requiring an important degree of manpower to be able to provide periodical maintenance such as replacing their batteries. They also present the same level of exposure to vandalism as other static solutions. 2.4 Satellite-Based Remote Sensing Wildfire tracking using satellite imagery and multi-spectral sensing has been often employed when large extensions of land or forest are affected by a severe fire. Several 13 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES satellites currently in orbit have special equipment to serve for this purpose (Ganesan, Sathish, & Sajiv, 2016; Veraverbeke et al., 2014). Some examples of such satellite families include the the Global Monitoring for Environment and Security (GMES) from ESA or the Moderate Resolution Imaging Spectroradiometer (MODIS) from NASA. Unfortunately these satellites can only provide images of the regions of interest every two days approximately, besides the quality of satellite images can also be affected by local weather conditions (Mitsopoulos, Mallinis, Paschalidou, & Emmanouloudis, 2016). Figure 2.3 shows a satellite image from the California wildfires occurred in 2012 obtained by NASA’s Terra (EOS AM-1) satellite using its MODIS instrument. FIGURE 2.3. California 2012 wildfires detected by NASA´s Terra (EOS AM-1) satellite. Source: NASA (EOS AM-1). Most of existing satellite-based wildfire observation and tracking solutions present some key limitations that directly affect the effectiveness in controlling forest areas against wildfires. Most of the these limitations are based on the fact that Earth observation satellites are usually located in Geostationary Earth Orbits (GEO) or Low Earth Orbits (LEO) which means that they can be at an altitude of more than 35,000 km above the Earth’s surface. Under these circumstances, optical and infrared radiation emitted by wildfires in their early stages can end up being too weak to be correctly detected by a satellite. Taking into account that the intensity of the radiated thermal energy decreases dramatically in terms of distance, the angle between the normal direction of the satellite’s sensing instrument and the radiation beam becomes critical. Even the slightest mismatch in a satellite’s position or orientation can affect the detection of a forest wildfire, specially during its early phase. Another well known drawback associated with satellite-based remote sensing is the low level of immediacy with respect to image acquisition. Satellite images usually take some time to be processed and are not typically available until several hours after being acquired, this makes them a very limited tool where rapid decision-making is necessary. 14 2 Wildfire Detection and Prediction Systems Satellite re-visit periods can also be quite long depending on the satellite’s orbit. This means that the satellite will only pass over the area of interest a couple of times per day. This low periodicity is a limiting factor in order to properly keep track of the wildfire’s evolution over time. Also, a satellite located on a GEO orbit will permanently cover a specific area of land, but can’t be used to track other areas of the Earth. A satellite being able to cover several areas of the Earth in terms of monitoring wildfires would imply the collaboration between several official agencies or entities across different countries, which is usually a very challenging political task. Usually this type of detection is only useful for very large wildfires, as satellites aren’t able to provide any useful information for smaller fires. They are also used as a post-fire analysis tool in order to evaluate the overall impact of the wildfire on a large extent of land. However, they do provide automatic georeferencing capabilities, but the spatial resolution provided is not precise enough to serve as a strategy-planning tool for firefighting. 2.5 Behaviour Prediction There are several systems which allow wildfire behaviour prediction by modelling the fire spread over a specific area (Papadopoulos & Pavlidou, 2011; Filippi, Mallet, & Nader, 2014). This is usually achieved by using known fire spots as inputs to the core algorithms used by prediction software. In order for these algorithms to perform correctly, both the input data and several other meteorological variables should be updated on a regular basis, as they are used to model and estimate the possible propagation of fire during a certain time span. Wildfire spread is a spatial phenomenon that depends largely on multiple parameters. These parameters include not only weather conditions but also geographical and topographical information regarding the topology of the area where they are being applied. This is even more complicated if we take into consideration the fact that weather prediction is usually prone to errors, especially when there are extreme meteorological conditions. Based on these challenges there have been a series of international projects based on developing reliable wildfire spread simulators. One of this projects is Prometheus (Tymstra, Bryce, Wottn, & Armitage, 2009) which is a Canadian open source software used for deterministic fire growth simulation. It uses topographical data and fuel types along with weather information applied to the area of interest. The model implemented simulates fire growth using a principle of wave propagation, where the fire front itself is considered a wave front. However, most of the data used as an input by Prometheus remains static during the simulation process. There’s an alternative system called Phoenix (Tolhurst, Shields, & Chong, 2008) developed in Australia that can dynamically respond to changes in topographical, fuel and environmental conditions as the fire moves along the landscape, therefore providing a more realistic and accurate simulation. Phoenix works by dividing the terrain into a grid, where each cell can be as small as 5 m in order to provide a very detailed analysis, but this can impose high computational power requirements and act as a detriment in real-time environments. A grid size of between 100 m and 200 m is typically considered 15 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES acceptable for most situations, but can still fall far from a real-time scenario. Usually the interval between two contiguous determinations of the fire spread can last between 1 and 15 minutes for slow moving fires, which renders the system really limited for rapid decision-making scenarios. In spite of the previous limitations, there are many other simulators and models currently being developed. However, one of them has gained the recognition of many government agencies worldwide and is currently considered as the best model for predicting wildfire growth and behaviour. This system has been developed by the US and known as FARSITE (Finney, 1995; FARSITE: Fire Area Simulator-Model Development and Evaluation, 1998). It takes some design principles from both Phoenix and Prometheus, especially regarding the wave propagation approach analysis, but is able to handle much more complex scenarios such as wildfires with multiple fire fronts and rapid-changing scenarios due to wind direction changes or transitions of fuel or terrain type. Figure 2.4 shows a graphical representation of a fire spread as computed by FARSITE. FIGURE 2.4. FARSITE fire spread prediction overlaid on Google-Earth. Source: RedfishGroup. In order for FARSITE to operate to the best of its performance it relies on a series of fuel and vegetation maps which need to be applied as inputs to the system and correctly georeferenced with other terrain and topological characteristics. This is usually a difficult task, since most governments lack from precise information about these type of terrain details, or at the level of resolution required by FARSITE. The main problem associated with most of the previously described systems is that the generated predictions tend to be very inaccurate if the area where the wildfire is occurring has a steep gradient of meteorological variables. This usually happens on areas of abrupt terrain, or if only limited or inaccurate topographical information is available. Furthermore, these systems are usually not constantly and periodically fed with new and updated data containing known active fire locations or environmental parameters, making them prone to eventually diverge and produce barely usable predictions. Depending on geographical and environmental conditions model-based predictions can be inaccurate (Alexander & Cruz, 2013). If an erroneous prediction is used as an input during firefighting tasks it could lead to wrong and riskful decisions being made, potentially leading to a disastrous situation. Typically the input data is provided by firefighting staff that visually identifies the 16 2 Wildfire Detection and Prediction Systems location of the spots and manually inputs this data into the prediction software. As previously discussed this can add a subjectivity factor, as well as being a time-consuming and risky task for the staff involved. The RDMU system developed as part of this thesis can also serve as a source of real-time input data for prediction software. This would allow a more precise and rapid access to an automatically georeferenced representation of the wildfire and therefore inherently helping to improve the accuracy and convergence speed of the model-based prediction. 17 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES 3.3 Thermographic Cameras Although infrared radiation is not detectable by the human eye, an infrared camera can convert it to a visual image that depicts thermal variations across an object or scene. As previously outlined, infrared radiation covers a portion of the electromagnetic spectrum ranging from approximately 900 to 14,000 nanometres. Thermography is a type of imaging that is accomplished with an infrared camera calibrated to display temperature values across an object or scene. Therefore, thermography allows making non-contact measurements of an object’s temperature. An infrared camera construction is similar to a digital video camera. The main components are a lens that focuses infrared radiation onto a detector, plus electronics and software for processing and displaying signals and images. Instead of a charge coupled device that video and digital still cameras use, the infrared camera detector is a Focal Plane Array (FPA) of micrometer size pixels made of various materials sensitive to IR wavelengths. Figure 3.3 shows the typical bock diagram of a thermographic camera. Video Processing Electronics Detector User Interface Lens IR Radiation FIGURE 3.3. Block diagram of a thermographic camera. FPA resolution can range from about 160 ×120 pixels up to 1024 ×1024 pixels. Certain IR cameras have built-in software that allows the user to focus on specific areas of the FPA and calculate the temperature. Other systems use a computer or data system with specialised software that provides temperature analysis. Both methods can supply temperature analysis with ±0.01◦C precision or even better. An infrared detector acts as a transducer that converts thermal radiation into an electrical signal. This type of transducers are the core element of any piece of thermography equipment. An array of these transducers can be arranged in an FPA and serves as the image formation mechanism of a thermographic camera. The number of detector elements on the FPA determines the resolution of the thermographic image (number of pixels) and there are mainly two types of detector elements: quantum detectors, this ones usually need to be cooled, and microbolometers which don’t require thermal control. The most common type of thermal detector (Rogalski, 2012) is an uncooled microbolometer made of a metal or semiconductor material. These detectors typically have lower cost and a broader IR spectral response than quantum detectors. Microbolometers react to incident radiant energy, but are much slower and less sensitive than quantum detectors. Quantum detectors are made from materials such as InSb, InGaAs, PtSi, HgCdTe, and layered GaAs/AlGaAs for QWIP (Quantum Well Infrared Photon) detectors. The operation of a quantum detector is based on the change of state of electrons in a crystal structure reacting to incident photons. These detectors 24 3 Thermography are generally faster and more sensitive than thermal detectors. However, they require cooling, sometimes down to cryogenic temperatures using liquid nitrogen or a small Stirling cycle refrigerator unit. Quantum detectors are generally considered photodetectors mounted on a substrate that generates an electrical output. However, microbolometers act as thermometers. When a photon hits a microbolometerit suffers a change in temperature that modifies the conductivity of the material. This can be subsequently translated into an electric signal by flowing a reference current through the microbolometer. Uncooled microbolometers simplified the construction and operation of thermographic cameras, making them a much more widely available tool both for professional and research purposes. Regardless of the type of detector used in a thermographic camera, all of them generate an electric signal which is equivalent to the amount of thermal radiation received. If we consider this thermal radiation completely isolated from the environment, it will ultimately depend on the surface temperature of the object being monitored. However, in its most generic form, this dependency can be a complicated function of the wavelength, temperature, angle between the camera and radiation source and all associated atmospheric effects. It is usually considered that an average emissivity can be assumed independently of the radiated wavelength. 3.3.1 Influence of the Detector Bandwidth The frequency response presented by a particular thermographic detector is an important aspect that needs to be accounted for during the calibration process. Based on the black body radiator theory described at the beginning of this chapter, a given object will emit thermal radiation simultaneously over a series of wavelengths. This means that if a given detector is tuned to a specific infrared spectrum bandwidth, such as LWIR (8µm - 15µm), it will not detect the thermal radiation emitted by an object that falls out of this bandwidth and therefore incur on an error on the determination of its temperature. Depending on the object’s temperature, the peak of energy perceived by the detector will be concentrated mostly around a series of wavelengths within the detectable range, but these will shift with temperature variations and could fall out of the detectable range. A compensation factor that accounts for the amount of non-detected thermal radiation should be applied in specific cases. However, the frequency response of microbolometer detectors is mostly flat across the IR frequency range (Bhan, Saxena, Jalwania, & Lomash, 2009), so it will not impose a high distortion over the curves described by Wien’s displacement law in section 3.1. If we take into account the current application of detecting wildfires this factor is not critical, as the temperature levels targeted provide the peak thermal radiation frequency components well centred within the LWIR bandwidth and therefore the non-detected components are reasonably low as to pose high temperature interpretation errors. 25 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES 3.4 Sources of Thermal Radiation and Atmospheric Effects The scenario previously described was initially quite simple, however the source of radiation can’t be considered as coming from a single source. Other nearby objects and the atmosphere itself can also act as sources of thermal radiation. The amount of radiation that is received by the thermographic camera lens comes mainly from three different sources (Advanced Radiometry, 2013). The camera receives radiation from the target object, plus radiation from its surroundings that has been reflected onto the object’s surface. Both of these radiation components become attenuated when they pass through the atmosphere. Since the atmosphere absorbs part of the radiation, it will also radiate some itself according to Kirchhoff’s law. Given this situation, we can derive a formula for the calculation of an opaque object’s temperature from a calibrated camera’s output. The atmosphere is obviously non-opaque and therefore has a transmissivity component associated to it. 1. Emission from the object =ε·τ·Wob j, where εis the emissivity of the object and τis the transmissivity of the atmosphere. 2. Reflected emission from ambient sources =(1−ε)·τ·Wamb, where (1−ε)is the reflectance of the object. It is assumed that the temperature Tamb is the same for all emitting surfaces within the half sphere seen from a point on the object’s surface. 3. Emission from the atmosphere =(1−τ)·Watm, where (1−τ)is the emissivity of the atmosphere. The total radiation power received by the camera can now be written as shown in Equation (3.3), where εis the object emissivity, τis the transmission through the atmosphere, Tamb is the (effective) temperature of the object’s surroundings, or the reflected ambient temperature, and Tatm is the temperature of the atmosphere. Wcam = (1−τ)·Wob j + (1−ε)·τ·Wamb + (1−τ)·Watm (3.3) This equation can be further expanded in order to account to other elements which could affect the total radiation power perceived by a thermographic camera. Figure 3.4 shows an example where the effect of the camera lens, or window, has also been considered. These parameters need to be known or correctly estimated in order for a thermographic camera to be able to correctly determine the temperature of a given object. Of all the parameters previously described, the effect of the atmosphere is the one that plays the most important role in terms of a temperature measurement being erroneously interpreted by the camera, as it depends on the distance to the heat source. Being able to precisely determine the distance between the thermographic camera and the location of possible hot spots is key in order to achieve a correct temperature measurement. Most of the existing ground-based wildfire detection systems apply a generic distance calibration to the whole thermographic image, but the system proposed 26 3 Thermography FIGURE 3.4. Sources of thermal radiation and atmospheric effects. in this thesis will implement a pixel-by-pixel calibration approach, rendering much more detailed thermographic images capable of highlighting very subtle temperature differences. The process to obtain this pixel-by-pixel distance calibration is part of of the synthetic vision algorithm implemented in Chapter 4. 3.4.1 Atmospheric Transmissivity Modelling As discussed in the previous section, being able to correctly model the atmospheric transmissivity is one of the most important parameters needed to achieve a correct thermographic measurement at any given distance. Due to its direct relation with the distance between the thermographic camera and the observed location, every effort needs to be made to compute its value in the most representative way possible. Correctly measured atmospheric transmissivity allows the correct temperature of an object to be determined. The attenuation introduced by the atmosphere also depends on the wavelength. For some wavelengths there’s very little attenuation over several kilometres, whereas other wavelengths can be completely attenuated over a few meters. This is of special interest within the detectable LWIR bandwidth common to most thermographic cameras. There are several models for atmospheric transmissivity such as FASCODE, MODTRAN and SENTRAN (Isaacs, Wang, Worsham, & Goldenberg, 1987; Theriault et al., 1994; Pregowski, 2001; Pregowski & Swiderski, 1996; Payne & Schroeder, 2013). However, they mostly rely on a set of empirical variables which are quite difficult to determine and don’t fully account for the absorptivity of gaseous elements such as water vapour and carbon dioxide. A much more precise model is described in (Minkina & Klecha, 2016) using Passman-Larmore tables and has been the one selected for this thesis. Atmospheric transmissivity depends on several factors but mainly in relative humidity (RH), atmospheric temperature, distance and wavelength. This dependency is reflected on the Passman-Larmore tables by providing the absorption coefficients for water vapour and carbon dioxide. These absorption coefficients relate to the atmospheric transmissivity as described in (3.4). τatm ∼ =τH2O·τCO2(3.4) 27 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES Figure 3.5 shows a family of curves for atmospheric transmissivity over a distance variation ranging from 0 to 5 km and for different values of relative humidity and at a constant temperature of 30◦C. Even if the relative humidity and atmospheric temperature can be determined precisely enough, the relation of the atmospheric transmissivity τatmwith the distance remains critical. Over a distance of 5 km there’s a variation of τatm between 1 and 0.2, which could induce in temperature measurement errors of at least 500% in relation to the real temperature. Distance (km) 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Atmospheric Transsmisivity 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1Temperature = 30.00 ºC RH = 10% RH = 20% RH = 30% RH = 40% RH = 50% RH = 60% RH = 70% FIGURE 3.5. Atmospheric transmissivity at 5 km with RH varying from 10% to 70%. 3.5 Optical Considerations for Wildfire Detection Assuming that the optics of the thermographic camera do not introduce any distortions, or that these are negligible in a far-field situation, then the minimum possible error that the georeferencing system could have at a given distance in terms of being able to detect a wildfire would depend on the camera’s resolution, sensor size and focal distance. It must be clarified that in this particular section we are discussing how the camera’s optical parameters affect the detection of a wildfire based on its temperature and not about the precision in the geolocation process. A thermographic camera’s Field of View (FOV) is determined by its lens. In the scope of this project, a thermographic camera with a focal length of 24.6 mm was used, providing a FOV of 25◦x 18.8◦. However, there’s another parameter much more interesting than the FOV that will allow us to determine the minimum operating distance of the thermographic camera. This parameter is called the Instantaneous Field of View (IFOV) and accounts for the FOV covered by each individual pixel. The IFOV of a given camera is determined by both its FOV, which as we said before is inherently linked to its optics, 28 3 Thermography and also by its resolution. For this particular case, the used camera had a resolution of 640 x 480 pixels, providing an IFOV of 0.683 mrad. The area of terrain covered by each individual pixel is therefore determined by the IFOV and the geographical distance from the camera to the target. This can be determined by using equation (3.5), where Dpix stands for the distance between a particular pixel and the target in meters, Apix for the area of terrain covered by a single pixel and αI FOV for the IFOV angle in radians. If we assume that we would like each individual pixel to be able to detect an area of fire of 1m2, then the thermographic camera should be at a maximum distance of 1,464 m from the wildfire. Apix = (Dpix ·sin(αI FOV ))2(3.5) However, in most cases being able to successfully detect a wildfire or hotspot based on the information given by a single pixel is not enough and can lead to important errors. In this case Johnson’s criteria (Sjaardema, Smith, & Birch, 2015) determines that at least 1.5 pixels should be used to detect any feature, meaning that we will need to effectively multiply by a factor of 1.5x our IFOV. This affects the maximum operational distance of the thermographic camera, and using the same example as before the camera will need to be located at a maximum distance of 976 m to be able to detect a hot spot of 1m2 based on Johnson’s criteria. In general, this means that the maximum operational distance (Dpixmax ) of the thermographic camera can be determined by the area of the smallest detectable feature desired and therefore could be expressed as shown in equation (3.7). Dpixmax =ÆAf eature sin(αI FOV )(3.6) In this particular case it must be noted that we are assuming that the burnt area is completely perpendicular to the thermographic camera. However, if the terrain is at a slanted angle, then the maximum operational distance must be corrected by applying the following corrective factor. The angle αinc is considered the incident angle and for the fully-perpendicular case this angle would have a value of 90◦. Dpixmax =ÆAf eature sin(αI FOV )·sin(αinc)(3.7) 3.5.1 Distance to the Wildfire and Temperature Threshold Determination If the synthetic image is optically equivalent to the thermographic image on a pixel-to-pixel basis it’s then possible to determine the temperature of each pixel and also its geographical coordinates. The temperature value for any given pixel is directly related to the distance (Dpi x ) between the camera and the geographical location of the pixel, which can be computed by using equation (3.8) in UTM coordinates. This process will be covered in more detail in Chapter 4. XU T Mcam,YU T M cam and Zcam correspond to the 29 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES UTM latitude, longitude and altitude values of the thermographic camera and XU T M pix , YU T M pix and Zpix to the UTM latitude, longitude and altitude of each individual pixel on the synthetic image. Dpix =Ç(XU T M pix −XUT M cam)2+ (YU T M pi x YU T Mcam)2+ (Zpix −Zcam)2(3.8) The computed distance (Dpix ) from the camera to each individual pixel will allow to perform a precise depth-mapping of the thermographic image and therefore achieving a much more accurate temperature determination. Chapter 4 describes how the synthetic image is created using ray-tracing techniques based on the thermographic camera’s location and attitude and how it should be optically equivalent to the thermographic image. 3.5.2 False Alarms and False Negatives An object’s emissivity is one of the most important parameters when trying to correctly calculate its temperature by the method described in the previous section. The emissivity of an object can vary between 0 and 1 and for the particular case of forest vegetation it usually lies at approximately 0.8. This value however will become closer to 1 if the vegetation is on fire, as burnt wood presents a higher emissivity. Another important parameter that has a direct relation with the distance between the camera and the wildfire is the atmosphere’s transmissivity. This parameter depends on the atmosphere’s temperature and relative humidity and can vary between 0.2 and 1 for distances of up to 5 km. As the distance increases, the atmosphere’s transmissivity decreases. In the particular case of a generated thermographic and synthetic images for a given terrain, the variation in distance between the camera and the location of the individual pixels can be very large across the image, so applying a uniform distance computation to the whole image is not recommended. This is a key difference with respect to other fire detection systems, which generally apply a uniform value for the atmospheric transmissivity to the whole image and could yield erroneous temperature computations on specific areas of the image. Let’s assume that we select a fire detection threshold of 200◦C and that it will be used as a decision criteria to determine the presence or not of a fire. Depending on the distance between the camera and the object being measured we could be committing an error in the determination of the object’s temperature due to the effect of atmospheric transmissivity. For instance, if a given object is at a temperature of 100◦C and the distance from the camera to the object is small (high atmospheric transmissivity), then the object’s temperature could end up being detected as higher than than 200◦C and this could generate a false alarm. However, if we assume that the temperature of the object is now 300◦C and the camera is further away (low atmospheric transmissivity), then the object’s temperature would be detected as being lower than 200◦C and therefore producing a dangerous situation where the wildfire remains undetected. A simulation (Figure 3.6) has been performed based on the previous example to demonstrate the effect of atmospheric transmissivity on temperature determination. 30 3 Thermography Table 3.3 shows the results of this simulation. For each case, an object temperature Tob j of either 100◦C or 300◦C has been assumed. The atmospheric transmissivity has then been varied intentionally to demonstrate how the detected temperature actually changes depending on the distance. This shows that an erroneous computation of atmospheric transmissivity would yield large errors on temperature determination. The cases where there could have been either a false alarm or false negative have been highlighted in orange and red respectively. Distance (km) 012345678910 Atmospheric Transsmisivity 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1Temperature = 30.00 ºC, RH = 60% FIGURE 3.6. Atmospheric transmissivity simulation up to 10 km with Tamb =30◦C and RH =60%. TABLE 3.3. Influence of atmospheric transmissivity on temperature determination (Tamb =30◦C, RH =60%). Tob j τatm Equivalent distance Tcam 100◦C 0.2 5 km 100.0◦C 100◦C 0.5 1.8 km 161.8◦C 100◦C 0.8 0.4 km 204.9◦C 300◦C 0.2 5 km 300.0◦C 300◦C 0.1 8.3 km 217.9◦C 300◦C 0.08 9.2 km 195.5◦C Atmospheric transmissivity curves are non-linear and they become more steep for higher values of atmospheric transmissivity, resembling a saturation-style response. These curves also tend to diverge between them for lower values of atmospheric transmissivity. This means that trying to compute the atmospheric transmissivity for 31 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES distant objects (lower values of τatm) becomes critical, as any subtle variation on the determination of τatm could very likely lead to a false negative and therefore a possible wildfire not being detected. False negatives will always be more dangerous than false alarms. In order to prevent such high errors and riskful situations, the distance from the thermographic camera to each individual pixel is computed by using a synthetic image and then each obtained distance is used to compute an atmospheric transmissivity value for each individual pixel, therefore creating a much more precise temperature determination directly linked to a distance depth mapping. Other radiometric parameters such as object emissivity could be determined to further increase the precision in temperature determination. This could be achieved using appropriate GIS layers and accessing them based on the geographical information contained in the synthetic image. 32 SYNTHETIC VISION Chapter 4 This chapter describes the key element of the developed RDMU. This core element encompasses the automatic wildfire georeferencing subsystem, which is achieved by comparing a real thermographic image with a matching synthetic image of the same terrain. A synthetic image is an image obtained by a virtual camera located on a specific point of a virtual space which resembles a real-world scenario. The image being observed by the virtual camera is not the real scene, but a digital representation of such. In the particular case of willing to obtain synthetic images of a particular terrain or landscape, this digital representation should be the 3D equivalent of the real terrain. A Digital Elevation Model (DEM) is used as this 3D equivalent, where each point of the model contains latitude, longitude and altitude data extracted from the real terrain. An example of a rendered DEM file can be seen in Figure 4.1. To generate the synthetic image, ray tracing techniques are used over a correctly georeferenced DEM model. FIGURE 4.1. Example of a Digital Elevation Model (DEM). 33 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES 4.1.4 Spatial Transformations Throughout the development of the synthetic vision system two coordinate systems have been mainly used: cartesian and spherical. Given two points in 3D space such as p0={x0,y0,z0}and p1={x1,y1,z1}, the straight-line equation is given by (4.2). x=x0+kxt y=y0+kyt z=z0+kzt (4.2) Where kx,kyand kzare the unit vectors given by (4.3). kx=x1−x0 ky=y1−y0 kz=z1−z0 (4.3) In spherical coordinates the equation of a line would be slightly different and expressed by (4.4). x=x0+r·cos(α)·sin(θ) y=y0+r·sin(α)·sin(θ) z=z0+r·cos(θ) (4.4) Therefore, the relationship between both (4.2) and (4.3) is given by equation (4.5). r=q(x−x0)2+ (y−y0)2+ (z−z0)2 α=arctan(y−y0 x−x0) θ=arccos(z−z0 r) (4.5) The sign convention and axis specification for spherical coordinates are shown in Figure 4.7. Angle αcan vary between 0◦and 360◦, angle θBetween 0◦and 180◦and r can have a magnitude within 0 and infinity. Initially a series of functions were developed to generate the synthetic image from the vertical and horizontal FOV angles of the thermographic camera and by taking into account the angles of pitch, roll and yaw as indicated by the IMU, however it was soon found that this approach imposed a series of limitations and subsequent problems in case the inertial and body frames (IMU and Earth) where not aligned. This means that the axis of the IMU and the ones used over the DEM model are not equivalent to each other and there’s some rotation and translation between them. This in fact would be the usual case when the system is being used. In order to generalise the synthetic image creation and make it completely independent of the orientation of the thermographic camera, a need arised to apply tools which allowed to transform the synthetic image by a rotation equal to the attitude of the thermographic camera. After several tests with the rotation matrices, it was decided to use quaternion transformations in order to be able to apply the necessary rotations to 40 4 Synthetic Vision FIGURE 4.7. Relation between cartesian and spherical coordinate systems. the pixels of the synthetic image based on the thermographic camera’s attitude given in terms of pitch, roll and yaw angles provided by the IMU. 4.1.4.1 Body Frame Rotations using Quaternions A quaternion is a vector with four dimensions {h,i,j,k}. It has a real part and three imaginary components, or a scalar and a vector {i,j,k}. Therefore it can be expressed in any of the formats given in (4.6). Q= (a,b,c,d) = (s,v) = a+bi +c j +dk (4.6) Furthermore the relations specified in (4.7) are also fulfilled, which are known as the Hamilton equations in reference to the first mathematician to describe a quaternion, William Rowan Hamilton. It must be noted that the multiplication of quaternions is non-commutative but it is in fact associative (Horn, 1987; Chi, 1998). a,b,c,d∈R;i2=j2=k2=−1; ix j =k=−jxi;jxk =i=−kx j;kxi =j=−ixk (4.7) The unit quaternion is defined as {1,0,0,0}and the coordinates of a point in quaternion notation as {0, x0,y0,z0}. A rotation by a specified an angle φwith respect to an axis ris given by (4.8). Qφ=R(r,φ) = (cosφ 2,r·sinφ 2)(4.8) Therefore, the result of rotating a point P={x0,y0,z0}represented in quaternion 41 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES notation as Qp= (0, vp) = (0, x0,y0,z0)an angle φwith respect to an axis ris finally given by (4.9). Qs=Qφ·QP·Q∗ φ= (cosφ 2,r·sin φ 2)·(0, vP)·(cosφ 2,r·sinφ 2)∗(4.9) In order to be able to rotate any arbitrary point Pspecified in quaternion notation as Qp= (0, vp) = (0, x0,y0,z0)among the three rotational angles provided by the IMU: yaw (α), pitch (θ) and roll (φ) a specific rotation quaternion needs to be calculated which performs the full rotation in three stages. A particular order needs to be followed when doing this triple rotation, starting with the yaw angle (α), followed by the pitch angle (θ) and finally a rotation among the roll angle (φ). This new quaternion would then be applied in the same way as shown in(4.9), resulting in Qαθφ ·QP·Q∗ αθφ. This process has been implemented as part of the synthetic image generation process and uses as an input the three rotational angles (yaw, pitch and roll) given by an IMU installed on top of the thermographic camera. The process provides as an output the necessary rotation quaternion capable of converting any point from the body frame (thermographic camera or IMU to the inertial frame (DEM model). 4.1.5 Magnetic Declination It must be noted that the value of the yaw angle measured by the IMU is referenced to the magnetic north, so this value must be converted to true heading so that it can be equivalent the north reference used by the DEM model. The relation between the magnetic north and the true or geographical north is given by a parameter called magnetic declination and represents the difference between the magnetic and geographic north. Magnetic declination varies both from place to place and with respect to time. In most areas, the spatial variation reflects the irregularities of the Earth. However, in some areas, deposits of iron ore or magnetite in the Earth’s crust may contribute strongly to the declination. The magnetic declination in a given area may change slowly over time, around 2◦every hundred years, depending on the distance from the magnetic pole. Figure 4.8 shows the sign convention for the magnetic declination and how it mus be added to the yaw value provided by the IMU so that it can be converted to a bearing with respect to true or geographical north. FIGURE 4.8. Sign convention used for the correction of magnetic declination. 42 4 Synthetic Vision The value for the magnetic declination for a given area on Earth can be obtained by using the World Magnetic Model (WMM), consisting in a large representation of the Earth’s magnetic field which gets updated every five years. The current WMM model is WMM2015 (NOAA, 2014), released on December 2014 and will expire on December 2019 and superseded by WMM2020. As a curious fact, the current WMM model is installed on all Android and iOS devices, as well as navigational GPS receivers, to correct for the magnetic declination. The same principle has been applied in this thesis and the WMM2015 model has been installed on the RDMU to compute the local magnetic declination value at the location indicated by the GPS unit 4.1.6 Synthetic Image Generation As previously discussed, the final objective of generating a synthetic image representing the terrain seen by the thermographic camera is to be able to assign each individual pixel of the thermographic image its geographical coordinates and altitude. Therefore, the synthetic image will have the same resolution as the thermographic image but will contain a triad of information (latitude, longitude and altitude) for each individual pixel. This section describes how the process of generating the synthetic image is achieved, which is illustrated in Figure 4.9. This figure describes how a synthetic image can be obtained by means of the ray tracing technique previously described and using the same optical parameters as the lens from the thermographic camera. FIGURE 4.9. Synthetic image generation using ray-tracing techniques. A colour mapped representation of the synthetic image, where a blue colour will represent the lowest altitude and red the highest altitude, will generate a visual image similar to the one obtained by the thermographic camera. This will be used at later stages as a comparison tool to evaluate the precision of the generated synthetic image. Figure 4.10 shows the comparison between a thermographic and a synthetic image after the colour mapping has been applied. The thermographic camera chosen is a FLIR A615 like the one seen in Figure 4.11. The main reasons behind the selection of this particular model were both its high sensitivity and extended features and low-level controls allowing the implementation of custom calibrations. The main optical characteristics of this camera are presented in Table 4.1. The synthetic image generation process will rely entirely on these optical parameters together with the geographical location provided by the GPS and its pitch, roll and yaw 43 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES FIGURE 4.10. Thermographic and synthetic image comparison. FIGURE 4.11. FLIR A615 thermographic camera. 44 4 Synthetic Vision TABLE 4.1. Optical characteristics of the FLIR A615 thermographic camera. Property Value Horizontal Field of View (HFOV) 25◦ Vertical Field of View (VFOV) 18.8◦ Focal Distance 24.5 mm Horizontal Pixels 640 Vertical Pixels 480 attitude angles indicated by the IMU. These angles are contained within the following ranges: • Yaw (α): 0◦to 360◦. Yaw angle is referred to the magnetic north, so a value of α=0◦is aligned with the local magnetic north. • Pitch (θ): -90◦to 90◦. From the camera’s perspective, a negative pitch angle means that the camera is tilted downwards from the horizontal and upwards for a positive pitch angle. • Roll (φ): -180◦to 180◦. From the camera’s perspective, a negative roll angle means that the camera is rotated clockwise and counter-clockwise for a positive roll angle. The process described to compute the synthetic image can be described by means of the following steps: 1. The GPS unit provides the location data in latitude, longitude and altitude. The altitude component is discarded and substituted by the local DEM model’s altitude. 2. The IMU provides the yaw, pitch and roll angles of the thermographic camera with respect to the inertial frame. The magnetic declination is computed by using the WMM2015 model based on the location data provided by the GPS and gets applied to the yaw component. 3. Latitude, longitude, altitude, yaw, pitch and roll data gets feeded into the synthetic image generation algorithm together with the FOV and IFOV parameters (these remain constant). A synthetic image with the same resolution and FOV/IFOV parameters as the thermographic image is obtained. Each pixel of the synthetic image contains latitude, longitude and altitude information for its location on the terrain. 4. Both images (synthetic and thermographic) can now be used to access each other’s information. The fusion of both images could be considered as a multi-dimensional matrix which contains location and temperature information for each pixel. 5. The distance from the camera to each of the pixels of the synthetic image is computed and used to calculate pixel-specific atmospheric transmissivity values that will be used to perform temperature corrections as described in Chapter 3. 45 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES 4.2 Location and Attitude Inaccuracies The precision achieved when computing the distance from each of the camera’s pixels to its actual geographical location depends on both the accuracy of the generated synthetic image and the actual thermographic camera’s attitude and how accurately this one can be determined. This section describes how the errors associated to these processes could affect the final georeferencing of the wildfire. 4.2.1 GPS Receiver The vertical component (Z axis or altitude) of a GPS fix is between two and five times less accurate than the horizontal components (X and Y or longitude and latitude) (Wing & Frank, 2011; Rodriguez-Perez, Alvarez, Sanz, & Gavela, 2006). In order to achieve the best possible accuracy when determining a GPS fix, the satellites in view should be spaced as much as possible between them, ideally 120◦degrees from each other, but this ideal condition is not always met. At least three satellites are needed to achieve a 2D fix and four of them are required to achieve a 3D fix which includes altitude information. A 2D fix from a GPS can be usually considered sufficiently good when the Horizontal Dilution of Precision (HDOP) reported by the GPS unit presents a value lower than 1. A value below this threshold can be typically achieved after the GPS unit has been on for a few minutes and computing the actual location and converged to a stable solution in both latitude and longitude. In order to obtain the same accuracy on the vertical component than on the horizontal ones, the satellites used to compute the fix should also be spaced 120◦degrees perpendicular to the receiver. However, this is not possible, as the terrain itself will always interfere within at least one of the satellites. Even if we considered a completely flat terrain, only two satellites would be available within the 120◦ideal separation span. Even if satellites where closer together, they would be too close to the horizon, where propagation conditions are far from ideal and therefore cause additional errors in the signals, thus incurring on a further altitude determination inaccuracy. In addition to the previous limitation, GPS receivers use a WGS-84 ellipsoid (World Geodetic System 1984, 2004) to determine the reported altitude, which is based on a theoretical ellipsoid that at some points falls far from the real topographical altitude of a given point. This renders the vertical component almost not reliable at all for achieving a precise synthetic image of the terrain. To overcome this issue, the implemented process takes only the 2D solution of the GPS when an HDOP <1 has been achieved and then obtains the altitude component from locally stored DEM map. 4.2.2 Inertial Measurement Unit (IMU) The attitude of the thermographic camera is determined by means of an Inertial Measurement Unit (IMU) attached to the housing of the thermographic camera and 46 4 Synthetic Vision aligned with its optical axis. IMUs typically contain an accelerometer, a gyroscope and a magnetometer to compute the attitude of the IMU unit with respect to an inertial frame, in this case the Earth. By combining the output of each of of previous elements, an IMU computes its spatial attitude in terms of three rotational components pitch, roll and yaw. Pitch and roll are measured with respect to the body frame of the IMU itself, but as previously described the yaw component is measured in reference to the Earth’s magnetic north, where an angle of Yaw =0◦or Yaw =180◦means that the IMU is aligned with the magnetic north. In order to be able to correctly generate a synthetic image optically equivalent to the thermographic image, the attitude of the thermographic camera needs to be precisely determined, as well as its GPS coordinates and altitude. Any error on these input parameters would incur in a lack of alignment between the synthetic and thermographic images and therefore in considerable errors during the automated georeferencing process. The error introduced due to the IMU would depend on the error during the calculation of each of the rotational components (pitch, roll and yaw) and the distance from the thermographic camera to the wildfire. For instance, an error of ±0.5◦in the determination of any of the rotational angles would induce a georeferencing error of 5 m at a distance of 573 m. However, as discussed in the previous section an error of 5 m in the georeferencing of a wildfire is almost negligible and comparable with the error already introduced by the DEM map itself. If more precision was necessary, higher-end IMUs could be used but this would unnecessarily increase the overall cost of the solution without providing a very notable difference in terms of georeferencing accuracy. Magnetic interference suffered by the IMU’s magnetometer are another important source of error that can increase the uncertainty in the determination of the Yaw rotational component (Liu & Zhu, 2014). The magnetometer within the IMU is affected by what are commonly known as hard and soft iron distortions. As the thermographic camera can be deployed in almost any type of terrain or location, nearby metallic structures or buildings can contribute to this effect and distort the sensed magnetic field, which ideally should only be due to the Earth’s magnetic field. A good calibration procedure for the magnetometer is imperative to overcome these magnetic interferences and must be carried out periodically to ensure the health of the sensor. 4.2.2.1 Magnetometer Calibration for Hard and Soft Iron Effect Compensation Hard iron distortions are created by objects that produce a magnetic field. For example, a speaker or piece of magnetised iron could cause a hard iron distortion. If the piece of magnetic material is physically attached to the same reference frame as the sensor, then this type of hard iron distortion will cause a permanent bias in the sensor output. Soft iron distortions are considered deflections or alterations in the existing magnetic field. These distortions will stretch or distort the magnetic field depending upon which direction the field acts relative to the sensor. This type of distortion is commonly caused by metals such as nickel and iron. In most cases hard iron distortions will have a much larger contribution to the total uncorrected error than soft iron. 47 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES A common way of visualising and correcting hard and soft iron distortions is to plot the output of the magnetometer on a 2D graph. Figure 4.12 shows measurements taken by the magnetometer as the device is slowly rotated around the Z axis. FIGURE 4.12. Absence of hard iron and soft iron distortions. If there are no hard or soft iron distortions present, the measurements should form a circle centred at X=0, Y=0. The radius of the circle is equal to the magnitude of the magnetic field. Hard iron distortions will cause a permanent bias to be present on the outputs. The effect of hard iron distortions on the plot will be to shift the centre of the circle. As it can be observed in Figure 4.13 the centre of the circle with hard iron distortions is now at X=200, Y=100. From this we can conclude that there is 200 mGauss hard iron bias in the X axis and 100 mGauss hard iron bias in the Y axis. Hard iron distortions will only shift the centre of the circle away from the origin and will not distort the shape of the circle. Soft iron distortions on the other hand distort and warp the existing magnetic fields. Soft iron distortions are easy to recognise since they will distort the circular output and turn the circle into an elliptical shape. An example of the magnetometer output when both hard and soft iron distortions are present can be seen in Figure 4.14. The circle has now been distorted into an ellipse. The centre of the ellipse is still located at X=200 mGauss and Y=100mGauss since the hard iron distortions are still present. Every ellipse has a major and minor axis which corresponds its long and short dimensions respectively. As shown in the previous plot the ellipse has its major axis aligned 30◦up from the body frame in the X direction. It’s possible to eliminate the effects of both hard and soft iron distortions on the magnetometer outputs by applying a calibration as the one sown in shown in (4.10) to correct for hard and soft iron distortions. 48 4 Synthetic Vision FIGURE 4.13. Presence of hard iron distortions. FIGURE 4.14. Presence of both hard iron and soft iron distortions. 49 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES Among the wide variety of both voice and data services offered by TETRA, there’s one in particular which is of interest for low-volume data transfers, the Short Data Service (SDS). This is a data transmission service provided by TETRA to transmit short messages comparable to the SMS approach within GSM. A TETRA SDS message can carry up to 140 bytes per message, which it may seem lower than the 340 bytes carried by Iridium, but the latency of a TETRA network is much lower and down to fractions of a second, so the data throughput that can be achieved is significantly higher. TETRA seems like the ideal telecommunication system that could be used to rapidly integrate with already other emergency services as most countries and regions already have an existing TETRA infrastructure seamlessly integrated into the telecommunications infrastructure of key government entities. However, the deployment of a TETRA infrastructure is a major and costly engineering task and can take up to several years to perform an operational planning and implementation. 5.1.5 DMR Digital Mobile Radio (DMR) is a standard that has been developed by the European Telecommunications Standards Institute (ETSI) which sets out a digital radio specification for professional, commercial and private radio users (Digital Mobile Radio (DMR)), 2016). DMR is a two slot Time-Division Multiple Access (TDMA) system offering low cost and flexible digital voice and data solutions and uses a well established FSK modulation scheme. It can provide long-range communications over a frequency band that ranges from 30MHz up to 1GHz and also supports the transmission of data packets or messages. DMR is a scalable system that can be used in both unlicensed and licensed modes, subject to national frequency planning and has been developed in three tiers: • Tier 1 is the low-cost, licence-exempt digital PMR446 standard. • Tier 2 is for the professional market offering peer-to-peer mode and repeater mode and requires a license. • Tier 3 is for trunked operation and also requires licensing. DMR has been mostly designed to fit into existing licensed PMR bands in an effort to favour the transition from analogue to digital. The new standard imposes no fundamental changes in the architecture of either conventional or trunked systems and has been considered as low-cost alternative to TETRA, specially for users who do not need the complexity of a conventional TETRA infrastructure. With regards to M2M-style data transmission and in comparison with the aforementioned SDS protocol for TETRA, there’s a specific data transfer protocol for DMR known as the Packet Data Protocol or (PDP). This protocol supports the transmission of short data packets of up to 1508 bytes in both unconfirmed and confirmed transmission modes and also offers IP capabilities. The implementation of a DMR network is much more simpler and more cost-effective than a TETRA network as it only needs a series of interconnected base stations. However, 56 5 Emergency Data Relay this type of radio communication system has not achieved the popularity of TETRA due to its lower reliability and not so demanding encryption and redundancy requirements. 5.1.6 Mobile Broadband Mobile broadband on any of its 2G, 3G or 4G generations is undoubtedly the most popular telecommunications network nowadays, especially after the popularisation of smartphone devices in the early-2000s. This network is used every single day by millions of users worldwide and has the added benefit of having the most predominantly deployed infrastructure in almost every country. Mobile broadband provides a high data bandwidth at a quite reasonable cost, with access technologies and devices being so popular that really advanced and miniaturised mobile broadband modems can be found easily off-the-shelf. It also has the capability of implementing end-to-end TCP/IP stacks, so not only the transmission and reception of telemetry and telecommand data is quite straightforward and reliable, but also images and video can be transmitted in real time. It’s not within the scope of this thesis to provide a detailed description of how mobile broadband telecommunication networks actually work, as it is a very widely adopted technology which has been extensively covered in literature, technical notes and standards (Vriendt, Laine, Lerouge, & Xu, 2002). 5.1.7 WiMAX WiMAX or Worldwide Interoperability for Microwave Access falls under the IEEE 802.16 (IEEE 802.16 Broadband Wireless Metropolitan Area Networks (MANs), 2012) family of wireless communication standards. This technology has been commonly considered an over-the-air alternative to coaxial cable or DSL connectivity or even as a possible alternative to the 4G implementation of mobile broadband. It resembles a structure and functionality similar to classical WiFi or IEEE 802.11, but with a series of key differentiating features such as its increased range and infrastructure-creation capability. The data transfer speeds that can be achieved with WiMAX can range up to 1 Gb/s. The costs of deploying a WiMAX architecture is also much lower than those of mobile broadband architectures, so it results economically viable to provide broadband Internet access in remote or rural areas. WiMAX can provide communication links at distances over 50 km compared to the approximately 100 m provided by conventional WiFi (Rakesh, Maheswari, & Srivatsa, 2014). However, as the distance increases the speed that can be achieved with WiMAX decreases, but it can still provide speeds of around 10 Mb/s at distances of around 10 km. Although WiMAX and WiFi are designed for different situations they are also complementary to each other, where WiMAX can provide the long-range link and the rely on a local WiFi connection to interconnect several equipment. WiMAX requires a certain degree of infrastructure to be put in place in order for the network to be operative, specially regarding the installation of Subscriber Stations (SS) to provide access to the network. Although it’s quite a popular technology, it has not 57 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES been widely adopted yet by countries to provide rural area data coverage and the range of commercially available modems is relatively limited. 5.2 Emergency Data Relay Dispatcher This implemented dispatcher works under Linux inside a computer controlling the RDMU and is able to select from three different types of communication links based on a pre-selected criteria. The three telecommunication systems which have been finally chosen for data relay are Mobile Broadband, Terrestrial Trunked Radio (TETRA) and Iridium Short Burst Data (SBD) satellite service. The criteria behind the selection of these three systems or services responds to several unique features to each one of them and have been outlined in the previous section. The key features supporting the selection of these technologies for the particular case of the RDMU implementation are exposed hereafter: •Mobile Broadband: Popularity of mobile broadband, high speed data transfer rates and wide variety of inexpensive COTS modems available. •TETRA SDS Service: Secure, reliable and seamless integration with many official emergency response services, as well as wide adoption and current availability of TETRA infrastructures in many countries and regions. The RESCAN TETRA network in the Canary Islands has been specifically used during the development of this thesis. SDS service provides low packet data capacity but can be compensated with the network reliability and low-latency. •Iridium SBD Service: Global coverage and completely infrastructure-independent that doesn’t rely on the use of GEO satellites and therefore no precise aiming of the transceiver’s antenna is required. SBD packets are limited in capacity but can end up being the only way of transmitting data if the other systems fail. The idea behind the implemented data dispatcher module is to provide a resilient system capable of transparently relaying data in both directions to and from the RDMU. Figure 5.5 illustrates the principle behind the data dispatcher. FIGURE 5.5. Overview of the emergency data relay dispatcher. 58 5 Emergency Data Relay Regardless of how the information is sent from the RDMU, the data is transparently routed to the core network used by the emergency distress team. The dispatcher is also able to reconfigure the generated data on-the-fly in order to transmit it on each of the previous systems, regardless of the available bandwidth. The selection of each of the previous communications systems can be determined either by cost restrictions or by bandwidth requirements. If no cost restrictions are applied, the RDMU will always try first to send the data through the system with the higher bandwidth available, which in this case would be mobile broadband. If this link is interrupted, possibly due to mobile base stations being damaged by the wildfire or saturated by user traffic, the telemetry and telecommand dispatcher will fall-back to TETRA and lastly to Iridium SBD. If a cost restriction is applied, TETRA would be the first option, followed by mobile broadband and lastly Iridium. The cost restriction could be applied for instance when the RDMU is used as a preventive measure and it’s not currently tracking an ongoing wildfire. This setting can be remotely changed and fully customised by the operator at any given moment. The following sections provide further details about each of these telecommunications services and how they have been integrated in order to send and receive data through each one of them. 5.2.1 Iridium Short Burst Data Service (SBD) Iridium’s SBD service is often used for asset tracking or remote monitoring (Dahal, Petersen, & Meng, 2008). SBD messages being sent by an Iridium transceiver can either be delivered via HTTP as an email attachment to a pre-configured email address, or as a TCP socket connection to a specified server. A location report using Iridium satellite triangulation is usually also attached to the email message, providing the end user with an approximate location of where the transceiver was located when it transmitted the message. The latency of these messages is usually in the range of 6 to 22 seconds, depending on the conditions at the moment where the message was transmitted. Figure 5.6 describes the Iridium SBD architecture and the different entities involved. Depending on the Iridium transceiver selected, the options and capabilities which are available within the SBD service can vary slightly. For the purpose of this thesis an Iridium 9603 transceiver was selected, mainly due to its small footprint, low power consumption and advanced features. Figure 5.7 shows an Iridium 9603 transceiver like the one used. It most be noted that this transceiver can only operate with Iridium’s SBD service and not with other data relay options offered by the network. Depending on the direction of SBD messages, this can either be considered Mobile Terminated (MT) or Mobile Originated (MO), where by mobile we are referring to the Iridium transceiver unit. Mobile terminated SBD messages can have a length of up to 270 bytes, whether mobile originated messages can have a length of up to 340 bytes. Table 5.3 shows the structure of an SBD message. The Information Elements (IE) section of an SBD packet contains several fields which help identify the packet when it’s sent over the Iridium network and also to recover the 59 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES FIGURE 5.6. Iridium SBD system architecture. FIGURE 5.7. Iridium 9603 SBD transceiver. TABLE 5.1. SBD message structure. Field Length (Bytes) Values Protocol Revision Number 1 1 Overall Message Length 2 N Information Elements N Varies between MO and MT 60 5 Emergency Data Relay payload contained within. There are four types of Information Elements which vary slightly depending if it’s an MO or MT SBD message. These are described hereafter: •Header IE: This IE contains all the necessary information correctly identify an SBD MO message, including the IMEI of the Iridium transceiver and a series of time-stamps and session identifiers. If it’s an MT message then the IMEI shall also be included in order to correctly route the SBD message to the desired transceiver, but in this case a flag field is also included and can serve to mark the MT message as a high priority or clear previously queued MT messages, among other features. •Payload IE: In both MO and MT cases, this IE contains the actual data to be transmitted. As previously outlined, the maximum MT message size is 270 bytes and the maximum MO message size is a bit larger, 340 bytes. •Location Information IE: This IE contains an estimated location of the Iridium transceiver who transmitted the SBD message. It must be noted that this location is the result of triangulation by Iridium satellites and there’s no GPS involvement at all, so the precision usually lies in the range of several kilometres. •Confirmation IE: Both the Iridium transceiver and user’s server can exchange this type of IE to confirm that either a MO or MT message has been received. Iridium by default provides a mechanism to indicate if the transmissions have occurred successfully up to the Iridium network, but no end-to-end acknowledgement is performed unless this IE is explicitly sent. The Iridium 9603 transceiver can be controlled by a host device using AT commands. This type of commands are a series of short text strings which have been historically used to control modem devices and commanding them to perform several actions such as dialling, hanging up or changing parameter configurations. These commands are send as text strings to the 9603 over a serial connection. In particular, the following AT commands shown in Table 5.2 have been used to interface with the Iridium 9603 transceiver. However, it must be noted that the repertoire of AT commands supported by the transceiver is more extense and supports additional features (ISU AT Command Reference, 2012), although they are not of particular interest for this particular case. TABLE 5.2. AT control messages used on the Iridium 9603 SBD transceiver. AT Message Description AT+SBDWB Write binary data to an MO SBD message AT+SBDIX Initiate extended SBD session for MO and MT exchange AT+SBDRB Read binary data from an MT SBD message AT+CSQ Check signal quality and network coverage AT+SBDS Check buffer status of MO and MT messages By using this set of commands, the implemented emergency data relay subsystem can transmit telemetry messages and parse received telecommands over the Iridium network, 61 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES which provides the wildfire detection system with worldwide remote coverage for both telemetry and telecommand functions. The integrity and availability of the service is also monitored as part of the emergency data relay subsystem, which is used as an input into the decision making mechanism in case a fallback is necessary to an alternative service such as TETRA or 3G. The following sections describe how data transactions take place in both a mobile originated (telemetry) and mobile terminated (telecommand) scenario. 5.2.1.1 Mobile-Originated (MO) SBD An MO message starts with the decision being made by the emergency communications dispatcher that a telemetry message needs to be sent over the Iridium transceiver. This message needs to be written to the MO buffer of the Iridium transceiver and then triggered to initiate a transmission to the Iridium network. The process can be described in terms of the following high-level steps and AT command transactions: 1. Communications dispatcher sends a AT+SBDWB=N command to the transceiver, indicating that it wishes to transmit a message with length of N bytes. As this is a MO message, N must be equal or less than 340 bytes. 2. Transceiver replies with READY, indicating that it’s ready to receive the binary data-stream and place it in its MO message buffer. 3. The binary transfer of N bytes starts taking place from the communications dispatcher to the transceiver. 4. The communications dispatcher sends a AT+SBDIX command to instruct the transceiver to initiate an SBD transfer. 5. Transceiver replies with a status message indicating if the MO SBD message transfer successfully took place. 6. Finally the communications dispatcher can issue a AT+SBD0 command to request the transceiver to clear the MO message buffer. Once the MO message has been transmitted by the transceiver it will reach a specified server via TCP/IP protocol after having been relayed by an Iridium gateway. It’s the end user’s responsibility to desencapsulate the complete SBD message upon reception until the payload IE has been extracted. 5.2.1.2 Mobile-Terminated (MT) SBD In case the emergency response centre wishes to send a telecommand to the RDMU, this transaction needs to be initiated by a custom software which sends a correctly formatted MT SBD message over TCP/IP protocol to a specific server provided by Iridium. 62 5 Emergency Data Relay Once the MT message has been received by the Iridium server this one will be checked for integrity and then uploaded to the space segment so that it can be relied to the targeted transceiver via one of the Iridium satellites. The IMEI needs to exactly match the one of the transceiver which should receive the message, as this is the only way of identifying and routing the message within the Iridium network. When the message is finally available to be retrieved by the transceiver, the following process can be followed to recover its payload by the communications dispatcher: 1. Communications dispatcher sends a AT+SBDIX command to either send a previously queued MO message or to check with the Iridium network if there’s an MT message which needs to be downloaded. If this last case is applicable, the message will be automatically downloaded to the transceiver. 2. If an MT message was downloaded during the previous step, a notification will be sent to the communications dispatcher to indicate that a message has been received and is waiting to be read. During this step the length of the incoming message in bytes is also passed along. 3. The communications dispatcher sends a AT+SBDRB command to trigger the binary transfer of the message from the transceiver to the communications dispatcher. Once the incoming MT message has been transferred by the transceiver there’s no need to perform any additional desencapsulation, as the transceiver already strips down the MT message from all its headers and protocol overhead and only delivers the payload IE to the communications dispatcher. 5.2.1.3 Custom SBD Data Segmentation In order to be able to transmit telemetry data which exceeds the maximum 340 bytes specified for Iridium for MO messages, a custom segmentation and encapsulation scheme had to be implemented. This means that if a given data packet exceeds 340 bytes, it will need to be divided into several MO SBD messages with a series of control fields which allow the receiver to assemble the whole packet in the correct order, regardless of the order on when each of the SBD message segments received. For the purpose of the communications dispatcher implemented in this thesis, the segmentation scheme defined in Figure 5.8 has been adopted. The least significant byte of every SBD message has been selected as a status and sequencing field and the next byte as an ID field, allowing to identify the SBD segment from others belonging to a different SBD packet. The SEQ field is coded with 6 bits and indicates the sequence number of an SBD segment, so that the full message can be reassembled in the correct order at the receiver. Using 6 bits yields 26=64 possible segments. The maximum SBD packet than can be transmitted is actually 338 bytes x 64 segments =21632 bytes. However, it should be noted that transmitting such a large packet over the Iridium network is unfeasible due to 63 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES FIGURE 5.8. SBD MO data segmentation for packets larger than 340 bytes. the latency of the network. This network latency usually falls within a few seconds with peaks of up to a minute. The STATUS field uses two bits to indicate if the segment is the last of the sequence. A binary value of 0b00 means that there are still more segments pending and 0b11 means that it’s the last segment. The other two binary values of 0b01 and 0b10 are not currently used and reserved for future applications. Finally, the ID field is incrementally initialised for every SBD packet transmission and remains constant and non-reusable during the transmission of the full packet and all of its associated sequence messages. If a packet has less than 338 bytes it can be transmitted on a single SBD message and no segmentation is needed. In this particular case the STATUS field is set to 0b11 and the SEQ field to 1. 5.2.2 TETRA Short Data Service (SDS) This is a data transmission service provided by TETRA to transmit short messages comparable to the SMS approach within GSM. A TETRA SDS message can carry up to 140 bytes per message, which it may seem lower than the 340 bytes carried by Iridium MO messages, but the latency of a TETRA network is much lower than Iridium, so the data throughput is significantly higher. SDS messages can be addressed to specific TETRA terminals using their TETRA Subscriber Identity (TSI), which can be considered as an analogy of an IP address on a classical computer network. Each TETRA terminal has one or more TSI identifier and consists of a 48-bit code which must be unique within the TETRA network. As seen on Figure 5.9 the TSI itself is composed of three fields, the Mobile Country Code (MCC), the Mobile Network Code (MNC) and the Short Subscriber Identity (SSI). FIGURE 5.9. Structure of a TETRA TSI identifier. The MCC consists of 10 bits and is used to identify the country of origin, for the particular case of Spain the MCC is equal to 214. The MNC consists of 14 bits and 64 5 Emergency Data Relay identifies the different TETRA networks within a country. The developed communications dispatcher has been integrated with the existing RESCAN TETRA network from the Government of the Canary Islands, and therefore the RESCAN specific MNC was used during the development of this thesis. The SSI identifies subscribers and system components within a the network. It consists of 24 bits and is divided into further fields: the Individual Short Subscriber Identity (ISSI) is used to uniquely identify a specific a terminal, the Group Short Subscriber Identity (GSSI) enables group formations within the network and the Alias Short Subscriber Identity (ASSI) is used for addressing foreign participants. Also integrated into the SSI are the Tetra system-wide addresses. With respect to the SDS message itself, Table 5.3 shows the structure of an SBD message. TABLE 5.3. SBD message structure. Field Length (Bytes) Values Header 2 or 4 0x0201, 0x8204FF01 or 0x820CFF01 Payload Data Max. 140 Any binary content The content of the SDS message header varies depending if the message is to be transmitted with no delivery confirmation (0x0201), with delivery confirmation (0x8204FF01) or with delivery and read confirmation (0x820CFF01). The destination ISSI is specified as part of the SDS service definition phase and it’s not contained within the message itself, but rather specified to the TETRA terminal as part of the preparation for the transmission of an SDS message. In a similar way than with the Iridium 9603 transceiver, the Sepura SRG3900 TETRA terminal can be controlled by a host device using AT commands. The AT commands are also sent as text strings to the 9603 over a serial connection. In particular, the following AT commands shown in Table 5.4 have been used to interface with the Sepura SRG3900 terminal. However, it must be noted that the repertoire of AT commands supported by this terminal is more extense and supports additional features, although they are not of particular interest for this thesis TABLE 5.4. AT control messages used on the Sepura SRG3900 TETRA terminal. AT Message Description AT+CTSP TETRA service profile AT+CTSDS TETRA service definition for SDS service AT+CMGS Send SDS Message AT+CSQ Check signal quality and network coverage AT+CNUM Subscriber identity number 65 RDMU PROTOTYPE INTEGRATION Chapter 6 As discussed in the previous chapters, one of the main motivations behind this thesis is to demonstrate the viability of a Rapidly-Deployable Mobile Unit (RDMU) that allows the remote detection and tracking of wildfires based on thermographic imaging. Furthermore, the task of georeferencing the identified fire spots shall be carried out automatically, contributing to reduce the level of subjectivity and uncertainty associated to this task. In order to achieve the desired objective in terms of a prototype, several key subsystems have been integrated and interfaces between them had to be designed. The general block diagram of the RDMU is showed in Figure 6.1. All subsystems presented in this block diagram have either been integrated, adapted or designed during the development of this thesis. In particular, the TM/TC block is directly related with Chapter 5 and the Data Processing and Georeferencing Unit (DPGU) implements both the synthetic image generation and thermographic calibration processes described in Chapters 3 and 4. To evaluate the feasibility of the RDMU in real environments the prototype has been developed around a portable form factor which can be easily carried and deployed by firefighting staff, as well as providing a handy way of storing the equipment when it is not being used. For this reason, the chosen design for the first prototype integration has been a commercial backpack that was heavily modified as seen in Figure 6.2, where all the critical control and communication systems have been fitted inside the backpack, including the batteries used to power the system. The front section of the backpack contains a custom laser-cut panel which serves as a connection hub to the external sensors and mechanical components, such as the thermographic and visible channel cameras, the IMU and the servo positioning mechanism, as well as providing power outputs and a charging connector for the batteries. These external components are carried on a separate backpack and are easily 73 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES TM / TC Data Processing and Georef. Power Distribution Optical Sensors (IR + VIS) Inertial Measurement Unit Positioning Mechanism Wired Connection Backpack Tripod FIGURE 6.1. Block diagram of the Rapidly Deployable Mobile Unit (RDMU). connected upon arrival at the desired observation point. FIGURE 6.2. RDMU custom backpack containing all control, power and communication subsystems. A particular emphasis has been placed on making the RDMU as simple as possible to operate, as one of the main requirements made by the local firefighting department was that it should not require any advanced training or qualification to be operated. This has been taken as a key input during the development of the prototype, up to the point where it just requires to be connected as shown in Figure 6.3 and then pressing a start button. From this moment onward the RDMU will operate autonomously and begin searching for and georeferencing wildfires, as well as starting all telecommunication subsystems described in Chapter 5. This allows the RDMU to be monitored and remotely operated when required. Any anomalies detected by the RDMU will be transmitted to a centralised control centre, where appropriate actions can be taken. 74 6 RDMU Prototype Integration FIGURE 6.3. RDMU backpack connected to the external equipment. 6.1 Optical Sensors and Positioning Mechanism The core element of the RDMU consists on a set of visible channel and thermographic cameras fixed to a servo-motorised base. As observed in Figure 6.4 this motorised camera head can be easily attached to a generic tripod, allowing a rapid initial setup. FIGURE 6.4. Visible channel and thermographic cameras. The cameras don’t need to be precisely oriented to the fire or region of interest, as they are capable of being remotely moved through a telemetry and telecommand (TM/TC) interface. Both the cameras and the servos need to be connected the central Data Processing and Georeferencing Unit (DPGU). This unit also supplies power among other control and data-processing features and will be discussed in the following sections. Both cameras have a resolution of 640x480 pixels and are equipped with lenses that 75 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES provide an horizontal field of view (FOV) of 25◦. This allows the cameras to cover a linear region of approximately 450 m at a distance of 1 km. They have also been aligned and collimated side to side so that the images obtained by both of them are equivalent. The main fire detection element will be the thermographic camera, while the visible channel camera will only be used as a visual aid for assisting firefighting brigades on decision-making tasks. Both cameras are connected to a network switch inside the backpack that in turn is connected to a central computer capable of controlling both cameras, acting as the DPGU and running the Linux operating system. The image being produced from any of both cameras can be accessed at any given time by a remote operator. In particular, the chosen thermographic camera (FLIR A615) uses the GigE Vision protocol to communicate with the central computer. This protocol is used by high-performance industrial cameras and provides a framework for transmitting high-speed video and control data over Ethernet networks. The visible channel camera also contains a BNC connector connected to a local analogue video monitor. The use of this monitor is optional and its main objective is to provide the operator deploying the RDMU with a visual feedback of what the camera head is actually seeing. Once the RDMU is operating on an autonomous way this video monitor is no longer needed. FIGURE 6.5. Pan (yaw) and tilt (pitch) servos attached to torque-multiplying gearboxes. The full camera head is attached to a motorised system capable of moving both cameras in pan and tilt movements, or from the camera´s perspective, in yaw and pitch. This action is controlled by a set of two servos that can rotate in perpendicular axis with respect to each other. Due to the weight of the cameras, both servos have been fitted within gearboxes capable of multiplying their torque value. The fully assembled gearbox system can be seen in Figure 6.5. The position of both servos is controlled by means of PWM signals applied to their inputs, which are generated by a microcontroller connected to the central computer. The neutral or half-way position for each servo is achieved when a PWM signal with a period of 1500 µs is applied to their input. Increasing this period would move the pitch servo 76 6 RDMU Prototype Integration up and the yaw servo clockwise, whether the opposite would occur for a smaller period. An example of these PWM signals can be seen graphically in Figure 6.6. Under maximum load the servos can draw up to several Amps of current and could be a potential drain of battery power. However, the camera head has been carefully balanced in order to reduce the torque resistance induced to the servo, therefore minimising the mechanical effort necessary to rotate the whole assembly. FIGURE 6.6. PWM signal used to control the servo’s angular position. 6.2 Location and Attitude Determination Subsystem Once the camera head has been attached to a generic tripod and the system is initialised, the first step carried out is to obtain the geographical coordinates and altitude of the camera, as well as its attitude expressed in yaw, pitch and roll angles. This is achieved by the Internal Measurement Unit (IMU) installed on top of the thermographic camera (Figure 6.7) and a high-precision GPS receiver (Figure 6.8). The GPS is connected to an external antenna which can be magnetically attached to any surface. Communication between both the GPS and IMU units and the central computer is performed by means of serial connections. In the particular case of the GPS, NMEA 0183 sentences (NMEA 0183 Standard 4.0, 2008) are sent constantly by the GPS receiver to the central computer. NMEA 0183 sentences contain information about the geographical location of the GPS receiver as well as other important parameters. An example of an NMEA sentence can be seen in Figure 6.9. All NMEA sentences sent by GPS receivers start with the string $GP and the first field of each sentence defines how the rest of the sentence shall be interpreted. Every GPS unit sends a series of these sentences containing different sets of information. For instance the $GPGGA sentence shown above contains fix information, but other sentences can contain detailed GPS satellite data ($GPGSV sentence) or ground speed if the GPS unit is moving ($GPVTG sentence), among many others. 77 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES FIGURE 6.7. Inertial Measurement Unit (IMU). FIGURE 6.8. High-precision GPS receiver. $GPGGA,123519,4807.038,N,01131.000,E,1,08,0.9,545.4,M,46.9,M,,*47 FIGURE 6.9. Example of a ($GPGGA) NMEA 0183 GPS sentence. 78 6 RDMU Prototype Integration The user of a GPS unit is free to decide which of the NMEA sentence needs to interpret and which ones to ignore. The GPS unit will always send all of the supported NMEA sentences unless individually disabled. For the particular case of the RDMU, the main sentence we are interested in is in fact the $GPGGA sentence shown in Figure 6.9. This sentence contains UTC time, latitude and longitude and also HDOP information to evaluate the precision of the computed solution. The way to decode a $GPGGA sentence is shown in Table 6.1. TABLE 6.1. Field decoding of a $GPGGA NMEA 0183 sentence Field Meaning $GPGGA Global Positioning System Fix Data 123519 Fix taken at 12:35:19 UTC 4807.038,N Latitude 48 deg 07.038’ N 01131.000,E Longitude 11 deg 31.000’ E 1 Fix quality 08 Number of satellites being tracked 0.9 Horizontal Dilution of Precision (HDOP) 545.4,M Altitude (AMSL) 46.9,M Height of geoid (MSL) above WGS-84 (empty) Time in seconds since last DGPS update (empty) DGPS station ID number *47 Checksum (always begins with *) After the initial power-on the GPS unit waits until a reasonable number of satellites are available to calculate the position fix. The minimum threshold established for this condition to be met is to have at least 7 satellites available. Furthermore, the Horizontal Dilution of Precision (HDOP) parameter is also measured, which gives an idea of the precision of the calculated fix, and only values below 1.0 are taken as acceptable. Usually the 2D precision of a GPS receiver is quite acceptable when there are a reasonable number of satellites in view, but the altitude parameter typically has a higher degree of uncertainty and can quite often be off by several meters. To reduce this source of error, the geographic coordinates (latitude and longitude), once they are considered accurate enough, are used to access a pre-stored DEM model of the area where the RDMU has been deployed, in order to quantify this error and apply a correction factor. After having successfully determined the latitude, longitude and altitude the pitch, roll and yaw angles of the camera head are then determined by the IMU. The yaw angle is initially measured with respect to the magnetic north and expressed as a magnetic 79 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES heading, which then gets converted to true or geographical heading by computing the magnetic declination at the specific location and date previously determined by the GPS receiver. The DPGU contains a worldwide database of magnetic declination values which can be accessed based on geographical location. As discussed in Chapter 4, the set of location (latitude, longitude and altitude) and attitude (pitch, roll and yaw) data obtained by both the GPS and IMU in this section are the key element to correctly implement the automatic wildfire georeferencing capability using a synthetic image. 6.3 Data Processing and Georeferencing Unit (DPGU) The full control of the RDMU relies within this subsystem. The DPGU is in charge of controlling the camera head, obtaining the synthetic images and performing the automatic georeferencing, as well as coordinating the telemetry and telecommand operations. These tasks are performed by a central computer inside the RDMU. This computer is equipped with an SSD drive to prevent failures during rough handling situations and also for increased data transfer speeds. Figure 6.10 describes the different steps carried out during the execution of this software. For simplicity reasons, only the high-level steps directly involved with the detection and georeferencing of wildfires have been described. Other processes are also continuously running in the background for housekeeping and communication services. The detected hot spots are georeferenced after each iteration and are made readily available to be used by external GIS applications. These georeferenced spots are transmitted regularly to a central server, where other tools or software can make use of them during the suppression process and emergency response planning. The system can also receive instructions to point the cameras at specific locations and to retrieve live visual or thermographic images of the area under surveillance, as long as the communications link provides enough bandwidth. 6.3.1 Multimode Resilient Data Relay The multi-mode resilient data relay has been integrated and tested separately from the RDMU and is currently an optional accessory at this development stage. This is mainly due to the power requirements and dimensions of the TETRA terminal used, as the Motorola TOM100 solution was not available at this point. 80 6 RDMU Prototype Integration System Initialization Wait for GPS Fix HDOP < 1? Obtain Altitutde Data from DEM DEM Obtain Pitch, Roll and Yaw Data from IMU Capture IR Image T > Threshold? Rotate Camera Head Georeference IR Image YES YES NO NO Geographical Information System (GIS) FIGURE 6.10. Flowchart describing the execution of the DPGU software running inside the RDMU. 81 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES A: Thermographic image. B: Synthetic image. FIGURE 7.3. Synthetic image with large disalignment with respect to the thermographic Image. the profile, or interface curve, between the terrain and sky in both the synthetic and thermographic images, so that a set of error offsets can be computed and applied to the pitch, roll and yaw angles. As previously discussed, the synthetic image contains the coordinates of each pixel as well as its altitude. During the generation of this synthetic image, any pixel which is not part of the terrain is given an altitude of zero. This is achieved during the ray-tracing process, where any ray which doesn’t impact with the terrain is considered to be part of the sky. For every column in the synthetic image, a search is conducted to find any pixel immediately preceding a pixel with an altitude value of zero. Every time this occurs, the row at which that occurrence happened is saved in a 640-element vector. This process determines the boundary between sky and terrain in the synthetic image as it can be seen in Figure 7.4. FIGURE 7.4. Computed sky-terrain interface using the synthetic image. The next step in the process is to find the same information but within the thermographic image, in order to be able to match both profiles afterwards. In this case, 88 7 Functional Evaluation obtaining the boundary between the sky and the terrain is not as immediate, as the thermographic image contains temperature values. However, the temperature values of sky and terrain are quite different between each other, so the determination of where the actual interface occurs can be computed by performing inter-pixel absolute differences. A: Thermographic image converted to 256-level grayscale. B: Binarized temperature differences image. FIGURE 7.5. Binarized temperature differences image generation based on the thermographic image. The thermographic image is first converted into a 256-level grayscale image. From this image, a matrix containing absolute temperature differences between each row is then calculated by assuming |Zpix [j+1,i]−Zpi x [j,i]|. Every value that doesn’t fall above a given threshold (10 in this case) gets assigned a value of zero. This is to limit the number of differences to those actually significant enough to be representative of a sky-terrain interface. After this, the absolute difference image gets binarized, where each pixel now has a value of either 1 or 0 depending on whether its temperature difference falls above or below the threshold. This new image has a dimension of 640 x 479 pixels and can be seen in Figure 7.5. Once both curves have been obtained, a correlation process is applied to both the synthetic and thermographic sky-terrain interface in order to compute the pitch, roll and yaw difference which should be applied to the synthetic image. This is required so that it becomes optically equivalent to the thermographic image. Figure 7.6 shows the output of this process and how an offset of ∆Yaw =5.2◦, ∆Pitch =0.2◦and ∆Roll =−0.5◦has been computed. After the offset has been determined the corrected synthetic image can then be generated, which will be then be input to the wildfire georeferencing process. Once the alignment has been performed, the result can be seen in Figure 7.7. It should also be noted that if there’s vegetation relatively close to the thermographic camera it can cause errors when determining the location of the sky-terrain interface curve, as the top of the trees could be interpreted as terrain. Due to the fact that the DEM map doesn’t contain any vegetation altitude data, this difference could be a source of error during the previously described process. Depending on the distance from the thermographic camera to the vegetation, this error could be computed as an angle relative to the terrain. For instance, an error of 0.2◦would occur if vegetation had an altitude of 3.5 m at a distance of 1 km. This error can be generically evaluated as in Figure 7.8. 89 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES FIGURE 7.6. Pitch, roll and yaw offset determination based on sky-terrain interface. A: Thermographic image. B: Fully aligned synthetic Image. FIGURE 7.7. Final result and comparison of the thermographic and synthetic images after performing a terrain profile alignment. FIGURE 7.8. Error due to the presence of vegetation on the sky-terrain interface. 90 7 Functional Evaluation 7.3 Field Test during Prescribed Burns Several outdoor tests with real fire were carried out in order to evaluate the overall effectiveness of the proposed system. These tests were part of a functional validation campaign during a series of controlled burns in the island of Gran Canaria, Spain. These controlled burns where coordinated and executed by professional firefighting staff from the local council and the military emergency response unit (UME) as part of their routinary firewall maintenance. All results were then carefully analysed in order to quantify the errors induced during the process as well as the overall operability of the system. Figure 7.9 shows one of the locations where these tests took place. These images serve only as an indication of the sort of environment where the system was operating. In this particular case the RDMU was located at a distance of 2.5 km from the controlled burn. A: General view. B: Close-up view. FIGURE 7.9. Controlled burn carried out by professional firefighters. The generated synthetic and thermographic images can be seen in Figure 7.10. After generating the synthetic image, this one was fed together with the thermographic image into the terrain profile alignment process described in the previous section. This process determined that an error compensation offset of 0.99◦in the pitch angle was required, translating to an error of 44.23 m at the operational distance of 2.5 km. The georeferencing process also accounted for an additional 0.13◦due to the presence of vegetation with an estimated height of approximately 6 m. The difference before and after the terrain profile alignment process can be seen in Figure 7.11. The correctly georeferenced wildfire was then automatically exported to a GIS visualisation tool, in this case Google-Earth, for visualisation purposes as seen in Figure 7.11. The full identification and geolocation of the wildfire, in this case represented as a prescribed burn, has been demonstrated to be achievable with no human intervention at all, other than setting up the camera and powering the unit. All of these results were also relayed in real-time to a nearby operations centre, which had full bidirectional access to telecommand and telemetry from the whole system over a triple-redundant telecommunications networks. 91 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES A: Visible channel. B: Thermographic image. C: Synthetic image. FIGURE 7.10. Set of images produced by the RDMU during nominal operation. A: Before terrain profile alignment. B: After terrain profile alignment. FIGURE 7.11. Effectiveness of the terrain profile alignment process. FIGURE 7.12. Automated geolocation of the wildfire on Google-Earth. 92 7 Functional Evaluation The final result of the RDMU system for another controlled burn can be seen in Figure 7.13, which shows the full graphical representation over a GIS tool, in this case Google-Earth, as it would be seen at the emergency dispatch room during a wildfire outburst. The blue area indicates the FOV of the camera and it will rotate dynamically to show where the camera is pointing at any given moment and the area of terrain currently being monitored. The red dots represent the georeferenced hot spots detected by the thermographic camera and correctly placed on the map in a fully autonomous way by using the synthetic vision technology implemented in this thesis. FIGURE 7.13. Final result of the RDMU in full autonomous operation. 7.3.1 Low-Bandwidth Data Transmission Test During the previous field tests the resilient data relay functionality was evaluated in order to test its behaviour on the worst case scenario. This occurs when mobile broadband connectivity has failed and only TETRA or Iridium connectivity remain active. In this particular case the data throughput capability of either TETRA or Iridium doesn’t allow the transmission of thermographic images or complex GIS files generated by the RDMU containing wildfire locations with their associated metadata. Instead, a binarized 640x480 image will be transmitted indicating the location where hot spots have been detected after applying the pixel-specific atmospheric transmissivity corrections described in chapters 3 and 4. In a binarized image each pixel is treated as a single bit, indicating either the presence of fire with a binary 1 or no fire with a binary 0. If we follow this scheme, the whole binarized image could be coded with 640 x 480 =307,200 bits. However, due to the fact that usually less than 0.5% of the binarized image will actually contain pixels identified as fire, the 307,200 bit binary sequence will mostly contain 0s and only a small number of 93 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES 1s. This makes it quite feasible to be compressed by classical compression algorithms such as Lempel-Ziv-Welch (LZW), Huffman or Run Length Encoding (RLE) (Marcos-Agulhari, Rocha-Silveira, & Sebastiao-Bonatti, 2007). During the evaluated tests the compressed binarized image was able to be transmitted by using only two Mobile-Originated Iridium SBD packets, with the average transmitted data size being in the range of 500 bytes. The same transmission needed approximately four TETRA SDS messages, but due to the fact that TETRA presents a lower network latency the binarized image arrived sooner. Apart for transmitting the binarized image, the RDMU also needs to transmit the GPS and IMU parameters so that the georeferencing process can be completed at the receiver side with the same algorithm running inside the RDMU. The transmission of the whole set of GPS and IMU parameters can be transmitted in plain text format within a single Iridium SBD or TETRA SDS message. 94 CONCLUSIONS AND FUTURE WORK Chapter 8 The developed system has proved to be able to achieve the objective proposed at the beginning of this thesis. It has particularly demonstrated that the automatic geolocation of wildfires can be carried out by performing a data fusion of thermographic images and synthetic vision, at the same time as reducing the limitations of current solutions based on fixed observation points. The possibility of achieving a pixel-level calibration in terms of atmospheric transmissivity has been a novel approach and contribution of this thesis. As far as the state of the art analysis carried out, no similar system has been identified that performs this type of calibration based on synthetic vision. This capability not only enables a much more precise calibration of the temperature perceived by a thermographic camera, but can also yield a much broader dynamic range on the generated thermographic images. All studied thermographic analysis cases applied a uniform radiometric calibration to the whole image and this has proven to be an important source of temperature determination error if the distance to the object being measured is not precisely known. A high-precision GPS and a correctly calibrated IMU are also mandatory for the correct generation of the synthetic image. During the test campaigns it was observed that the GPS receiver can occasionally take a large amount of time to converge to the required HDOP level. This can be mitigated by storing within the GPS receiver’s a recent ephemeris of the GPS constellation, enabling a faster convergence. The IMU has also demonstrated to be prone to magnetic interference in several environments, especially in the vicinity of power lines or metallic structures. This causes the yaw component to usually have an error of several degrees, but it has been possible to compensate it by using the terrain profile alignment procedure described in Chapter 7. After performing some field tests with this method in place, the RDMU was able to compensate attitude determination errors higher than 5◦. The accuracy level achieved when measuring the remote temperature of a hot spot 95 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES using the thermographic camera has demonstrated to be absolutely dependant on the correct determination of the atmospheric transmissivity. If both the atmospheric temperature and relative humidity are precisely determined, the distance between the thermographic camera and the hot spot still remains as the main error-inducing factor. If this distance is erroneously calculated then the chances of having false alarms or false positives get severely increased. The determination of this distance relies entirely on the alignment of both the thermographic and synthetic image as well as the accuracy of the synthetic image. A more computational-intensive approach to the ray tracing technique could be applied in order to improve the synthetic vision algorithm and make it more robust with respect to different terrain configurations. It must be noted that a special emphasis has been made in using low-cost components for the development of the RDMU prototype and this should also remain the main philosophy even for a possible commercial implementation of this system. The strength of the developed system lies in its portability and reconfigurability. During a wildfire outbreak it would be common for the different RDMU units to be dynamically reallocated during the wildfire. Each of the individual units doesn’t need to have high-spec components in order to achieve its functionality but thanks to having several units working at the same time and providing real-time information, a much more detailed view of the wildfire evolution is achieved. This means that it would be much more effective both from cost and an operational perspective to have several low-cost units providing a much larger scale analysis than a single expensive observation point with high-spec thermographic cameras. It has been highlighted in Chapter 6 that the implemented prototype presents an important limitation with respect to power autonomy, but this could be easily solved by using larger capacity batteries, more power-efficient internal components or considering other power sources such as solar cells, although this last option could affect the portability of the system. Apart from the thermographic and synthetic vision methods implemented and in-depth analysis of telecommunication systems for rural environment data transmission has been performed. This has served to determine what would be the most desirable telecommunication services to use in an emergency situation and how to make them work seamlessly together to be able to transmit and receive data even in adverse situations. The development of the resilient multi-mode communications dispatcher presented some important challenges in terms of protocol-level integration within the RDMU and implied implementing a data-convergence layer within the RDMU software to accommodate for the different types of packetisation used by each service. This layer proved to be transparent to the end-point telecommand and telemetry services, which were able to correctly relay the desired data independently of the selected system. It would have been desirable if the complete resilient data relay could have been integrated inside the RDMU at this point, but the impossibility to source an appropriate TETRA terminal which fulfilled the desired characteristics was a limiting factor. However, even if the data relay was used as an external accessory at this point, it still allowed to fully implement and validate the complete telemetry and telecommand chain across three very different telecommunication services such as Iridium, TETRA and 3G mobile networks. This posed some really interesting challenges due to the technical difference among their communication protocols. However, reverting to the low-level AT commands, that are 96 8 Conclusions and Future Work widely implemented around many modems and transceivers, served as a key tool to successfully integrate all three terminals. This resilient data relay can also serve as a derived development from this thesis, since its use is not limited or dedicated solely to the RDMU, but it can be used in conjunction with any other emergency system which needs robust and redundant communication capabilities. A very important emphasis was made in trying to make the RDMU as easy as possible to setup and operate by firefighting staff and only requiring a minimal training of one hour to explain how the system works. The operational procedure of using the RDMU in a real case simply requires the firefighter to deploy a general-purpose tripod and install the camera head, which is then connected to a custom-modified backpack that houses the complete system. At this point, all the firefighter needs to do is press a start button and the system will start-up in a fully autonomous way and start searching for wildfires and relaying geolocated data straight away. This process has proved to be totally successful, however a second iteration of the implemented prototype is focusing in being able to implement the same functionality on a more lightweight and portable form-factor. Feedback from the firefighting staff is critical to fine-tune the performance of the system. Positive feedback has been received outlining that the automated georeferencing capability developed and presented in this thesis has served to speed up decision making mechanisms, as well as reducing the uncertainty of the generated geographical data. The portable form factor has also been highly appraised due to its increased manoeuvrability and dynamical configuration options while performing wildfire suppression tasks. Several field tests with prescribed burns were carried out in Spain, more precisely on the Canary Islands (Gran Canaria, Tenerife, La Gomera and La Palma) and in Valencia. These locations presented very different terrain configurations that served to evaluate the feasibility of the RDMU with direct involvement of professional wildfire suppression staff and under real operational conditions. 8.1 Future Work Some future work lines derived from the work carried out for this thesis are presented hereafter: • Keep improving the RDMU in terms of portability and power consumption, exploring new technologies in the fields of thermographic imaging and single-board or embedded computers. Further testing with different types of battery and possibly solar arrays would be desirable in order to further improve the autonomy of the whole system. • Explore the possibility of using the developed synthetic vision technology to achieve automated georeferencing capabilities for other applications, such as surveillance and security. This technique could also be used to provide much more enhanced thermographic images thanks to the pixel-level atmospheric transmissivity calibration performed. • The same principle of the RDMU could be applied to an airborne platform, so a feasibility study focused on using this system on-board a UAV would be a very 97 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES estudios realizados. Todas estos trabajos de fin de título fueron supervisados tanto académica como técnicamente por el autor de esta tesis. En un aspecto más social, varios periódicos locales y nacionales, así como canales de televisión han protagonizado el prototipo de la RDMU implementada como parte de esta tesis2. Tanto el prototipo como el sistema desarrollado también han sido aplaudidos por diversos colectivos profesionales y calificado como un elemento disruptivo en la detección temprana de incendios forestales. FIGURA A.1. Portada de la edición de febrero de 2016 de la revista IEEE Aerospace and Electronic Systems Magazine. Fuente: IEEE. 2http://www.idetic.ulpgc.es/idetic/index.php/es/proyecto-selif 104 A Sistema de Geolocalización Autónoma de Incendios Forestales basado en Técnicas de Visión Sintética y Termográfica A.2 Sistemas de Detección y Predicción de Incendios Forestales Esta sección ofrece una visión general de los sistemas de detección de incendios forestales más utilizados, que también pueden utilizarse como herramientas de seguimiento en caso de que el incendio forestal ya haya comenzado. Varias de sus principales ventajas y limitaciones se discuten en cada caso. Finalmente se presenta una breve descripción de cómo se pueden combinar las técnicas de predicción de comportamiento de incendios forestales. A.2.1 Detección Óptica La mayoría de las técnicas de detección terrestres se basan en observaciones de campo realizadas por personal de vigilancia de incendios forestales o por sistemas ópticos, ya sea en canal visible o mediante cámaras termográficas. Los sistemas de canal visibles suelen basarse en cámaras de alta resolución que buscan detectar la aparición temprana de humo o llamas. Existen varios algoritmos (Labati et al., 2013; Dimitropoulos et al., 2015) usados sobre imágenes de canal visible digitalizadas para detectar un incendio forestal durante su fase inicial. Por otro lado, las cámaras termográficas habitualmente exploran un área específica de bosque y tratan de detectar puntos calientes por encima de un umbral de temperatura específico y predefinido, lo cual permite determinar la posible ocurrencia de un incendio forestal. Este segundo tipo de soluciones de detección óptica de incendios forestales será el objeto principal de estudio en esta tesis. Los sistemas fijos de detección terrestre suelen estar compuestos por cámaras fijadas en torres de observación u otras infraestructuras similares (Sistema Bosque, 2011), que proporcionan las interfaces de alimentación y comunicación necesarias. Estas cámaras se suelen montar en bases motorizadas que permiten que la cámara se mueva dentro de un determinado rango. El tipo de cámara más utilizada son las cámaras termográficas en el espectro del infrarrojo cercano (NIR), capaces de detectar cambios sutiles en la temperatura para determinar con precisión la presencia de fuego. Con la combinación adecuada de ópticas infrarrojas y sensores adecuados, estas cámaras son capaces de detectar fuego en rangos de hasta varios kilómetros, incluso a través de humo denso o durante la noche. Con el fin de poder vigilar grandes extensiones de bosque se deben instalar varios de estos puntos de observación (FireWatch: Early Detection of Forest Fires for Large Areas, 2014), idealmente modelando el campo de visión de cada cámara de forma individual y teniendo en cuenta sus rangos de movimiento. La presencia de obstáculos circundantes, como valles o colinas también pueden limitar su efectividad. En la figura A.2 se puede ver un ejemplo de una torreta de observación con una cámara termográfica. Este tipo de detección en tierra presenta algunas limitaciones específicas que se analizarán a continuación: • Las cámaras están permanentemente expuestas a las inclemencias meteorológicas, incluyendo altos niveles de humedad, lluvia y polvo. Esto exige un mantenimiento 105 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES FIGURA A.2. Cámara termográfica instalada en una torreta de observación. Fuente: Sistema Bosque, Universitat Politecnica de Valencia (UPV). muy exhaustivo para mantener las cámaras en buen estado de funcionamiento. También son propensas al vandalismo por el hecho de estar expuestas 24 horas al día durante todo el año, incluso cuando no hay riesgo de incendio. • Cada uno de los puntos de observación tiene un campo de visión limitado. Esto exige un gran número de puntos de observación para cubrir un área extensa. Además, debido a que son puntos fijos, no pueden ser reubicados dinámicamente si el fuego cae fuera de su campo de visión, dejando algunos puntos de observación inutilizados. • Cuando se detecta un incendio su geolocalización se realiza manualmente por el personal de extinción de incendios comparando la imagen termográfica con la cartografía específica del zona . Estas tareas suelen complementarse con el uso de Sistemas de Información Geográfica (GIS), pero están expuestas al error humano y por tanto a un cierto grado de subjetividad. • Los costes de mantenimiento de estos puntos de observación suelen ser bastante altos y requieren varios equipos de personal calificado para realizar tareas de mantenimiento. Una técnica complementaria para la detección terrestre consiste en el uso de vehículos con cámaras termográficas instaladas en postes extensibles. Esto aumenta la movilidad y reconfigurabilidad de los puntos de observación, así como la reducción de la posibilidad de daños causados por la meteorología y el vandalismo. Sin embargo, sufren de la misma subjetividad con respecto a la geolocalización de incendios forestales y plantea un alto riesgo para los operadores de vehículos. Dos de los sistemas de detección terrestre comerciales más ampliamente implementados son FireWatch (FireWatch: Early Detection of Forest Fires for Large Areas, 106 A Sistema de Geolocalización Autónoma de Incendios Forestales basado en Técnicas de Visión Sintética y Termográfica 2014) y ForestWatch (ForestWatch Wildfire Detection, 2015). Ambos sufren importantes limitaciones con respecto a su funcionalidad, tal y como se describe en (Ahmad & Alkhalib, 2014). En concreto se determinó que la capacidad de detección de ambos sistemas era en parte función del tamaño y la distancia entre el incendio forestal y la cámara. Ambos sistemas eran capaces de detectar grandes incendios a distancias de 70 km o más, pero no detectaban incendios más moderados a distancias de 10 o 20 km. También presentan algunos problemas a la hora de determinar donde estaban ubicados físicamente los incendios detectados, ya que estos sistemas no utilizan mapas de topografía del terreno con fines de geolocalización. Los sensores ópticos en general, especialmente los de canal visible, también son propensos a un alto nivel de falsas alarmas debido a diversos efectos dinámicos que se observan en los bosques, como el movimiento de árboles debido al viento, sombras de nubes o reflejos y actividad humana. Estas dificultades en el procesamiento de imágenes forestales están derivadas principalmente de la naturaleza variable del paisaje forestal y de los eventos dinámicos que pueden ocurrir bajo diferentes condiciones ambientales, como la meteorología, la distancia, el ángulo del sol, objetos que interfieren en la visión y muchos más. Se han realizado algunas propuestas (Kremens et al., 2003; Fernandez-Berni et al., 2012) para implementar redes de sensores móviles o fijos que sean capaces detectar la ocurrencia de incendios forestales, pero normalmente carecen de cierto grado de reconfigurabilidad dinámica y facilidad de uso. A.2.2 Detección Aérea La detección aérea puede parecer una alternativa plausible para superar algunas de las limitaciones observadas en la detección terrestre. A menudo se utilizan helicópteros e hidroaviones durante el transcurso de incendios forestales, tanto para tareas de extinción de incendios como de vigilancia, ya que tienen la capacidad de rociar agua sobre el fuego y también sirven como puntos de observación a gran altitud. La mayoría de estos helicópteros y aviones están equipados con avanzados sistemas electrónicos y subsistemas de comunicaciones, utilizados principalmente para la supervisión y transmisión de información táctica a bomberos. Algunos de estos sistemas pueden incluir unidades GPS, cámaras multi-espectrales y conectividad a redes de datos móviles entre otros. Las aeronaves que sobrevuelan un incendio forestal proporcionan un campo de visión mucho más amplio y se pueden mover dinámicamente de un lugar a otro a medida que el incendio forestal evoluciona. Esto las hace mucho más versátiles y eficientes que las soluciones usadas en tierra, pero siguen siendo propensas a una serie de limitaciones importantes. La tripulación de la aeronave también se encuentra expuesta a un riesgo significativo debido a la proximidad al fuego. Las corrientes convectivas formadas en estas situaciones y el mal tiempo pueden generar turbulencias locales y afectar gravemente a la estabilidad de la aeronave. Estas circunstancias pueden ser mortales y han sido consecuencia 107 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES directa de muchos accidentes ocurridos en el pasado (Aerial Accidents Report in Wildland Firefighting in Spain, 2015). Como alternativa, los vehículos aéreos no tripulados (UAVs) están comenzando a ser ampliamente utilizados para tareas de monitorización en situaciones de riesgo (Beard et al., 2006). Al ser operados de forma remota o autónoma, eliminan la necesidad de tener una tripulación a bordo. La figura A.3 muestra la plataforma UAV Ikhana (MQ-9 Reaper) de la NASA, que actualmente se utiliza como una herramienta para la detección temprana de incendios forestales y el rastreo en ubicaciones remotas (Ambrosia & Hinkley, 2008). Este UAV está equipado con cámaras de imagen termográfica y puede volar a 15.000 m de altitud durante más de 20 horas. FIGURA A.3. UAV Ikhana (MQ-9 Reaper) Fuente: NASA. Sin embargo, hay una incipiente preocupación a la hora hora de afrontar la pérdida de control de un UAV si está compartiendo el mismo espacio aéreo con aviones tripulados. En caso de que un UAV pierda el control podría chocar contra otro avión cercano y causar un grave accidente. Actualmente las regulaciones con respecto al espacio aéreo no segregado no han sido completamente implementadas y este tipo de situaciones no están permitidas, aunque se está avanzando en la implementación de este tipo de escenarios (Ramalingam et al., 2011). Otras limitaciones típicas de los UAV comercialmente disponibles son su limitada autonomía, baja capacidad de carga útil y su capacidad limitada para soportar fuertes vientos o turbulencias. A.2.3 Redes de Sensores Inalámbricos En lugar de usar imágenes para detectar la ocurrencia temprana de incendios forestales, actualmente está surgiendo una nueva oportunidad basada en el uso de Redes de Sensores Inalámbricos (WSN), tales como los trabajos descritos en (Manatakis & Manolakos, 2011) y (Yoon et al., 2012). Se trata de unidades individuales con subsistemas de detección y comunicación incrustados en el mismo dispositivo y que pueden medir magnitudes físicas como la temperatura, presión y humedad, así como 108 A Sistema de Geolocalización Autónoma de Incendios Forestales basado en Técnicas de Visión Sintética y Termográfica parámetros químicos como el dióxido de carbono y niveles de dióxido de nitrógeno. La mayoría de estos sensores o nodos pueden funcionar de forma autónoma y auto-organizada (Gasull et al., 2011). Una de las tecnologías clave utilizadas para la implementación de WSNs es ZigBee, que está basado en el estándar IEEE 802.15.4 (802.15.4 IEEE Standard for Low Data Rate Wireless Networks, 2005). Esta tecnología utiliza dispositivos alimentados por baterías de bajo coste y en la mayoría de los casos pueden incluso alimentarse con pequeños paneles solares y proporcionar un enlace de comunicaciones con un servidor centralizado capaz de procesar las variables medidas y realizar cálculos adicionales para activar alarmas bajo situaciones predefinidas. Esta solución tiene el potencial de poder ser aplicada en muchos escenarios diferentes. Cada uno de los nodos debe ser suficientemente económico para que un gran número de ellos pueda ser desplegado en áreas extensas. Esto está vinculado al hecho de que su principio de funcionamiento se basa en la observación de un punto en particular en lugar de percibir una visión global de una gran extensión de bosque, como suele ser el caso con los sistemas de detección óptica terrestre. También es muy posible que algunos de los nodos sean destruidos por el incendio en sí mismo, por lo que se debe tener en cuenta una posible pérdida de elementos de detección y diseñar este tipo de soluciones con margen suficiente en términos de redundancia. Con el fin de poder utilizar este tipo de sensores como un mecanismo de detección temprana de incendios forestales, estos también deben ser capaces de proporcionar altos niveles de reactividad, fiabilidad y robustez. Para lograr estas características, tanto la propia unidad como la infraestructura de telecomunicaciones que le da soporte deben estar bien diseñadas y dejar margen suficiente en términos de escalabilidad y redundancia. Esto también significa que cada sensor nunca es completamente autónomo por sí mismo, sino que se basa en el uso de nodos centrales que actúan como un relay de información o en una infraestructura de telecomunicaciones dedicada. Su facilidad de replicación y su despliegue extenso permiten cubrir grandes extensiones de terreno. Sin embargo, esto incorpora una alta complejidad en cuanto a sus tareas de mantenimiento. La mayoría de estos sensores estarán normalmente ubicados en lugares remotos o poco accesibles, requiriendo de un trabajo bastante laborioso para poder realizar un mantenimiento periódico, como reemplazar las baterías de los mismos. También presentan el mismo nivel de exposición al vandalismo que otras soluciones estáticas. A.2.4 Teledetección Espacial La detección y seguimiento de incendios forestales utilizando imágenes por satélite y detección multi-espectral se ha utilizado habitualmente cuando existen grandes extensiones de tierra o bosque afectadas por un incendio forestal de gran escala. Varios satélites actualmente en órbita cuentan con un equipos especiales que sirve para este propósito (Ganesan et al., 2016; Veraverbeke et al., 2014). Algunos ejemplos de estas familias de satélites incluyen el Global Monitoring for Environment and Security (GMES) de la ESA o el Moderate Resolution Imaging Spectroradiometer (MODIS) de la NASA. Por desgracia, estos satélites sólo pueden proporcionar imágenes de las regiones de 109 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES interés cada dos días aproximadamente, además del hecho de que la calidad de las imágenes proporcionadas por satélite también puede verse afectada por condiciones climáticas locales (Mitsopoulos et al., 2016). La figura A.4 muestra una imagen de satélite de los incendios forestales de California ocurridos en 2012, obtenidos por el satélite Terra (EOS AM-1) de la NASA usando su instrumento MODIS. FIGURA A.4. Incendios de California en 2012 detectados por el satélite Terra (EOS AM-1) de la NASA. Fuente: NASA (EOS AM-1). La mayoría de las soluciones de observación y seguimiento de incendios forestales por satélite presentan algunas limitaciones clave que afectan a su capacidad para detectar incendios. La mayoría de estas limitaciones se basan en el hecho de que los satélites de observación de la Tierra se localizan generalmente en órbitas geoestacionarias (GEO) o en órbitas terrestres de baja altitud (LEO), lo que significa que pueden estar a una altitud de más de 35.000 km sobre la superficie de la Tierra. En estas circunstancias, la radiación óptica e infrarroja emitida por los incendios forestales en sus primeras etapas puede llegar a ser demasiado débil como para ser detectada correctamente por un satélite. Teniendo en cuenta que la intensidad de la energía térmica radiada disminuye dramáticamente en términos de distancia, el ángulo entre la dirección normal del instrumento de detección del satélite y el haz de radiación es crítico. Incluso el menor desajuste en la posición o orientación de un satélite puede afectar negativamente la detección de un incendio forestal, especialmente durante su fase inicial. Otro inconveniente habitualmente asociado con la teledetección espacial es el bajo nivel de inmediatez con respecto a la adquisición de imágenes. Las imágenes por satélite suelen tardar bastante tiempo en ser procesadas y no suelen estar disponibles hasta varias horas después de su adquisición, lo que las convierte en una herramienta muy limitada cuando surge la necesidad tomar decisiones de forma rápida. Los periodos de revisita de este tipo de satélites también pueden resultar bastante 110 A Sistema de Geolocalización Autónoma de Incendios Forestales basado en Técnicas de Visión Sintética y Termográfica largos dependiendo de la órbita en la que se encuentren. Esto significa que el satélite únicamente pasará sobre el área de interés un par de veces al día. Esta baja periodicidad es un factor bastante limitante para poder seguir adecuadamente la evolución del incendio forestal a lo largo del tiempo. Además, un satélite ubicado en una órbita GEO cubrirá permanentemente un área específica de tierra, pero no podrá ser usado para rastrear otras áreas. Un satélite capaz de cubrir varias áreas de la Tierra en términos de monitorización de incendios forestales implicaría la colaboración entre varias agencias o entidades oficiales en diferentes países, lo cual suele ser una tarea política muy difícil. Por lo general, este tipo de detección sólo es útil para incendios forestales de grandes dimensiones, ya que este tipo de satélites no son capaces de proporcionar ninguna información útil para incendios más pequeños. También se utilizan como una herramienta de análisis a posteriori para evaluar el impacto global que los incendios hayan podido tener sobre una gran extensión de tierra. Sin embargo, sí que proporcionan una capacidad de geolocalización automática, pero la resolución espacial proporcionada no suele ser lo suficientemente precisa como para servir de herramienta de planificación estratégica a la hora de combatir incendios forestales. A.2.5 Sistemas Predictores Existen varios sistemas que permiten la predicción del comportamiento de incendios forestales mediante el modelado de la propagación del incendio sobre un área específica (Papadopoulos & Pavlidou, 2011; Filippi et al., 2014). Esto se puede lograr utilizando puntos de fuego conocidos como datos de entrada a los algoritmos del software utilizado para realizar la predicción. Para que estos algoritmos funcionen correctamente, tanto los datos de entrada como otras variables meteorológicas deben actualizarse regularmente, ya que se utilizan para modelar y estimar la posible propagación del fuego durante un cierto lapso de tiempo. La propagación de incendios forestales es un fenómeno espacial que depende en gran medida de múltiples parámetros. Estos parámetros incluyen no sólo las condiciones meteorológicas, sino también la información geográfica y topográfica con respecto a la topología del área donde se están aplicando. Esto es aún más complicado si tenemos en cuenta el hecho de que la predicción meteorológica suele ser propensa a errores, especialmente cuando existen condiciones meteorológicas extremas. En base a estos desafíos han surgido una serie de proyectos internacionales basados en el desarrollo de simuladores de propagación de incendios forestales. Uno de estos proyectos es Prometheus (Tymstra et al., 2009). Se trata de un un software canadiense de código abierto utilizado para la simulación determinista del crecimiento de un incendio forestal. Utiliza datos topográficos y tipos de combustible junto con información meteorológica aplicada al área de interés. El modelo implementado simula el crecimiento del fuego usando un principio de propagación de ondas, donde el propio frente del incendio se considera un frente de onda. Sin embargo, la mayoría de los datos de entrada utilizados por Prometheus permanecen estáticos durante el proceso de simulación. Existe un sistema alternativo llamado Phoenix (Tolhurst et al., 2008) desarrollado en Australia que puede responder dinámicamente a los cambios en las condiciones 111 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES topográficas, de combustible y ambientales a medida que el fuego se mueve a lo largo del terreno, proporcionando así una simulación más realista y precisa. Phoenix realiza un mallado del terreno, donde cada celda puede llegar a representar un área de tan solo 5 m para proporcionar un análisis muy detallado. Sin embargo, este tipo de resolución puede imponer altos requerimientos de potencia computacional y actuar como un detrimento en entornos en tiempo real. Un tamaño de mallado de entre 100 y 200 m suele ser considerado aceptable para la mayoría de las situaciones, pero incluso con esta simplificación tampoco podría llegar a utilizarse en entornos en tiempo real. Por lo general, el intervalo entre dos determinaciones consecutivas de la propagación del fuego puede durar entre 1 y 15 minutos en incendios de movimiento lento, lo que hace que el sistema resulte realmente limitado en la toma de decisiones. A pesar de las limitaciones anteriores, hay muchos otros simuladores y modelos actualmente en desarrollo. Sin embargo, uno de ellos ha ganado el reconocimiento de muchas agencias gubernamentales y actualmente se considera como el mejor modelo para predecir el crecimiento y el comportamiento de incendios forestales. Este sistema ha sido desarrollado por Estados Unidos y se conoce como FARSITE (Finney, 1995; FARSITE: Fire Area Simulator-Model Development and Evaluation, 1998). Se basa en algunos principios de diseño tanto de Phoenix como de Prometheus, especialmente en lo que respecta al análisis de propagación de ondas, pero a diferencia de ellos es capaz de manejar escenarios mucho más complejos como incendios forestales con múltiples frentes de fuego y escenarios con cambios bruscos en la dirección del viento o transiciones de combustible o de tipo de terreno. La figura A.5 muestra una representación gráfica de una propagación de incendio forestal calculada por FARSITE. FIGURA A.5. Predicción de la propagación de un incendio forestal utilizando FARSITE. Fuente: RedfishGroup. Para que FARSITE funcione correctamente es necesario utilizar una base de datos de combustible y vegetación que deben ser aplicadas como entradas al sistema y georreferenciadas correctamente con otras características topológicas del terreno. Esto suele ser una tarea difícil, ya que la mayoría de los países carecen de información precisa sobre este tipo de detalles del terreno, o al nivel de resolución requerido por FARSITE. El principal problema asociado con la mayoría de los sistemas descritos anteriormente es que las predicciones generadas tienden a ser muy imprecisas si el área donde se está produciendo el incendio forestal tiene un gradiente pronunciado en términos de variables meteorológicas. Esto suele suceder en áreas de terreno abrupto, o si sólo 112 A Sistema de Geolocalización Autónoma de Incendios Forestales basado en Técnicas de Visión Sintética y Termográfica se dispone de topografía limitada o inexacta. Además, estos sistemas necesitan ser suministrados de forma constante y periódica con datos nuevos y actualizados que contengan localizaciones de fuego activo o parámetros ambientales. Si esto no se realiza de forma adecuada puede ocurrir que los algoritmos acaben divergiendo y produciendo soluciones que se alejen de la realidad. Dependiendo de las condiciones geográficas y ambientales, las predicciones basadas en modelos pueden ser poco precisas (Alexander & Cruz, 2013). Si una predicción errónease utiliza durante las tareas de extinción podría tener consecuencias catastróficas. Normalmente, los datos de entrada son proporcionadospor el personal de extinción de incendios que identifica visualmente la ubicación de los puntos y introduce manualmente estos datos en el software de predicción. Como se ha comentado anteriormente, esto puede añadir un factor de subjetividad, además de ser una tarea ardua y arriesgada para el personal involucrado. El sistema RDMU desarrollado como parte de esta tesis también puede servir como una fuente de datos de entrada en tiempo real para un software de predicción. Esto permitiría tener un acceso más preciso y rápido a una representación georreferenciada automáticamente del incendio forestal y, por lo tanto, ayudará a mejorar la precisión y la velocidad de convergencia de los modelos utilizados. A.3 Termografía Con el fin de obtener una imagen termográfica que presente mediciones precisas de temperatura de un objeto o escena en particular, es necesario hacer un análisis de varios parámetros radiométricos. Entre los muchos parámetros radiométricos implicados, dos de ellos han sido identificados como de especial interés: la emisividad del objeto y transmisividad atmosférica. Ambos serán presentados y discutidos dentro de esta sección. Las cámaras termográficas aplican calibraciones radiométricas a toda la imagen basadas en los parámetros radiométricos anteriormente mencionados. Sin embargo, esto sólo es válido si el objeto o escena está completamente contenido dentro de una distancia específica y conocida con respecto a la cámara termográfica. Este no es el caso cuando se obtiene una imagen termográfica de un terreno distante, donde el primer plano puede estar ubicado a pocos metros de la cámara, pero otras zonas de la imagen puede estar a varios kilómetros. Este aspecto requiere una técnica de calibración radiométrica a nivel de píxel para lograr mediciones de temperatura precisas, la cual se ha desarrollado e implementado como parte de esta tesis. A.3.1 Radiación Térmica Cualquier objeto o con una temperatura superior al cero absoluto emitirá ondas electromagnéticas. Esto se debe al movimiento de los átomos y moléculas inducido por la energía térmica asociada con un aumento de temperatura. Este movimiento de partículas cargadas genera campos eléctricos y magnéticos acoplados que producen radiación electromagnética, llamada radiación térmica. La radiación térmica representa 113 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES FIGURA A.12. Cámara termográfica FLIR A615. TABLA A.2. Características ópticas de la cámara termográfica FLIR A615. Propiedad Valor Campo de Visión Horizontal (HFOV) 25◦ Campo de Visión Vertical (VFOV) 18.8◦ Distancia Focal 24.5 mm Número de Píxeles Horizontales 640 Número de Píxeles Verticales 480 • Yaw (α): 0◦a 360◦. El ángulo de yaw está referido al norte magnético, por lo que el valor de α=0◦está alineado con el norte magnético local. Será necesario realizar un ajuste de declinación magnética para transformar este valor en un rumbo referido al norte geográfico. • Pitch (θ): -90◦a 90◦. Desde la perspectiva de la cámara, un ángulo de pitch negativo significa que la cámara está inclinada hacia abajo desde la horizontal y hacia arriba para un ángulo de pitch positivo. • Roll (φ): -180◦a 180◦. Desde la perspectiva de la cámara, un ángulo de roll negativo significa que la cámara se encuentra girada en el sentido de las agujas del reloj y en sentido contrario a las agujas del reloj para un ángulo de roll positivo. El proceso para calcular la imagen sintética se puede describir por medio de los siguientes pasos: 1. La unidad GPS proporciona los datos de ubicación en forma de latitud, longitud y altitud. La componente de altitud se descarta y se sustituye por la altitud del modelo DEM local. 120 A Sistema de Geolocalización Autónoma de Incendios Forestales basado en Técnicas de Visión Sintética y Termográfica 2. La IMU proporciona los ángulos de pitch, roll y yaw de la cámara termográfica con respecto al sistema de referencia inercial. La declinación magnética se calcula utilizando el WMM2015 basado en los datos de ubicación proporcionados por el GPS y se aplica a la componente de yaw. 3. Los datos de latitud, longitud, altitud, pitch, roll y yaw se introducen en el algoritmo de generación de imágenes sintéticas junto con los parámetros FOV e IFOV (estos últimos permanecen constantes). Posteriormente, se obtiene una imagen sintética con la misma resolución y parámetros FOV/IFOV que la imagen termográfica. Cada píxel de la imagen sintética contiene información de latitud, longitud y altitud que permiten su geolocalización sobre el terreno. 4. Ambas imágenes (sintética y termográfica) se pueden utilizar llegado este momento para acceder a la información contenida en ambas. La fusión de ambas imágenes podría considerarse como una matriz multidimensional que contiene información de ubicación y temperatura para cada píxel. 5. La distancia desde la cámara a cada uno de los píxeles de la imagen sintética se calcula y se utiliza para calcular valores de transmisividad atmosférica específicos para cada uno de los píxeles, que se utilizarán para realizar correcciones de temperatura como se describe en el Capítulo 3. A.5 Relay de Datos Críticos Multimodo Uno de los principales problemas durante las situaciones de emergencia es que la mayoría de los enlaces de telecomunicaciones disponibles se ven afectados por congestión o incluso por interrupciones del servicio debido que infraestructuras clave resultan dañadas. Las infraestructuras de red móvil suelen ser uno de los primeros servicios en colapsarse durante situaciones de emergencia, ya que las personas tienden a hacer un uso muy exhaustivo de sus teléfonos móviles durante un intervalo de tiempo muy corto. Un sistema vital que está se encuentra continuamente generando datos para proporcionar información acerca de un incendio, tal como la unidad RDMU, no puede depender solamente de un único método de comunicaciones para transmitir la información generada Bajo circunstancias nominales, una vez que la unidad RDMU se encuentra operativa y la evolución del incendio forestal está siendo georreferenciada correctamente, la información generada se debe enviar regularmente a un centro de control remoto donde se utilizará para coordinar los equipos de extinción. Con el fin de garantizar que este enlace de comunicación se mantiene incluso en condiciones adversas, se ha implementado un gestor de telemetría y telecomando multimodo. En las siguientes secciones se analizan las tecnologías de telecomunicación existentes para entornos rurales, así como la funcionalidad prevista y la implementación técnica del gestor de comunicaciones. 121 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES A.5.1 Sistemas de Telecomunicación en Entornos Rurales Esta sección ofrece una breve descripción de las principales tecnologías de telecomunicación inalámbrica que han sido consideradas dentro del desarrollo de la presente tesis para la transmisión de datos en entornos rurales, ya que éste será el principal entorno operacional de la RDMU. Cada uno de ellos presenta algunas ventajas y desventajas específicas en términos de cobertura, volumen de datos, coste y requisitos de infraestructura. A.5.1.1 Enlaces en VHF y Microondas La comunicación directa por radio utilizando módems VHF es una forma sencilla de realizar comunicaciones de datos y voz en entornos rurales (Dagefu et al., 2015; Seker, 1992). Debido a la banda de frecuencias utilizada (30 MHz a 300 MHz) estos enlaces VHF pueden alcanzar distancias de varios kilómetros y salvar obstáculos como montañas y valles. Dependiendo de las distancias a cubrir, un único enlace VHF punto a punto podría ser suficiente para llegar hasta a una instalación operativa centralizada que pueda transmitir los datos por otros medios. Si se necesita cubrir distancias más grandes se pueden incorporar repetidores de VHF. El principal problema con las comunicaciones de datos VHF es que el volumen de datos que se pueden enviar con módems VHF es bastante bajo, normalmente en el rango de 10 kb/s en la mayoría de los casos. Además, estos módems de radio no suelen ser muy eficientes en términos de consumo energético y requieren varios vatios de potencia para maximizar su distancia operativa. La figura A.13 muestra un ejemplo de un módem de radio VHF que podría utilizarse para implementar este tipo de enlaces. FIGURA A.13. Módem de datos VHF. La limitación de los módems VHF podría salvarse mediante el uso de enlaces de microondas SHF punto a punto (Goktas et al., 2016), capaces de proporcionar velocidades de aproximadamente 10 Mb/s a una frecuencia de 3 GHz. Sin embargo, esto requiere un cierto grado de infraestructura y el correcto apuntamiento de las antenas de microondas entre ambos extremos del enlace. Un enlace de estas características afectaría a la idea de tener una unidad RDMU que pudiese operar en cualquier lugar sin requerir un alto nivel de ajustes manuales. 122 A Sistema de Geolocalización Autónoma de Incendios Forestales basado en Técnicas de Visión Sintética y Termográfica A.5.1.2 VSAT VSAT es una tecnología de comunicaciones por satélite (Weerackody & Gonzalez, 2007) utilizada para el acceso a Internet por estaciones terrestres o terminales que utilizan pequeñas antenas parabólicas con diámetros de entre 75 cm y 1,2 m. Las velocidades de transferencia de datos que se pueden alcanzar con esta tecnología son amplias y pueden variar entre los 4 kbit/s y los 16 Mbit/s. La mayoría de satélites que ofrecen servicios VSAT se encuentran en órbitas GEO y por tanto aparentan estar en una ubicación fija en el cielo con respecto a la estación terrestre. Las estaciones terrestres necesitan un transceptor VSAT que sirva como una pasarela de acceso y se comunique con el satélite que ofrece el servicio. Los terminales VSAT pueden utilizarse para la transmisión y recepción de voz, datos o vídeo, así como para el acceso a Internet para aplicaciones TCP/IP. VSAT tiene el beneficio de no requerir una infraestructura de tierra específica y proporciona acceso inmediato a Internet para el relay de datos. Sin embargo, debido al hecho de que los satélites VSAT están ubicados en órbitas GEO se vuelve necesario un apuntamiento preciso de la antena del terminal terrestre. Esto puede resultar complicado si el transceptor está en un valle o en las proximidades de un gran obstáculo, como un edificio alto o una montaña. El tamaño de la antena parabólica es también un factor limitante en términos de portabilidad, ya que incluso una pequeña antena parabólica con un diámetro de 75 cm puede ser bastante voluminosa y choca con la filosofía portátil prevista para la RDMU. También debe considerarse el factor de coste, ya que las tarifas de suscripción para este tipo de servicio son bastante altas. A.5.1.3 Iridium Iridium es una constelación de satélites compuesta por 66 satélites (Figura A.14) utilizados principalmente para proporcionar comunicaciones de voz y datos a nivel mundial utilizando tanto teléfonos móviles como pequeños transceptores. Uno de sus ventajas es que puede dar servicio a toda la Tierra, incluyendo los polos, que suele ser el área donde otras constelaciones carecen de cobertura de servicio adecuada. El nombre de la constelación proviene del elemento químico Iridio, debido al hecho de que la constelación iba a contar inicialmente con 77 satélites y este es precisamente el número atómico del Iridio. Los satélites Iridium están situados en seis órbitas LEO polares a una altitud de aproximadamente 780 km como se ve en la figura A.15. Aparte de los 66 satélites activos, también hay satélites de reserva que se mantienen también en este tipo de órbita para servir como unidades redundantes en caso de fallo. Se utiliza banda Ka para las comunicaciones entre satélites y banda L para las comunicaciones terrestres. El uso de la banda L para las comunicaciones terrestres significa que los transceptores Iridium generalmente necesitan tener una visión directa del cielo para que las transmisiones de datos se den correctamente. 123 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES FIGURA A.14. Satélite Iridium. FIGURA A.15. Planos orbitales de Iridium. 124 A Sistema de Geolocalización Autónoma de Incendios Forestales basado en Técnicas de Visión Sintética y Termográfica Aunque Iridium puede ofrecer tanto servicios de voz como de datos, nos centraremos únicamente en sus servicios de transmisión de datos, ya que es la parte clave del subsistema de relay de datos de emergencia implementado durante la presente tesis. Los servicios de transmisión de datos de Iridium se pueden utilizar para una amplia gama de aplicaciones que requieran de una transmisión de datos de bajo ancho de banda para fines de telecomando o telemetría, tales como aplicaciones remotas de registro de datos o monitorización. Un ejemplo de este tipo de aplicación podrían ser los sistemas de alerta de tsunamis, donde una serie de boyas flotantes utilizan satélites Iridium para comunicarse con una estación base y activar una alarma en caso de tsunami. Uno de los servicios de transmisión de datos más utilizados por Iridium es el servicio SBD (Short Burst Data), que puede considerarse como un caso particular de protocolo M2M para transmitir mensajes cortos definidos por el usuario. El tamaño de estos mensajes puede variar entre 270 y 340 bytes y la latencia de la red oscila entre unos segundos y un minuto para la transmisión de un mensaje SBD. Iridium proporciona una solución global para el relay de datos y comunicaciones de voz. Sin embargo, una de sus principales limitaciones está vinculada al hecho de que los transceptores Iridium necesitan tener una visión directa del cielo para llegar a enlazar correctamente con la constelación de satélites. Esto a veces puede resultar un factor limitante si el transceptor necesita ser utilizado en un ambiente rural bajo árboles o en valles que podrían bloquear la línea de visión directa entre el transceptor y el satélite. También tiene un alto coste por unidad de datos transmitidos y por lo tanto sólo sería aconsejable utilizarlo para la transmisión de datos de bajo volumen que no puedan transmitirse por ningún otro medio. A.5.1.4 TETRA TETRA o TErrestrial Trunked RAdio es un estándar de telecomunicaciones de radio móvil definido por el Instituto Europeo de Normas de Telecomunicaciones (ETSI) (Terrestrial Trunked Radio (TETRA) Voice plus Data (V+D), 1997) para gestionar redes PMR y PAMR. Los principales usuarios de TETRA son generalmente los cuerpos de seguridad del estado, unidades de respuesta de emergencia, bomberos y muchos otros servicios críticos de seguridad pública. Algunas de las muchas ventajas de TETRA incluyen una gran flexibilidad, seguridad y capacidad de cifrado, facilidad de uso y amplia cobertura. El desarrollo de esta tecnología se inició a principios de los años 90 como una iniciativa de la la Unión Europea con el objetivo de combinar las características de los muchos sistemas de comunicación móvil de radio y servicios disponibles en ese entonces. De esta manera, TETRA fue diseñado para proporcionar un mejor servicio a los usuarios de PMR y PAMR, así como para para homogeneizar los primeros sistemas PMR. TETRA es ampliamente utilizado por casi todos los países y es el método preferido para retransmitir comunicaciones críticas entre diferentes organismos de respuesta en emergencias. Una arquitectura típica de TETRA se puede ver en la figura A.16. La arquitectura TETRA se asemeja a una topología de red móvil, donde las estaciones base son utilizadas para enrutar el tráfico desde los terminales móviles y proporcionar una serie de servicios 125 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES a toda la red. FIGURA A.16. Ejemplo de la arquitectura de red típica en TETRA. Los terminales TETRA se conectan a la estación base (BS) a través de una interfaz aérea o air interface. Varias BSs dentro de un área regional están conectadas por un intercambiador digital TETRA (DXT). Esta conexión se implementa siguiendo una topología de anillo, donde ambos extremos de anillo terminan en dos DXT diferentes de manera que al menos una vía de conexión esté siempre disponible. La conexión a un centro de control TETRA se realiza a través de tres tipos de interfaces o servicios de enlace (LS) desde un DXT: • LS-1: Transmisión punto a punto de voz encriptada. • LS-2: Direccionamiento de equipos y tráfico SDS, incluyendo enlaces con protocolo TCP/IP. • LS-3: Monitorización de la red. Entre la gran variedad de servicios de voz y datos ofrecidos por TETRA, hay uno en particular que es de interés para las transferencias de datos de bajo volumen, el Short Data Service (SDS). Se trata de un servicio de transmisión de datos proporcionado por TETRA para transmitir mensajes cortos comparables al caso de SMS dentro de GSM. Un mensaje TETRA SDS puede transportar hasta 140 bytes por mensaje, lo que puede parecer inferior a los 340 bytes de Iridium, pero la latencia de una red TETRA es mucho menor (del orden de decenas de segundo), por lo que la tasa de transferencia que se puede alcanzar es significativamente mayor que en el caso de Iridium. TETRA parece ser el sistema de telecomunicaciones ideal para ser integrado con otros servicios de emergencia, ya que la mayoría de los países y regiones ya cuentan 126 A Sistema de Geolocalización Autónoma de Incendios Forestales basado en Técnicas de Visión Sintética y Termográfica con una infraestructura TETRA existente y perfectamente integrada con los servicios de seguridad. Sin embargo, el despliegue de una infraestructura TETRA es una tarea de ingeniería importante y de elevado coste, que puede requerir de varios años de estudio e implementación hasta ser completamente operativa. A.5.1.5 DMR Digital Mobile Radio (DMR) es un estándar desarrollado por el Instituto Europeo de Normas de Telecomunicaciones (ETSI), que establece una especificación de radio digital para usuarios profesionales, comerciales y privados (Digital Mobile Radio (DMR)), 2016). DMR es un sistema de acceso múltiple por división en el tiempo (TDMA) que ofrece soluciones de voz y datos digitales flexibles y de bajo coste y utiliza un esquema de modulación FSK. Puede proporcionar comunicaciones de largo alcance sobre una banda de frecuencia que oscila entre los 30MHz y 1GHz y también soporta la transmisión de paquetes de datos o mensajes. DMR es un sistema escalable que se puede utilizar tanto con o sin licencia. Está sujeto a la planificación nacional de frecuencias y se ha desarrollado en tres niveles o tiers: • Tier 1: Servicio PMR446 exento de licencia. • Tier 2: Diseñado para aplicaciones profesionales incluyendo conexiones punto a punto y modo repetidor. Requiere el uso de licencia. • Tier 3: Operación en modo trunking. También requiere licencia. DMR ha sido diseñado principalmente para ubicarse en las bandas existentes de PMR con licencia, en un esfuerzo por favorecer la transición de servicios analógicos a digitales. La nueva norma no impone cambios fundamentales en la arquitectura de radiocomunicación profesional y ha sido considerada como una alternativa de bajo costo a TETRA, especialmente para los usuarios que no necesitan la complejidad de una infraestructura TETRA convencional. Con respecto a la transmisión de datos de tipo M2M y en comparación con el protocolo SDS anteriormente mencionado, existe un protocolo de transferencia de datos específico para DMR conocido como Packet Data Protocol o (PDP). Este protocolo soporta la transmisión de paquetes de datos cortos de hasta 1.508 bytes y también ofrece capacidad de implementar servicios TCP/IP. La implementación de una red DMR es mucho más simple que una red TETRA, ya que sólo necesita una serie de estaciones base interconectadas entre sí. Sin embargo, este tipo de sistema de radiocomunicación no ha alcanzado la popularidad de TETRA debido a su menor fiabilidad y bajo nivel de cifrado, así como a la baja redundancia a nivel de enlaces dentro de su arquitectura de red. 127 AUTONOMOUS WILDFIRE GEOLOCATION SYSTEM BASED ON THERMOGRAPHIC AND SYNTHETIC VISION TECHNIQUES A.5.1.6 Banda Ancha Móvil La banda ancha móvil en cualquiera de sus generaciones 2G, 3G o 4G es sin duda la red de telecomunicaciones más extendido hoy en día, sobre todo después de la popularización de los smartphone a principios de los años 2000. Esta red es utilizada todos los días por millones de usuarios en todo el mundo y tiene la ventaja añadida de tener la infraestructura más predominantemente desplegada en casi todos los países. La banda ancha móvil proporciona un ancho de banda de datos elevado a un coste bastante razonable, con tecnologías de acceso y dispositivos avanzados ampliamente disponibles en el mercado. También tiene la capacidad de implementar comunicaciones TCP/IP completas, por lo que no sólo la transmisión y recepción de datos de telemetría y telecomando es bastante sencillo y fiable, sino que también se pueden transmitir imágenes en alta resolución y vídeo en tiempo real. No está dentro del alcance de esta tesis proporcionar una descripción detallada de cómo funcionan las redes de banda ancha móvil, pues es una tecnología ampliamente adoptada que se ha cubierto extensamente en literatura, notas técnicas y estándares (Vriendt et al., 2002). A.5.1.7 WiMAX WiMAX o Worldwide Interoperability for Microwave Access pertenece a la familia de estándares de comunicación inalámbrica IEEE 802.16 (IEEE 802.16 Broadband Wireless Metropolitan Area Networks (MANs), 2012). Esta tecnología se ha considerado comúnmente como una alternativa inalámbrica a los servicios DSL o incluso como una posible alternativa a la implementación 4G de banda ancha móvil. Se asemeja en estructura y funcionalidad similar a la WiFi clásica o IEEE 802.11, pero con una serie de características clave que la diferencian, como su mayor alcance e infraestructura más robusta. Las velocidades de transferencia de datos que pueden alcanzarse con WiMAX pueden llegar hasta 1 Gb/s. Los costes de despliegue de una arquitectura WiMAX también son mucho más bajos que los de las arquitecturas de banda ancha móvil, por lo que resulta económicamente viable para proporcionar acceso a Internet de banda ancha en áreas rurales o remotas. WiMAX puede proporcionar enlaces de comunicación a distancias superiores a 50 km en comparación con los aproximadamente 100 m proporcionados por WiFi. Sin embargo, a medida que aumenta la distancia la velocidad que se puede lograr con WiMAX disminuye, pero sigue pudiendo proporcionar velocidades alrededor de 10 Mb/s a distancias de 10 km. Aunque WiMAX y WiFi están diseñados para entornos y situaciones diferentes, también son complementarios entre sí, ya que WiMAX puede proporcionar el enlace de largo alcance y luego depender de una conexión WiFi para interconectar varios equipos locales. WiMAX requiere un cierto grado de infraestructura para que la red funcione correctamente, especialmente con respecto a la instalación de las estaciones de abonado o Subscriber Stations (SS) que proporcionan acceso a la red. Aunque es una tecnología muy conocida, aún no ha sido ampliamente adoptada y la gama de módems disponibles 128 A Sistema de Geolocalización Autónoma de Incendios Forestales basado en Técnicas de Visión Sintética y Termográfica es relativamente limitada. A.5.2 Gestor de Comunicaciones Este gestor de comunicaciones implementado funciona bajo Linux dentro de un ordenador central que controla la RDMU y es capaz de seleccionar entre tres tipos diferentes de enlaces de comunicación basados en un criterio preestablecido. Los tres sistemas de telecomunicación que finalmente se han elegido para el transmisión de datos son Iridium SBD, TETRA SDS y banda ancha móvil. Los criterios de selección establecidos para estos tres sistemas o servicios responden a varias características únicas. A continuación se exponen las características clave que apoyaron la selección de estas tecnologías para la implementación del gestor de comunicaciones utilizado en la RDMU: •Banda Ancha Móvil: Popularidad de la banda ancha móvil, tarifas de transferencia de datos de alta velocidad y una amplia variedad de módems disponibles. •Servicio SDS de TETRA: Ofrece una integración segura, fiable e integrada con muchos servicios oficiales de respuesta a emergencias, así como una amplia adopción y disponibilidad actual de infraestructuras TETRA en muchos países y regiones. La red TETRA de RESCAN en Canarias ha sido utilizada específicamente durante el desarrollo de esta tesis. El servicio SDS proporciona una capacidad de transmisión de datos relativamente baja, pero puede verse compensada con la fiabilidad que ofrece la red y la baja latencia de la misma. •Servicio SBD de Iridium: Cobertura global y totalmente independiente libre de infraestructura terrestre. No se basa en el uso de satélites GEO y por tanto no requiere de un apuntamiento preciso de la antena del transceptor. Los paquetes SBD tienen una capacidad limitada pero pueden ser la única forma de transmitir datos si fallan los otros dos sistemas. La idea detrás del módulo gestor de comunicaciones implementado es proporcionar un sistema resiliente capaz de transmitir y recibir datos de forma transparente hacia y desde la unidad RDMU. La figura A.17 ilustra el principio detrás del distribuidor de datos. FIGURA A.17. Descripción del gestor de comunicaciones multimodo. 129