MODIS Sensor Capability to Burned Area Mapping¿Assessment of Performance and Improvements Provided by the Latest Standard Products in Boreal Regions

sensors Article MODIS Sensor Capability to Burned Area Mapping—Assessment of Performance and Improvements Provided by the Latest Standard Products in Boreal Regions JoséA. Moreno-Ruiz 1, JoséR. García-Lázaro 1, Manuel Arbelo 2,* and Manuel Cantón-Garbín1 1Departamento de Informática, Universidad de Almería, 04120 Almería, Spain; [email protected] (J.A.M.-R.); jrgar[email protected] (J.R.G.-L.); [email protected] (M.C.-G.) 2Departamento de Física, Universidad de La Laguna, 38200 San Cristóbal de La Laguna, Spain *Correspondence: [email protected] Received: 10 August 2020; Accepted: 20 September 2020; Published: 22 September 2020   Abstract: This paper presents an accuracy assessment of the main global scale Burned Area (BA) products, derived from daily images of the Moderate-Resolution Imaging Spectroradiometer (MODIS) Fire_CCI 5.1 and MCD64A1 C6, as well as the previous versions of both products (Fire_CCI 4.1 and MCD45A1 C5). The exercise was conducted on the boreal region of Alaska during the period 2000–2017. All the BA polygons registered by the Alaska Fire Service were used as reference data. Both new versions doubled the annual BA estimate compared to the previous versions (66% for Fire_CCI 5.1 versus 35% for v4.1, and 63% for MCD64A1 C6 versus 28% for C5), reducing the omission error (OE) by almost one half (39% versus 67% for Fire_CCI and 48% versus 74% for MCD) and slightly increasing the commission error (CE) (7.5% versus 7% for Fire_CCI and 18% versus 7% for MCD). The Fire_CCI 5.1 product (CE =7.5%, OE =39%) presented the best results in terms of positional accuracy with respect to MCD64A1 C6 (CE =18%, OE =48%). These results suggest that Fire_CCI 5.1 could be suitable for those users who employ BA standard products in geoinformatics analysis techniques for wildfire management, especially in Boreal regions. The Pareto boundary analysis, performed on an annual basis, showed that there is still a potential theoretical capacity to improve the MODIS sensor-based BA algorithms. Keywords: remote sensing: burned area; wildfire; MODIS; MCD45A1; MCD64A1; fire_CCI; pareto boundary 1. Introduction Wildfires cause deforestation and habitat loss, and they are responsible for releasing a huge amount of aerosol particles and greenhouse gases into the atmosphere. These emissions vary depending on the Burned Area (BA) extension and on the type of biomass present in the region where the fire occurs. For example, Equatorial Asia, which is responsible for only 0.6% of the Global Burned Area (GBA), generates CO 2 and CH 4 emissions of 8% and 23%, respectively. Meanwhile, boreal forests, responsible for 2.5% of GBA, emit 9% of global CO 2 and 15% of CH 4 emissions [ 1 ]. Annually, between 5 and 15 million ha are burned in boreal forests, mainly in Siberia, Canada and Alaska, and the projections of different climate models estimate from decreases to increases in BA, which, as suggested by Kitzberger et al. [ 2 ], generates uncertainty in boreal regions, where global warming may create contrary effects. Sensors 2020,20, 5423; doi:10.3390/s20185423 www.mdpi.com/journal/sensors Sensors 2020,20, 5423 2 of 23 An accurate BA estimation is therefore essential for predicting changes in the global climate system, increased greenhouse gas concentrations or the changing chemical composition of the atmosphere due to fire emissions. Detailed spatial and temporal knowledge of BA is also essential in dynamic global vegetation models (DGVM) where, together with other geospatial data, many ecological variables can be quantified and projected. We should also not forget the importance of BA maps, in combination with socioeconomic and meteorological data, in signaling which factors control the recurrence of fire and how long they last at the regional or global level [3]. Since the early 1970s, sensors onboard numerous Earth observation missions, such as AVHRR (Advanced Very High Resolution Radiometer), SPOT-VGT (Satellite Pour l’Observation de la Terre-Vegetation), ATSR (Along Track Scanning Radiometer), MODIS (Moderate Resolution Imaging Spectroradiometer), or Landsat, have made it possible to derive BA products on a global and/or regional scale. Among these products, we can highlight GLOBSCAR [ 4 ], GBA2000 [ 5 ], GBS (Global Burned Surfaces) [ 6 ], GLOBCARBON [ 7 ], L3JRC [ 8 ], GEOLAND2 [ 9 ], Global Fire Emission Database (GFED) [ 10 ], BAECV (Burned Area Essential Climate Variable) product [ 11 ] and GIO-GL1 (Copernicus Global Land Service burned area product) based on the Tansey et al. algorithm [ 8 ]. While some of the above-mentioned products remain operational, the main products currently in use are Fire_CCI 5.1 developed by the ESA [ 12 ] and MCD64A1 C6 developed by the University of Maryland [ 13 ]. Both BA detection methods are based on reflectance derived from solar reflective bands in combination with thermal anomaly maps from active fires (hotspots) of MODIS [ 14 ]. The MODIS sensor has been operational since 2000. It has 36 spectral bands from 0.45 µ m to 14.385, a 12-bit radiometric resolution, and spatial resolutions of 250 m, 500 m, and 1 km [15,16]. Various international scientific programs that deal with global fire assessment defined key objectives of spatial and temporal accuracy, and a set of basic features that BA products must fulfill. The work by Mouillot et al. [ 3 ] presented the summarized instructions of the Integrated Global Observing Strategy (IGOS) [ 17 , 18 ], Global Terrestrial Observing System (GTOS), the Group on Earth Observations (GEO) Carbon strategy [ 19 ], the Global Climate Observing System (GCOS) and the NASA White Paper on Fire Earth System Data Records (Fire ESDR) [ 20 ]. On the one hand, long time series (greater than several decades) that are consistent and temporally stable are required to understand the interaction between climate, vegetation, and fire. A spatial resolution of between 250 and 500 m would be desirable. With regard to the spatial accuracy of the products, although some users state that BA products are acceptable when omission and commission errors are balanced [ 21 – 23 ], most set a maximum of around 20% for both CE and OE [ 23 – 25 ]. Do the Fire_CCI 5.1 and MCD64A1 C6 products meet those requirements? Given the technical limitations of the MODIS sensor, can BA mapping results be improved by modifying the algorithmic strategy? The answers to these questions, as well as quantifying the accuracy of these two products, turns out to be very valuable information for properly managing wildfires and their consequences using geoinformatics analysis techniques [26]. In this paper, we present a detailed study of the temporal and spatial accuracy of both datasets, focusing on the boreal region of Alaska. The previous versions of both products (Fire_CCI 4.1 and MCD45A1 C5) have also been included to analyze the impact of the changes made in the new versions. The Alaskan region was selected for two reasons. First, it is one of the few regions in the world that has an official database with detailed records of the area burned by all fires since 1940. These data were used as a reference set to assess the accuracy of all the products. Second, the scars left by burned areas in a boreal region such as Alaska persist for longer, thus facilitating detection and more accurate mapping [27,28]. The objectives pursued in this work are as follows: • To assess the spatiotemporal accuracy of each of the annual time series of the burned area products versus the reference data (AFS) for the 2000–2017 period. • Concerning to temporal accuracy, to calculate the percentages of the annual burned area detected by each product and to analyze the temporal correlation with the reference data. Sensors 2020,20, 5423 3 of 23 • In relation to spatial accuracy, to estimate the main metrics derived from the confusion matrix (commission and omission errors) and determine the Pareto Boundary (PB) for the native spatial resolutions of each product from the reference data to separate the errors of each product from the intrinsic errors associated with its spatial resolution. • To intercompare the spatiotemporal performance of the latest versions of the Fire_CCI 5.1 and MCD64A1 C6 products and to analyze any possible improvements over previous versions (i.e., Fire_CCI 4.1 and MCD45A1 C5.1). • To quantify the contribution of burned area fragmentation to the classified map errors, linking the area under the annual Pareto boundary curve with the total annual errors of each product to its spatial resolution. 2. Materials and Methods 2.1. Study Region The study area spans a large section of Alaska, extending 10 ◦ in latitude (60 ◦ N–70 ◦ N) and 27.5 ◦ in longitude (Figure 1). This area is dominated by boreal forest, a complex set of plant communities modulated mainly by fire, soil type and drainage. The boreal forest forms a mosaic of hardwood–conifer mixed stands with closed canopy in well-drained areas, while in those with permafrost, open spruce stands predominate. Boreal forest makes up 90% of Alaska’s forests, an area of approximately 42 million ha [29]. Sensors 2020, 20, x FOR PEER REVIEW 3 of 24 • To intercompare the spatiotemporal performance of the latest versions of the Fire_CCI 5.1 and MCD64A1 C6 products and to analyze any possible improvements over previous versions (i.e., Fire_CCI 4.1 and MCD45A1 C5.1). • To quantify the contribution of burned area fragmentation to the classified map errors, linking the area under the annual Pareto boundary curve with the total annual errors of each product to its spatial resolution. 2. Materials and Methods 2.1. Study Region The study area spans a large section of Alaska, extending 10° in latitude (60° N–70° N) and 27.5° in longitude (Figure 1). This area is dominated by boreal forest, a complex set of plant communities modulated mainly by fire, soil type and drainage. The boreal forest forms a mosaic of hardwood– conifer mixed stands with closed canopy in well-drained areas, while in those with permafrost, open spruce stands predominate. Boreal forest makes up 90% of Alaska’s forests, an area of approximately 42 million ha [29]. Figure 1. The study region includes the entire boreal forest of Alaska (70° N–60° N, 168.5° W–141° W). 2.2. Reference Data Polygons delimiting the area burned by wildfires in Alaska are available from the Alaska Fire Service (AFS, Fort Wainwright, AK, USA). AFS has compiled a very complete and accurate database since 1940. In addition to the geographic coordinates of the fire site and perimeter, AFS contains information such as the name of the fire; start and extinction dates; estimated BA; cause (naturally (e.g., lightning), human negligence or maliciously) or municipality of origin. Fire perimeters have always been delineated using the best available data source, from traditional hand-drawing on topographic maps in the early decades, to the interpretation of recent fine-scale satellite images with spatial resolutions less than 30 m [30]. This information was used as the reference data (the ground truth) for the accuracy assessment of the MODIS-derived BA products. Figure 1. The study region includes the entire boreal forest of Alaska (70 ◦ N–60 ◦ N, 168.5 ◦ W–141 ◦ W). 2.2. Reference Data Polygons delimiting the area burned by wildfires in Alaska are available from the Alaska Fire Service (AFS, Fort Wainwright, AK, USA). AFS has compiled a very complete and accurate database since 1940. In addition to the geographic coordinates of the fire site and perimeter, AFS contains information such as the name of the fire; start and extinction dates; estimated BA; cause (naturally (e.g., lightning), human negligence or maliciously) or municipality of origin. Fire perimeters have always been delineated using the best available data source, from traditional hand-drawing on Sensors 2020,20, 5423 4 of 23 topographic maps in the early decades, to the interpretation of recent fine-scale satellite images with spatial resolutions less than 30 m [ 30 ]. This information was used as the reference data (the ground truth) for the accuracy assessment of the MODIS-derived BA products. For the study period, from 2000 to 2017, AFS recorded 1868 fires [ 31 ]. The total BA exceeded 11.6 million ha, with an annual average of 0.65 million ha, although strong year-on-year fluctuations were found (see Figure 2): in 12 of the 18 years analyzed, a total BA of 0.5 million ha was not exceeded, with 2001 and 2008 recording the lowest levels, 0.09 and 0.04 million ha, respectively. On the other hand, in 2004 and 2015, the total BA exceeded 2 million ha, with values of 2.71 and 2.08 million ha, respectively. Sensors 2020, 20, x FOR PEER REVIEW 4 of 24 For the study period, from 2000 to 2017, AFS recorded 1868 fires [31]. The total BA exceeded 11.6 million ha, with an annual average of 0.65 million ha, although strong year-on-year fluctuations were found (see Figure 2): in 12 of the 18 years analyzed, a total BA of 0.5 million ha was not exceeded, with 2001 and 2008 recording the lowest levels, 0.09 and 0.04 million ha, respectively. On the other hand, in 2004 and 2015, the total BA exceeded 2 million ha, with values of 2.71 and 2.08 million ha, respectively. Figure 2. Temporal distribution of the annual burned area in Alaska during the period 2000–2017 and distribution per year of the number of fires by size (in thousands of hectares), according the Alaska Fire Service (AFS, Fort Wainwright, AK, USA). Five categories were considered for the sizes (BA extension in ha) of the fires: very small (<100 ha), small (≥100 ha and <1000 ha), medium (≥1000 ha and <10,000 ha), large (≥10,000 ha and <100,000 ha) and very large (≥100,000 ha). Throughout the period considered, small and very small fires account for an average of 60.50% of the total registered fires, but they only represented 1.90% of the total burned area (Figure 3). In contrast, large and very large fires accounted for 13.60% of the total fires and 82.04% of the total burned area. Between them, the 16 fires of more than 100,000 ha burned 20.07% of the total burned area in the study period (Figure 3). Figure 2. Temporal distribution of the annual burned area in Alaska during the period 2000–2017 and distribution per year of the number of fires by size (in thousands of hectares), according the Alaska Fire Service (AFS, Fort Wainwright, AK, USA). Five categories were considered for the sizes (BA extension in ha) of the fires: very small (<100 ha), small ( ≥ 100 ha and <1000 ha), medium ( ≥ 1000 ha and <10,000 ha), large (≥10,000 ha and <100,000 ha) and very large (≥100,000 ha). Throughout the period considered, small and very small fires account for an average of 60.50% of the total registered fires, but they only represented 1.90% of the total burned area (Figure 3). In contrast, large and very large fires accounted for 13.60% of the total fires and 82.04% of the total burned area. Between them, the 16 fires of more than 100,000 ha burned 20.07% of the total burned area in the study period (Figure 3). To construct the annual reference data maps, all the perimeters of fires occurring during the MODIS era (2000–2017) were downloaded from AFS. The annual vector layers were projected to the Albers Conical Equal Area projection using the maximum area method to assign a burned/non-burned class label [32]. The final size of each pixel in the reference maps was 50 m ×50 m. Sensors 2020,20, 5423 5 of 23 Sensors 2020, 20, x FOR PEER REVIEW 5 of 24 Figure 3. Percentage of the total number of recorded fires by the Alaska Fire Service (AFS) and total area burned according to the five categories considered for the sizes of the fires in Figure 2 for the 18 years of study (2000–2017). To construct the annual reference data maps, all the perimeters of fires occurring during the MODIS era (2000–2017) were downloaded from AFS. The annual vector layers were projected to the Albers Conical Equal Area projection using the maximum area method to assign a burned/nonburned class label [32]. The final size of each pixel in the reference maps was 50 m × 50 m. 2.3. Burned Area Products This study compared the main global burned area products using MODIS sensor data: Fire_CCI 5.1 from the ESA project of the same name, led by the University of Alcalá de Henares, and the official NASA MODIS Direct Broadcast Monthly Burned Area Product, developed by the University of Maryland. Both of their most recent versions (Fire_CCI v. 5.1 and MCD64A1 C6), along with the previous versions (Fire_CCI v. 4.1 and MCD45A1 C5.1), were considered to analyze possible performance improvements. Table 1 shows their main characteristics. The main concerns of ESA and NASA when updating their products are to extend the time series and improve the algorithms to obtain the best validation results. Besides, NASA as owner and developer of MODIS sensors periodically reprocesses the entire data archive to incorporate better calibration and improved upstream data into all MODIS products. Table 1. Burned area products. Product Time Span Sensor Method Spatial Resolution Algorithm Reference Fire_CCI 4.1 2005– 2011 MERIS + Terra MODIS Reflectance + hotspots 300 m [33] Fire_CCI 5.1 2001– today Terra MODIS Reflectance + hotspots 250 m [12] MCD45A1 C5.1 2000– 2016 Terra/Aqua MODIS Reflectance 500 m [34–36] MCD64A1 C6 2000– today Terra/Aqua MODIS Reflectance + hotspots 500 m [13] To construct the annual BA maps for Alaska, the respective monthly composites of the four products were downloaded. As with the reference maps, all these maps were re-projected to the Albers Conical Equal Area, with a pixel size of 50 m × 50 m, and finally combined on an annual basis. Figure 4 shows the annual maps of each of the products generated for the year 2009. Figure 3. Percentage of the total number of recorded fires by the Alaska Fire Service (AFS) and total area burned according to the five categories considered for the sizes of the fires in Figure 2for the 18 years of study (2000–2017). 2.3. Burned Area Products This study compared the main global burned area products using MODIS sensor data: Fire_CCI 5.1 from the ESA project of the same name, led by the University of Alcal á de Henares, and the official NASA MODIS Direct Broadcast Monthly Burned Area Product, developed by the University of Maryland. Both of their most recent versions (Fire_CCI v. 5.1 and MCD64A1 C6), along with the previous versions (Fire_CCI v. 4.1 and MCD45A1 C5.1), were considered to analyze possible performance improvements. Table 1shows their main characteristics. The main concerns of ESA and NASA when updating their products are to extend the time series and improve the algorithms to obtain the best validation results. Besides, NASA as owner and developer of MODIS sensors periodically reprocesses the entire data archive to incorporate better calibration and improved upstream data into all MODIS products. Table 1. Burned area products. Product Time Span Sensor Method Spatial Resolution Algorithm Reference Fire_CCI 4.1 2005–2011 MERIS +Terra MODIS Reflectance +hotspots 300 m [33] Fire_CCI 5.1 2001–today Terra MODIS Reflectance +hotspots 250 m [12] MCD45A1 C5.1 2000–2016 Terra/Aqua MODIS Reflectance 500 m [34–36] MCD64A1 C6 2000–today Terra/Aqua MODIS Reflectance +hotspots 500 m [13] To construct the annual BA maps for Alaska, the respective monthly composites of the four products were downloaded. As with the reference maps, all these maps were re-projected to the Albers Conical Equal Area, with a pixel size of 50 m × 50 m, and finally combined on an annual basis. Figure 4 shows the annual maps of each of the products generated for the year 2009. Three of the four products base their algorithmic strategy on a hybrid approach. They use the MODIS active fires product (hotspots) in combination with changes in daily surface reflectance to identify burned pixels. In contrast, the fourth product (MCD45A1) only uses daily surface reflectance imagery. The following is a brief description of the algorithms used in each of the products. Sensors 2020,20, 5423 6 of 23 Sensors 2020, 20, x FOR PEER REVIEW 6 of 24 Figure 4. Annual burned area maps for the Alaskan region in 2009. Red, burned pixels; white, study region; green, political borders and coastlines. Three of the four products base their algorithmic strategy on a hybrid approach. They use the MODIS active fires product (hotspots) in combination with changes in daily surface reflectance to identify burned pixels. In contrast, the fourth product (MCD45A1) only uses daily surface reflectance imagery. The following is a brief description of the algorithms used in each of the products. 2.3.1. MCD45A1 Collection 5.1 This BA product uses the MCD45 algorithm to identify burned pixels at a 500 m spatial resolution. It detects changes in the time series of daily bi-directional reflectance in bands 2 (0.841– 0.876 µm) and 5 (1.23–1.25 µm) of MODIS sensor [37]. MCD45A1, which is currently deprecated, was the official product until the release of MCD64A1 C6. The algorithm compares the observed daily reflectance values for each pixel with the predicted values using a bi-directional reflectance model in a 16-day-minimum time window that selects the candidate dates for burning in the forward and backward directions. If both dates match, the pixel is considered burned. That date is then used as a seed to identify, through a contextual iterative process, whether neighboring pixels can be classified as burned or not. In a final step, unselected candidate pixels are considered burned if they have at least three neighbors burned, with their burn date being the average of their neighbors. The algorithm finally excludes those pixels already burned in previous seasons and years [34–36]. Figure 4. Annual burned area maps for the Alaskan region in 2009. Red, burned pixels; white, study region; green, political borders and coastlines. 2.3.1. MCD45A1 Collection 5.1 This BA product uses the MCD45 algorithm to identify burned pixels at a 500 m spatial resolution. It detects changes in the time series of daily bi-directional reflectance in bands 2 (0.841–0.876 µ m) and 5 (1.23–1.25 µ m) of MODIS sensor [ 37 ]. MCD45A1, which is currently deprecated, was the official product until the release of MCD64A1 C6. The algorithm compares the observed daily reflectance values for each pixel with the predicted values using a bi-directional reflectance model in a 16-day-minimum time window that selects the candidate dates for burning in the forward and backward directions. If both dates match, the pixel is considered burned. That date is then used as a seed to identify, through a contextual iterative process, whether neighboring pixels can be classified as burned or not. In a final step, unselected candidate pixels are considered burned if they have at least three neighbors burned, with their burn date being the average of their neighbors. The algorithm finally excludes those pixels already burned in previous seasons and years [34–36]. 2.3.2. MCD64A1 Collection 6 This product applies the MCD64 algorithm to identify burned pixels using 500 m daily surface reflectance products for bands 5 ( ρ 5: 1.23–1.25 µ m) and 7 ( ρ 7: 2.105–2.155 µ m), along with daily active fire products (hotspots), both derived from the imaging products of the MODIS sensor on board the Terra and Aqua satellites [38]. The algorithm initially calculates the maximum daily changes in the time series of a burn-sensitive vegetation index VI =( ρ 5 −ρ 7)/( ρ 5+ ρ 7) for two preand post-date temporal windows. From these dates, it assigns the burn/unburn label to the pixel using the MODIS daily active fire product. It then extracts a set of training samples and performs an initial supervised classification of all pixels based on Sensors 2020,20, 5423 7 of 23 the normalized distance measurement of each pixel to the nearest pixel in the training set. The final classification is obtained by using contextual information (nearest neighbors) [13]. 2.3.3. Fire_CCI 4.1 The Fire_CCI 4.1 product, which has now been discontinued, covers only the period from 2005 to 2011. It identifies burned pixels at a 300 m spatial resolution using time series of daily surface reflectances from the MERIS sensor on board the ENVISAT satellite, and the active fires product derived from MODIS sensor images. The algorithm initially constructs monthly composites of surface reflectances by selecting the candidate pixels to be burned using the MODIS hotspot dates as criteria. It calculates cumulative distribution functions to discriminate the most clearly burned pixels by means of near-infrared reflectance thresholds. Then, it selects seed pixels in a 5 × 5 pixel window centered on the hotspot, to grow the burned regions by contextual analysis of the neighboring pixels. A final filter removes isolated pixels, both burned and non-burned [33]. 2.3.4. Fire_CCI 5.1 This is the latest version of the burned area product from the Fire_CCI project. The algorithm, similar to its predecessor, is based on a two-stage hybrid approach. In the first stage, the seed pixels are selected, guided by the daily active fires (thermal anomalies) derived from the MODIS sensors on board the Terra and Aqua satellites. In the second stage, the growth and delimitation of the burned area is performed using the daily surface reflectance products from MODIS sensor bands 1 (0.62–0.67 µ m) and 2 (0.841–0.876 µm) at a 250 m spatial resolution [12]. 2.4. Accuracy Assessment To validate the burned area products, an accuracy assessment was performed against the AFS reference data for those periods when each of the products was available (Table 1). The AFS perimeters used in this study are derived primarily from satellite images with spatial resolutions of less than 30 m, including the Landsat and Sentinel-2 missions, the latter since 2016. Figure 5shows a description of the workflow of this assessment exercise carried out on an annual basis. All annual burned area map time series were delimited to the study region with 50 m × 50 m pixels in the same projection as the reference data. Each pixel therefore represents an area of 0.25 ha. Pixels were labeled 0 (Not Burned) or 1 (Burned). This simplified the calculation of the total annual burned area (in hectares) of each product, which was obtained by multiplying the number of pixels labeled as 1 (Burned) by 0.25. To assess the temporal accuracy, the percentage of annual burned area for each product (P Year (%)) was calculated with respect to the reference set (Equation (1)). PYear(%)=Pn i=1BAPYear(i) Pn i=1BARYear(i)×100 (1) where BAP Year (i) is the value of the pixel i (0: Not Burned; 1: Burned) for the indicated year of BA product; BAR Year (i) is the value of the same pixel i for the reference data; and n is the total number of pixels in each burned area map. For each annual burned area percentage distribution, a centrality measure (total value for the entire period), calculated as the weighted average of the annual percentages of burned area ( PBAP (%)) (Equation (2)), was obtained. Y 0 and Y E are the first and last years of the BA product, respectively. Scatterplots of the annual percentages for each product were constructed against the reference data to analyze the linear dispersion around the central value and to identify the extreme cases (years with estimate percentages well below or above the mean). PBAP(%)=PYE Year =Y0Pn i=1BAP Year(i) PYE Year =Y0Pn i=1BAR Year(i)×100 (2) Sensors 2020,20, 5423 8 of 23 Sensors 2020, 20, x FOR PEER REVIEW 8 of 24 Figure 5. Flowchart followed for the accuracy assessment of standard BA products. To assess the temporal accuracy, the percentage of annual burned area for each product (PYear(%)) was calculated with respect to the reference set (Equation (1)). P(%)=∑()   ∑()   ×100 (1) where BAPYear(i) is the value of the pixel i (0: Not Burned; 1: Burned) for the indicated year of BA product; BARYear(i) is the value of the same pixel i for the reference data; and n is the total number of pixels in each burned area map. For each annual burned area percentage distribution, a centrality measure (total value for the entire period), calculated as the weighted average of the annual percentages of burned area (P (%)) (Equation (2)), was obtained. Y0 and YE are the first and last years of the BA product, respectively. Scatterplots of the annual percentages for each product were constructed against the reference data to analyze the linear dispersion around the central value and to identify the extreme cases (years with estimate percentages well below or above the mean). P (%)=∑(∑ ()   )   ∑(∑ ()   )  ×100 (2) Subsequently, a plot was constructed of the annual burned area temporal distribution of each product and that of the reference data. Correlation analysis of each time series was performed with the reference data, calculating the coefficient of determination R2 (the square of Pearson’s correlation coefficient). The spatial accuracy assessment for each annual BA map employed the confusion matrix method, which is commonly used to validate thematic maps [39]. Table 2 shows the confusion matrix for a pixel-level thematic classification with two classes (Burned and Non-Burned). The independent reference information (AFS) is located in the matrix columns, and the burned area map data for each Figure 5. Flowchart followed for the accuracy assessment of standard BA products. Subsequently, a plot was constructed of the annual burned area temporal distribution of each product and that of the reference data. Correlation analysis of each time series was performed with the reference data, calculating the coefficient of determination R 2 (the square of Pearson’s correlation coefficient). The spatial accuracy assessment for each annual BA map employed the confusion matrix method, which is commonly used to validate thematic maps [ 39 ]. Table 2shows the confusion matrix for a pixel-level thematic classification with two classes (Burned and Non-Burned). The independent reference information (AFS) is located in the matrix columns, and the burned area map data for each product in the rows. The diagonal elements are the correctly classified data (true Burned and true Non-Burned). The other cells indicate commission errors (CE), i.e., pixels classified as Burned that are not actually burned (false burned), or omission errors (OE), pixels that are actually burned but that have been classified as Non-Burned (false non-burned) [ 40 ]. Other commonly used metrics that can be derived from the confusion matrix are Overall Accuracy (OA) (the percentage of correctly classified pixels), Sensibility (S) (or producer’s accuracy) which calculates the rate of true burned pixels (the proportion of burned pixels that were correctly identified) and Specificity (Sp), or the rate of true Non-Burned pixels (the proportion of properly identified Non-Burned pixels) (Table 2). Sensors 2020,20, 5423 9 of 23 Table 2. Confusion matrix of a binary classification of burned area and the main metrics derived from it. OA, overall accuracy; S, sensibility or producer’s accuracy; Sp, specificity; CE, commission error; OE, omission error. Reference Data Burned Non-Burned Total Classified Data Burned n11 n12 n1c Non-Burned n21 n22 n2c Total n1r n2r n OA =n11+n22 nS=n11 n1r Sp =n22 n1r CE =n12 n1c OE =n21 n1r The OA and Sp are metrics that can create a false sense of correctness in classified maps, especially in this study, due to the asymmetry between the two classes considered (the Non-Burned class is the majority and its success rate would be very high). On the other hand, S is related to the omission error (S =1 − OE), so only commission and omission errors of the Burned class were considered. The two errors are not comparable since, although they both represent percentages of the pixels labeled as Burned, in one case they are Burned concerning the classified map (CE) and in the other case for the reference map (OE). Therefore, for the total error (TE) calculation, the weighted sum of both errors was considered, taking into account the percentage (P) of BA identified by the classified map, according to Equation (3). Total Error (ha)=CE ×BAP +OE ×BAR =(CE ×P+OE)×BAR Total Error (%)=Total Error (ha) BAR =CE ×P+OE (3) The annual distribution of OE and CE for each burned area product was calculated, as well as their average values (the totals of all years considered). To identify extreme values, CE scatterplots were constructed against the annual OE of each product, and the annual deviations from the average values were analyzed. Likewise, to analyze the possible relationship between the spatial accuracy and the annual burned area, scatterplots of commission errors and omission errors were constructed for each BA product, against the annual burned reference area. To determine whether fragmentation of the burned areas limits the spatial accuracy of the BA products, PB were constructed at 250, 300 and 500 m spatial resolutions [ 41 ]. To obtain each PB, the AFS reference map was used, with a high spatial resolution (50 m × 50 m) per pixel. Using a pixel spatial aggregation process, the reference data was resized to maps of 250, 300 and 500 m, where the value of each pixel contained the percentage of burned area. If one of these mixed pixels is classified as Burned, in a strict binary classification, a commission error equal to a 1-pixel value is being made. However, if it is classified as Non-Burned, an omission error equal to the pixel value is produced. The final decision on how it is classified depends on a p parameter selected in the range [0, 1], which sets the minimum threshold for assigning a mixed pixel to the Burned class. Then, for each degraded map, the CE and OE pairs, obtained by varying this parameter between 0 and 1 in equidistant steps, were calculated. The set of pairs {(CE i , OE i )} resulting from this process is the PB (Figure 6). The PB represents the lowest possible errors, obtained in a strict binary classification. These errors are attributable exclusively to the fragmentation of referenced burned areas that occur in mixed pixels when the spatial resolution of the data decreases. All points on the PB represent ideal classifications for a given spatial resolution. The PB further demonstrates that minimizing the commission and omission errors simultaneously is contradictory: a classification with minimum commission errors would mean greater failure of omission and, conversely, minimizing the OE would increase the CE. Sensors 2020,20, 5423 16 of 23 behavior were included, to provide a visual comparison of the PB. One can observe that in 2006 and 2008 the distance from the annual pair of errors (EC, EO) to the PB is much larger than in 2004 and 2015. It can also be seen that the PB are more separated from the Cartesian axes in those years. This indicates a greater fragmentation of the burned areas when the spatial resolution is degraded, which contributes, in part, to the errors of the corresponding annual map. Sensors 2020, 20, x FOR PEER REVIEW 16 of 24 3.3. Pareto Boundaries The Pareto boundaries were constructed at 250/300/500 m for all years in the study period. Figure 11 shows the Pareto boundaries at 250 and 500 m for 2004, 2006, 2008 and 2015, as well as the annual CE and OE for the MCD64A1 and Fire_CCI 5.1 products. Years 2006 and 2008 were selected because both have the biggest errors in both products. In addition, two years, 2004 and 2015, with good behavior were included, to provide a visual comparison of the PB. One can observe that in 2006 and 2008 the distance from the annual pair of errors (EC, EO) to the PB is much larger than in 2004 and 2015. It can also be seen that the PB are more separated from the Cartesian axes in those years. This indicates a greater fragmentation of the burned areas when the spatial resolution is degraded, which contributes, in part, to the errors of the corresponding annual map. Figure 11. Pareto boundaries at 250 and 500 m and annual CE and OE for the MCD64A1 and Fire_CCI 5.1 products for 2004, 2006, 2008 and 2015. To quantify the impact of the PB on the annual maps of the different BA products, the annual and total AUPB values were calculated at spatial resolutions of 250/300/500 m (Table 5). On average, the AUPB value at 500 m triples the 250 m value, in addition to having greater annual variability: [0.0019–0.0146] versus [0.0007–0.0048]. The year 2008 shows the highest AUPB values for all spatial resolutions. Figure 11. Pareto boundaries at 250 and 500 m and annual CE and OE for the MCD64A1 and Fire_CCI 5.1 products for 2004, 2006, 2008 and 2015. To quantify the impact of the PB on the annual maps of the different BA products, the annual and total AUPB values were calculated at spatial resolutions of 250/300/500 m (Table 5). On average, the AUPB value at 500 m triples the 250 m value, in addition to having greater annual variability: [0.0019–0.0146] versus [0.0007–0.0048]. The year 2008 shows the highest AUPB values for all spatial resolutions. Finally, Figure 12 shows the annual TE scatterplots for each product versus the annual PB area corresponding to their spatial resolution. A linear regression model was constructed for each BA product. The R 2 values are very low for the older versions of the BA products (0.00 and 0.18, respectively), but increase markedly in the new versions. Fire_CCI 5.1 has a slightly higher value than MCD64A1 C6 (0.45 vs. 0.41) and also a linear regression line slope that is almost double (63.1 vs. 32.8). Sensors 2020,20, 5423 17 of 23 Table 5. Areas ( × 10 −3 ) enclosed by the annual Pareto Boundaries (AUPB) at different resolutions and the weighted average value for all years. Year Spatial Resolution 250 m 300 m 500 m 2000 4.885 2001 1.670 4.940 2002 0.824 2.527 2003 1.341 4026 2004 0.656 1.909 2005 1.066 1.333 3.191 2006 2.171 2.810 6.480 2007 2.125 2.767 6.724 2008 4.818 6.249 14.610 2009 0.931 1.149 2.714 2010 2.257 2.891 7.106 2011 2.810 3.680 9.056 2012 2.715 8.642 2013 1.534 4.723 2014 0.778 2.192 2015 1.470 4.356 2016 4.120 12.685 2017 2.733 8.253 All years 1.195 1.625 3.612 Sensors 2020, 20, x FOR PEER REVIEW 17 of 24 Table 5. Areas (× 10−3) enclosed by the annual Pareto Boundaries (AUPB) at different resolutions and the weighted average value for all years. Year Spatial Resolution 250 m 300 m 500 m 2000 4.885 2001 1.670 4.940 2002 0.824 2.527 2003 1.341 4026 2004 0.656 1.909 2005 1.066 1.333 3.191 2006 2.171 2.810 6.480 2007 2.125 2.767 6.724 2008 4.818 6.249 14.610 2009 0.931 1.149 2.714 2010 2.257 2.891 7.106 2011 2.810 3.680 9.056 2012 2.715 8.642 2013 1.534 4.723 2014 0.778 2.192 2015 1.470 4.356 2016 4.120 12.685 2017 2.733 8.253 All years 1.195 1.625 3.612 Finally, Figure 12 shows the annual TE scatterplots for each product versus the annual PB area corresponding to their spatial resolution. A linear regression model was constructed for each BA product. The R2 values are very low for the older versions of the BA products (0.00 and 0.18, respectively), but increase markedly in the new versions. Fire_CCI 5.1 has a slightly higher value than MCD64A1 C6 (0.45 vs. 0.41) and also a linear regression line slope that is almost double (63.1 vs. 32.8). Figure 12. Scatterplots of the total annual TE errors (weighted sum of the annual commission and omission errors) versus the area enclosed by the annual Pareto boundary. Data for the years 2000 and 2001 were not included, as there were incomplete data from the MODIS sensor. Figure 12. Scatterplots of the total annual TE errors (weighted sum of the annual commission and omission errors) versus the area enclosed by the annual Pareto boundary. Data for the years 2000 and 2001 were not included, as there were incomplete data from the MODIS sensor. 4. Discussion The evaluation and validation of global BA products derived from satellite imagery require a set of reliable, independent and representative reference fire perimeters that cover as long a period as possible. It is necessary to understand the uncertainty of these products before incorporating them as input data into global carbon, vegetation or climate models, as well as for the management of all the Sensors 2020,20, 5423 18 of 23 wildfire phases. Numerous prior studies have evaluated the behavior of the MODIS sensor-derived products analyzed in this study, both as part of ESA’s Fire Climate Change Initiative Project and in the different versions [ 13 , 51 – 58 ] of the MODIS Direct Broadcast Monthly Burned Area Product. However, many of these works construct reference fire perimeters using images from better spatial resolution sensors such as Landsat TM/ETM or Sentinel-2, and which are limited to short time periods, usually one or several years, due to the difficulty in creating larger reference sets. This work presents an independent intercomparison exercise on the accuracy of the two main global BA products derived from the MODIS sensor covering an extensive boreal region over 18 years, containing all the AFS-registered fire perimeters. To the best of our knowledge, there have been no previous intercomparison studies of these four products over such a long period. In relation to the temporal accuracy (Figure 8), and except for the Fire_CCI 4.1 product, which has the shortest time series, the BA products analyzed conform to the time pattern of the reference data with determination coefficients above 0.97, and with results similar to those found by other authors in different ecosystems. Thus, for example, Turco et al. [ 56 ] reported high determination coefficients of 0.96 and 0.97 in the monthly BA estimates for Fire_CCI 5.1 and MCD64A1 C6 when compared to the reference dataset from the European Forest Fire Information System (EFFIS) that includes burned area data for some European countries in the Mediterranean basin (Portugal, Spain, Southern France and Greece). However, all the BA products analyzed underestimated the annual burned area (Table 3) with higher percentages than those obtained by other authors for regions other than Alaska’s boreal forest. Turco et al. [ 56 ] obtained an underestimation of only 14% for the Fire_CCI 5.1 product with the EFFIS reference set. Campagnolo et al. [ 57 ] found a 28% underestimation for the MCD64A1 product using a reference set of more than 100 fire perimeters (1.24 Mkm 2 ) derived from Landsat TM/ETM+images distributed around the world for 2008 and previously constructed by Padilla et al. [54]. From the results intercomparison determined for the two new versions (Table 3), one can show that MCD64A1 provided better BA estimates than Fire_CCI in some years, despite having a lower spatial resolution (500 vs. 250 m), even in the years with the largest BA, which are the ones that contributed more to the average values (2004, 2005 and 2015); nevertheless, it performed relatively worse in other years (2009, 2010 and 2011). For the MCD64A1 and Fire_CCI 5.1 products, in the years with the highest amount of BA (over 1 million ha), the annual estimate percentages tended to stabilize around the average value with lower variability (Figure 7). One should bear in mind that it is precisely these years that contribute most to the overall average throughout the study period. Conversely, the years with the least amount of BA (below 0.5 million ha) had greater variability; this was less pronounced in the Fire_CCI 5.1 product than in MCD64A1, probably due to its better spatial resolution (250 vs. 500 m). With regard to spatial accuracy (Table 4), a global analysis showed that the older versions, Fire_CCI 4.1 and MCD45A1 C5.1, had the lowest commission errors (5.9% and 6.6%, respectively) but the omission errors were very high (67.5% and 73.7%, respectively). The new versions, Fire_CCI 5.1 and MCD64A1 C6, had significantly reduced OE (up to 39.0% and 48.0% respectively), even at the cost of slightly worse CE (7.5% and 17.8%, respectively). As the percentage of BA of all products was below 100% (they underestimated the total BA), the weighted sum or TE tended to give a lower weight to the CE than to the OE, favoring the algorithms that reduce the omission error even at the cost of increasing the commission errors. This low imbalance between omission and commission errors (along with the sharp drop in OE) translates into a better BA estimate. These results are comparable to those obtained by other authors using other reference sets [ 51 , 58 ]. In a recent validation of Fire_CCI 5.1 using 1200 global samples over the 2003–2014 period, Lizundia-Loiola et al. [ 58 ] obtained values of CE =54.4% and OE =67.1%. In the Stage 3 validation of MCD64A1 C6 using 558 pairs of Landsat images from 2014 and 2015, Boschetti et al. [ 51 ] obtained values of CE =40.2% and OE =72.6% on a global scale, which improved significantly for boreal forest biomass (CE =23.9% and OE =27.0%). The OE from this latter study, which was specific to the boreal region, differs significantly from that obtained in the present work (48.0%) for the Alaska region; we understand that this may be due to the Sensors 2020,20, 5423 19 of 23 small reference dataset used by Boschetti et al. [ 51 ] for the entire boreal region. In contrast, our study used all the burned area perimeters recorded in the 2000–2017 period. The PB analysis (Figure 11) allowed us to partially explain the annual variability in the commission/omission errors of the BA products. The average AUPB values (Table 5) reflect an increase as the spatial resolution worsens, indicating that there is a higher percentage of commission and omission errors attributable to the data’s low spatial resolution. Again, there was high variability in the annual values relative to the average values, attributable to the fragmentation of the burned areas. This high accuracy variability attributable to landscape fragmentation was also reported by Rodrigues et al. [ 59 ], in the accuracy assessment of MCD64A1 C6 in the Brazilian Cerrado vs. Landsat perimeters for the 2011–2019 period. Rodrigues et al. [ 59 ] found that, in the northern Cerrado, which had larger areas affected by fire, the MCD64A1 C6 performance was significantly higher than in the southern Cerrado area, where a more fragmented landscape and smaller patches of fire predominated. The linear regression analysis between the TE and the AUPB found an upward linear trend for the new versions of the BA products; this partially justifies the high commission and omission errors encountered in some years. However, it is also apparent that in some years (e.g., 2016), the high level of fire fragmentation had little influence on the TE. Conversely, in years with low levels of fragmentation (e.g., 2002 and 2005 for Fire_CCI 5.1 and 2003 and 2014 for MCD64A1 C6), the TEs were significant. For these years, the observations were limited by other factors such as the poor behavior of the detection algorithms, the low severity of burned areas or certain environmental conditions; as indicated by Loboda et al. [ 60 ], these factors might explain such errors. The more precise linear fit of the new versions compared to the old ones, with R 2 values close to 0.5 and a higher slope, reflects greater TE sensitivity (CE and OE) to burned area fragmentation. Of all the years analyzed, and except for the years 2000 and 2001 (years with incomplete data from the MODIS sensor), there are two years in which the latest versions of MCD64A1 and Fire_CCI performed poorly. The year 2006 showed low detection rates (29.6% and 45.83%, respectively), high omission values (83.9% and 60.9%) and high commission values (29.6% and 14.7%). The burned area recorded (0.11 million ha) was below the annual average, as was the same percentage corresponding to large fires (69.82%), which was also below the annual average, while the PB areas at 250 (0.002257) and 500 m (0.006480) were almost double the average values at these resolutions. Similarly, in 2008, the year with the least amount of burned area (0.04 million ha) and the lowest percentage attributable to large fires (36.65%), there were high commission and omission error values (57.3% and 71.2% for MCD64A1 and 29.2% and 43.8% for Fire_CCI) but a large percentage of burned area detected (67.49% and 79.43%, respectively). For that year, the area enclosed by the PB at 250/500 m was the highest, so a significant part of the spatial errors was due to the data’s low spatial resolution. 5. Conclusions An independent and detailed study was carried out to evaluate the spatiotemporal accuracy of the latest versions (along with the previous versions) of the two main global scale BA products derived from MODIS-sensor satellite images, here restricted to the Alaskan boreal region. As reference data, we used all the polygons of the BA recorded by AFS over the study period. In addition, a detailed 50 m × 50 m pixel analysis of the accuracy of each BA product was performed. Fire_CCI 5.1 and MCD64A1 C6 presented significant improvements over their previous versions in terms of BA estimation. Improvements were achieved by reducing the imbalance between commission and omission errors and, especially, by greatly reducing omission errors even at the expense of worsening commission errors. Both products currently produce similar BA estimation percentages, although the positional accuracy of Fire_CCI is better than that of MCD64A1, which is in line with its higher spatial resolution (250 vs. 500 m). Fire_CCI 5.1 would be the option chosen for users who, through geoinformatics analysis techniques, use BA products for forest fire management. For those users who are studying the increase in greenhouse gas concentrations or the change in the chemical composition Sensors 2020,20, 5423 20 of 23 of the atmosphere due to fire emissions, any of the latest versions of the products analyzed could be suitable, but always taking into account the errors of omission (over 40%) in both cases. The high variability in annual results is noteworthy, both in the percentages of BA detection and in the omission and commission errors, which puts into question much accuracy assessment work that uses limited spatiotemporal reference data. It would be recommendable to use all possible reference data when available. The yearly analysis of burned-area fragmentation across the corresponding Pareto boundaries has allowed us to establish a quantitative measure of the same (AUPB), which relates to the total errors (weighted sum of commission and omission errors) obtained for the latest versions of the Fire_CCI and MCD64A1 products. The results from this work could be extrapolated to other boreal regions that do not have such accurate reference datasets as are available in the northern regions of North America. Author Contributions: Conceptualization and methodology, J.A.M.-R., J.R.G.-L. and M.A.; validation and formal analysis, J.A.M.-R., J.R.G.-L., M.A. and M.C.-G.; writing—original draft preparation, J.R.G.-L. and J.A.M.-R.; writing—review and editing, M.A. and M.C.-G.; project administration, J.A.M.-R. and M.A.; and funding acquisition, M.A. and J.A.M.-R. All authors have read and agreed to the published version of the manuscript. Funding: This research was funded by the Ministerio de Ciencia, Innovaci ó n y Universidades (MCIU), the Agencia Estatal de Investigaci ó n (AEI) and the Fondo Europeo de Desarrollo Regional (FEDER) through the project RTI2018-099171-B-I00. The Universidad de Almer í a and the FEDER-ANDALUCIA Operating Program partially financed this work through the UAL-TIC-A023-B1 bridge project, in the 2018 call. Acknowledgments: We thank the four anonymous peer reviewers for their valuable comments and suggestions. The authors express their gratitude to the Climate Change Initiative of the European Space Agency, the Fire_CCI project, NASA, the University of Maryland and AFS, for the processing and free distribution of the burned area product data used in this work. Conflicts of Interest: The authors declare no conflict of interest. References 1. Van der Werf, G.R.; Randerson, J.T.; Giglio, L.; van Leeuwen, T.T.; Chen, Y.; Rogers, B.M.; Mu, M.; van Marle, M.J.E.; Morton, D.C.; Collatz, G.J.; et al. Global fire emissions estimates during 1997–2016. Earth Syst. Sci. Data 2017,9, 697–720. [CrossRef] 2. Kitzberger, T.; Falk, D.A.; Westerling, A.L.; Swetnam, T.W. Direct and indirect climate controls predict heterogeneous early-mid 21st century wildfire burned area across western and boreal North America. PLoS ONE 2017,12, e0188486. [CrossRef] [PubMed] 3. Mouillot, F.; Schultz, M.G.; Yue, C.; Cadule, P.; Tansey, K.; Ciais, P.; Chuvieco, E. Ten years of global burned area products from spaceborne remote sensing—A review: Analysis of user needs and recommendations for future developments. Int. J. Appl. Earth Obs. Geoinf. 2014,26, 64–79. [CrossRef] 4. Simon, M.; Plummer, S.; Fierens, F.; Hoelzemann, J.J.; Arino, O. Burnt area detection at global scale using ATSR-2: The GLOBSCAR products and their qualification. J. Geophys. Res. D Atmos. 2004 ,109, 1–16. [CrossRef] 5. Tansey, K.; Gr é goire, J.M.; Stroppiana, D.; Sousa, A.; Silva, J.; Pereira, J.M.C.; Boschetti, L.; Maggi, M.; Brivio, P.A.; Fraser, R.; et al. Vegetation burning in the year 2000: Global burned area estimates from SPOT VEGETATION data. J. Geophys. Res. D Atmos. 2004,109, 1–22. [CrossRef] 6. Carmona-Moreno, C.; Belward, A.; Malingreau, J.P.; Hartley, A.; Garcia-Alegre, M.; Antonovskiy, M.; Buchshtaber, V.; Pivovarov, V. Characterizing interannual variations in global fire calendar using data from Earth observing satellites. Glob. Chang. Biol. 2005,11, 1537–1555. [CrossRef] 7. Plummer, S.; Arino, O.; Simon, M.; Steffen, W. Establishing a earth observation product service for the terrestrial carbon community: The globcarbon initiative. Mitig. Adapt. Strateg. Glob. Chang. 2006 ,11, 97–111. [CrossRef] 8. Tansey, K.; Gr é goire, J.M.; Defourny, P.; Leigh, R.; Pekel, J.F.; van Bogaert, E.; Bartholom é , E. A new, global, multi-annual (2000–2007) burnt area product at 1 km resolution. Geophys. Res. Lett. 2008 ,35, 01401. [CrossRef] Sensors 2020,20, 5423 21 of 23 9. Tansey, K.; Bradley, A.; Smets, B.; van Best, C.; Lacaze, R. The Geoland2 BioPar burned area product. In Proceedings of the European Geosciences Union General Assembly, Vienna, Austria, 22–27 April 2012; p. 4727. 10. Giglio, L.; Randerson, J.T.; Van Der Werf, G.R. Analysis of daily, monthly, and annual burned area using the fourth-generation global fire emissions database (GFED4). J. Geophys. Res. Biogeosci. 2013 ,118, 317–328. [CrossRef] 11. Hawbaker, T.J.; Vanderhoof, M.K.; Beal, Y.J.; Takacs, J.D.; Schmidt, G.L.; Falgout, J.T.; Williams, B.; Fairaux, N.M.; Caldwell, M.K.; Picotte, J.J.; et al. Mapping burned areas using dense time-series of Landsat data. Remote Sens. Environ. 1997,198, 504–522. [CrossRef] 12. Chuvieco, E.; Lizundia-Loiola, J.; Lucrecia Pettinari, M.; Ramo, R.; Padilla, M.; Tansey, K.; Mouillot, F.; Laurent, P.; Storm, T.; Heil, A.; et al. Generation and analysis of a new global burned area product based on MODIS 250 m reflectance bands and thermal anomalies. Earth Syst. Sci. Data 2018 ,10, 2015–2031. [CrossRef] 13. Giglio, L.; Boschetti, L.; Roy, D.P.; Humber, M.L.; Justice, C.O. The Collection 6 MODIS burned area mapping algorithm and product. Remote Sens. Environ. 2018,217, 72–85. [CrossRef] [PubMed] 14. Giglio, L.; Schroeder, W.; Justice, C.O. The collection 6 MODIS active fire detection algorithm and fire products. Remote Sens. Environ. 2016,178, 31–41. [CrossRef] [PubMed] 15. Justice, C.O.; Vermote, E.; Townshend, J.R.G.; Defries, R.; Roy, D.P.; Hall, D.K.; Salomonson, V.V.; Privette, J.L.; Riggs, G.; Strahler, A.; et al. The moderate resolution imaging spectroradiometer (MODIS): Land remote sensing for global change research. IEEE Trans. Geosci. Remote Sens. 1998,36, 1228–1249. [CrossRef] 16. Xiong, X.; Wenny, B.N.; Barnes, W.D. Overview of NASA Earth Observing Systems Terra and Aqua moderate resolution imaging spectroradiometer instrument calibration algorithms and on-orbit performance. J. Appl. Remote Sens. 2009,3, 032501. [CrossRef] 17. Ciais, P.; Moore, B.; Steffen, W.; Hood, M.; Quegan, S.; Cihlar, J.; Raupach, M.; Tschirley, J.; Inoue, G.; Doney, S.; et al. Integrated Global Carbon Observation Theme: A Strategy to Realise a Coordinated System of Integrated Global Carbon Cycle Observations. Available online: https://www.globalcarbonproject.org/ global/pdf/IGOS_CarbonBrochure.pdf (accessed on 21 September 2020). 18. Townshend, J.; Latham, J.; Arino, O. Integrated Global Observations of the Land: An IGOS-P Theme. Available online: http://www.fao.org/3/i0536e/i0536e00.htm (accessed on 21 September 2020). 19. Ciais, P.; Dolman, H.; Dargaville, R.; Barrie, L.; Bombelli, A.; Butler, J.; Canadell, P.; Moriyama, T.; Borges, A.; Boversmann, H.; et al. GEO Carbon Strategy. Available online: https://www.globalcarbonproject.org/global/ pdf/GEO_CARBONSTRATEGY_20101020.pdf (accessed on 21 September 2020). 20. Justice, C.O.; Giglio, L.; Roy, D.P.; Csiszar, I.; Boschetti, L.; Korontzi, S.; Wooster, M.J. White Paper on a NASA Fire ESDR. Available online: https://cce.nasa.gov/mtg2008_ab_presentations/Fire_Justice_whitepaper.pdf (accessed on 9 September 2020). 21. Smiraglia, D.; Filipponi, F.; Mandrone, S.; Tornato, A.; Taramelli, A. Agreement index for burned area mapping: Integration of multiple spectral indices using Sentinel-2 satellite images. Remote Sens. 2020 , 12, 1862. [CrossRef] 22. Bastarrika, A.; Chuvieco, E.; Mart í n, M.P. Mapping burned areas from landsat TM/ETM+data with a two-phase algorithm: Balancing omission and commission errors. Remote Sens. Environ. 2011 ,115, 1003–1012. [CrossRef] 23. Heil, A.; Yue, C.; Mouillot, F.; Kaiser, J.W. ESA Climate Change Initiative—Fire_cci D1.1 User Requirement Document (URD). Available online: http://esa-fire-cci.org/files/Fire_cci_D1.1_URD_v5.1.pdf (accessed on 9 September 2020). 24. Plummer, S.; Chuvieco, E.; Pettinari, M.L.; Ot ó n, G.; Storm, T.; Kloster, S.; Defourny, P.; Lamarche, C. Fire_cci User Requirements Document & Product Specification Document for AVHRR. Available online: https://www.esa-fire-cci.org/sites/default/files/Fire_cci_O2.D1_URD_PSD_AVHRR_v1.1.pdf (accessed on 9 September 2020). 25. Climate Modelling User Group Deliverable 1.1 Requirements Baseline Document. Available online: http://ensembles-eu.metoffice.com/cmug/CMUG_PHASE_2_D1.1_Requirements_v0.6.pdf (accessed on 21 September 2020). 26. Chuvieco, E.; Mouillot, F.; van der Werf, G.R.; San Miguel, J.; Tanasse, M.; Koutsias, N.; Garc í a, M.; Yebra, M.; Padilla, M.; Gitas, I.; et al. Historical background and current developments for mapping burned area from satellite Earth observation. Remote Sens. Environ. 2019,225, 45–64. [CrossRef] Sensors 2020,20, 5423 22 of 23 27. Hicke, J.A.; Asner, G.P.; Kasischke, E.S.; French, N.H.F.; Randerson, J.T.; Collatz, G.J.; Stocks, B.J.; Tucker, C.J.; Los, S.O.; Field, C.B. Postfire response of North American boreal forest net primary productivity analyzed with satellite observations. Glob. Chang. Biol. 2003,9, 1145–1157. [CrossRef] 28. Moreno-Ruiz, J.A.; Garcia-Lazaro, J.R.; Riano, D.; Kefauver, S.C. The synergy of the 0.05 ◦ ( ∼ 5 km) AVHRR long-term data record (LTDR) and landsat TM archive to map large fires in the North American boreal region from 1984 to 1998. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2014,7, 1157–1166. [CrossRef] 29. Wolken, J.M.; Hollingsworth, T.N.; Rupp, T.S.; Chapin, F.S.; Trainor, S.F.; Barrett, T.M.; Sullivan, P.F.; Mcguire, A.D.; Euskirchen, E.S.; Hennon, P.E.; et al. Evidence and implications of recent and projected climate change in Alaska’s forest ecosystems. Ecosphere 2011,2. [CrossRef] 30. AK Fire History Perimeters. Available online: https://www.arcgis.com/home/item.html?id= d4b8d89f226f4c488e1e4ba054e49be9 (accessed on 23 April 2019). 31. Alaska Fire Service (AFS) Alaska Wildland Fire Information Map Series. Available online: https://blmegis.maps.arcgis.com/apps/MapSeries/index.html?appid=32ec4f34fb234ce58df6b1222a207ef1 (accessed on 9 September 2020). 32. Arnone, E.; Francipane, A.; Scarbaci, A.; Puglisi, C.; Noto, L.V. Effect of raster resolution and polygon-conversion algorithm on landslide susceptibility mapping. Environ. Model. Softw. 2016 ,84, 467–481. [CrossRef] 33. Alonso-Canas, I.; Chuvieco, E. Global burned area mapping from ENVISAT-MERIS and MODIS active fire data. Remote Sens. Environ. 2015,163, 140–152. [CrossRef] 34. Roy, D.P.; Lewis, P.E.; Justice, C.O. Burned area mapping using multi-temporal moderate spatial resolution data—A bi-directional reflectance model-based expectation approach. Remote Sens. Environ. 2002 ,83, 263–286. [CrossRef] 35. Roy, D.P.; Jin, Y.; Lewis, P.E.; Justice, C.O. Prototyping a global algorithm for systematic fire-affected area mapping using MODIS time series data. Remote Sens. Environ. 2005,97, 137–162. [CrossRef] 36. Roy, D.P.; Boschetti, L.; Justice, C.O.; Ju, J. The collection 5 MODIS burned area product—Global evaluation by comparison with the MODIS active fire product. Remote Sens. Environ. 2008,112, 3690–3707. [CrossRef] 37. Boschetti, L.; Roy, D.; Hoffmann, A.A.; Humber, M. MODIS Collection 5.1 Burned Area Product—MCD45. Available online: http://modis-fire.umd.edu/files/MODIS_Burned_Area_Collection51_User_Guide_3.1.0.pdf (accessed on 9 September 2020). 38. Giglio, L.; Boschetti, L.; Roy, D.; Hoffmann, A.A.; Humber, M. Collection 6 MODIS Burned Area Product User’s Guide Version 1.0. Available online: https://modis-land.gsfc.nasa.gov/pdf/MODIS_C6_BA_User_ Guide_1.0.pdf (accessed on 9 September 2020). 39. Congalton, R.G. A review of assessing the accuracy of classifications of remotely sensed data. Remote Sens. Environ. 1991,37, 35–46. [CrossRef] 40. Stehman, S.V. Selecting and interpreting measures of thematic classification accuracy. Remote Sens. Environ. 1997,62, 77–89. [CrossRef] 41. Boschetti, L.; Flasse, S.P.; Brivio, P.A. Analysis of the conflict between omission and commission in low spatial resolution dichotomic thematic products: The Pareto Boundary. Remote Sens. Environ. 2004 ,91, 280–292. [CrossRef] 42. Bradley, A.P. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognit. 1997,30, 1145–1159. [CrossRef] 43. Huang, J.; Ling, C.X. Using AUC and accuracy in evaluating learning algorithms. IEEE Trans. Knowl. Data Eng. 2005,17, 299–310. [CrossRef] 44. Gu, X.; Wu, Z.; Zhang, Y.; Yan, S.; Fu, J.; Du, L. Prediction research of the forest fire in Jiangxi province in the background of climate change. Shengtai Xuebao 2020,40. [CrossRef] 45. Fern á ndez-Manso, A.; Quintano, C. A synergetic approach to burned area mapping using maximum entropy modeling trained with hyperspectral data and VIIRS hotspots. Remote Sens. 2020,12, 858. [CrossRef] 46. De Bem, P.P.; De Carvalho, O.A., Jr.; Matricardi, E.A.T.; Guimar ã es, R.F.; Gomes, R.A.T. Predicting wildfire vulnerability using logistic regression and artificial neural networks: A case study in Brazil’s Federal District. Int. J. Wildland Fire 2019,28, 35–45. [CrossRef] 47. Mitsopoulos, I.; Mallinis, G. A data-driven approach to assess large fire size generation in Greece. Nat. Hazards 2017,88, 1591–1607. [CrossRef] Sensors 2020,20, 5423 23 of 23 48. Gorsevski, P.V.; Gessler, P.E.; Foltz, R.B.; Elliot, W.J. Spatial prediction of landslide hazard using logistic regression and ROC analysis. Trans. GIS 2006,10, 395–415. [CrossRef] 49. Boroughani, M.; Pourhashemi, S.; Hashemi, H.; Salehi, M.; Amirahmadi, A.; Asadi, M.A.Z.; Berndtsson, R. Application of remote sensing techniques and machine learning algorithms in dust source detection and dust source susceptibility mapping. Ecol. Inform. 2020,56, 101059. [CrossRef] 50. Chang, Z.; Du, Z.; Zhang, F.; Huang, F.; Chen, J.; Li, W.; Guo, Z. Landslide susceptibility prediction based on remote sensing images and GIS: Comparisons of supervised and unsupervised machine learning models. Remote Sens. 2020,12, 502. [CrossRef] 51. Boschetti, L.; Roy, D.P.; Giglio, L.; Huang, H.; Zubkova, M.; Humber, M.L. Global validation of the collection 6 MODIS burned area product. Remote Sens. Environ. 2019,235, 111490. [CrossRef] 52. Moreno-Ruiz, J.A.; Garc í a-L á zaro, J.R.; Arbelo, M.; Riaño, D. A comparison of burned area time series in the alaskan boreal forests from different remote sensing products. Forests 2019,10, 363. [CrossRef] 53. Fornacca, D.; Ren, G.; Xiao, W. Performance of Three MODIS fire products (MCD45A1, MCD64A1, MCD14ML), and ESA Fire_CCI in a mountainous area of Northwest Yunnan, China, characterized by frequent small fires. Remote Sens. 2017,9, 1131. [CrossRef] 54. Padilla, M.; Stehman, S.V.; Ramo, R.; Corti, D.; Hantson, S.; Oliva, P.; Alonso-Canas, I.; Bradley, A.V.; Tansey, K.; Mota, B.; et al. Comparing the accuracies of remote sensing global burned area products using stratified random sampling and estimation. Remote Sens. Environ. 2015,160, 114–121. [CrossRef] 55. Garc í a-L á zaro, J.R.; Moreno-Ruiz, J.A.; Riaño, D.; Arbelo, M. Estimation of burned area in the Northeastern Siberian boreal forest from a Long-Term Data Record (LTDR) 1982-2015 time series. Remote Sens. 2018 , 10, 940. [CrossRef] 56. Turco, M.; Herrera, S.; Tourigny, E.; Chuvieco, E.; Provenzale, A. A comparison of remotely-sensed and inventory datasets for burned area in Mediterranean Europe. Int. J. Appl. Earth Obs. Geoinf. 2019 ,82, 101887. [CrossRef] 57. Campagnolo, M.L.; Oom, D.; Padilla, M.; Pereira, J.M.C. A patch-based algorithm for global and daily burned area mapping. Remote Sens. Environ. 2019,232, 111288. [CrossRef] 58. Lizundia-Loiola, J.; Ot ó n, G.; Ramo, R.; Chuvieco, E. A spatio-temporal active-fire clustering approach for global burned area mapping at 250 m from MODIS data. Remote Sens. Environ. 2020 ,236, 111493. [CrossRef] 59. Rodrigues, J.A.; Libonati, R.; Pereira, A.A.; Nogueira, J.M.P.; Santos, F.L.M.; Peres, L.F.; Santa Rosa, A.; Schroeder, W.; Pereira, J.M.C.; Giglio, L.; et al. How well do global burned area products represent fire patterns in the Brazilian Savannas biome? An accuracy assessment of the MCD64 collections. Int. J. Appl. Earth Obs. Geoinf. 2019,78, 318–331. [CrossRef] 60. Loboda, T.V.; Hoy, E.E.; Giglio, L.; Kasischke, E.S. Mapping burned area in Alaska using MODIS data: A data limitations-driven modification to the regional burned area algorithm. Int. J. Wildland Fire 2011 ,20, 487–496. [CrossRef] © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).