SoilSuite for Europe - Data Documentation

SoilSuite for Europe Data and Method Description Uta Heidena,, Pablo d’Angeloa,, Paul Karlsh¨ofera,, Kevin Kuehla, a–German Aerospace Center (DLR), The Remote Sensing Technology Institute (IMF)˝ Muenchener Str.20, Wessling, 82234, Bavaria, Germany Keywords: bare surface, soil, spectral reflectance, Sentinel-2, soil variability, temporal statistics, composites How to cite this document: Heiden, U., d’Angelo, P., Karlshoefer, P., Kuehl, K., 2025. SoilSuite for Europe. Technical report, January 2025, URL: https://download. geoservice.dlr.de/SOILSUITE/files/EUROPE_5Y/000_Data_Overview/SoilSuite_Data_ Description_Europe_V1.pdf Email addresses: [email protected] (Uta Heiden), [email protected] (Pablo d’Angelo), [email protected] (Paul Karlsh¨ofer), [email protected] (Kevin Kuehl) 1. Purpose of the document The document provides a preliminary description of the SoilSuite data package published at the EOC Geoservice platform. It contains a short description of the data generation, data access, metadata, potential application and limitations of the SoilSuite data package. It shall enable the reader to evaluate the content and purpose of the data products and shall help to employ the data into further models at the best possible way. The authors of this data package welcome any feedback from the recipient of the data in order to improve the processors and thus the quality of the data. It is expected that the exchange will take place in a lively and informal manner. Please send any feedback to: •[email protected] •[email protected] •paul.karlsho[email protected] A detailed publication of the SoilSuite data package is in preparation and will be published soon. In the meantime, please use the following references: •Heiden, U., d’Angelo, P., Karlshoefer, P., Kuehl, K., 2025. SoilSuite for Europe. Technical report, January 2025, URL: https://download.geoservice.dlr.de/SOILSUITE/ files/EUROPE_5Y/000_Data_Overview/SoilSuite_Data_Description_Europe_V1. pdf •SoilSuite (2024): Sentinel-2 5-year (2018-2022) composites at European Scale, German Aerospace Center (DLR), 10.15489/qkud8cudg596. •Heiden, U., d’Angelo, P., Schwind, P., Karlsh¨ofer, P., M¨uller, R., Zepp, S., Wiesmeier, M., Reinartz, P., 2022. Soil Reflectance Composites—Improved Thresholding and Performance Evaluation. Remote Sens. 2022, 14, 4526. https://doi.org/10.3390/rs14184526. (Transfer of the basic bare soil reflectance composite concept to Sentinel-2 data and evaluation) •Rogge, D., Bauer, A., Zeidler, J., Mueller, A., Esch, T., Heiden, U., 2018. Building an exposed soil composite processor (SCMaP) for mapping spatial and temporal characteristics of soils with Landsat imagery (1984–2014), Remote Sensing of Environment, 205, 1-17, ISSN 0034-4257, https://doi.org/10.1016/j.rse.2017.11.004. (Description of the principle idea of the SoilSuite products based on Landsat data) 2 2. SoilSuite data package overview The SoilSuite contains a collection of different image data products that provide information about the spectral and statistical properties of European soils and other bare surfaces such as rocks. It is created using DLR’s Soil Composite Mapping Processor (ScMAP), which utilises the Sentinel-2 data archive. SCMaP is a specialised processing chain for detecting and analysing bare soils/surfaces on a large (continental) scale. Bare surface and soil pixels are selected using a combined NDVI and NBR index (PVIR2) that optimises the exclusion of photosynthetically active and non-active vegetation. The index is calculated and applied for each individual pixel. All SoilSuite products are calculated based on the available Sentinel-2 scenes recorded between January 2018 and December 2022 in Europe. The data package excludes all scenes with a cloud cover of >80 % and a sun elevation of <20°. The spectral composite products are calculated from the mean value after extensive removal of clouds, haze and snow effects at both scene and pixel level. The spectral data products are available at a pixel size of 20 m and contain 10 Sentinel-2 bands (B02, B03, B04, B05, B06, B07, B08, B08A, B11, B12). In the following, all image data products of the SoilSuite Europe are introduced. The quicklook of each data product shows a European subset that spans from Paris and the Champagne province in France at the left of the image to Western part of the Czech Republic. The full data set covers the area shown in Figure 1. 3 Bare Surface Reflectance Composite - Mean Product abbreviation [*] SRC.tif Data type Int16 NoData value -10.000 Number of bands 10 Pixel size 20 m File Format Cloud optimized GeoTiff (COG) Compression LZW Formula Rbare =1 |Rbare|X r∈Rbare r(1) where Rbare is the set of valid bare surface reflectances. Band 1 B2 bare surface reflectance average Band 2 B3 bare surface reflectance average Band 3 B4 bare surface reflectance average Band 4 B5 bare surface reflectance average Band 5 B6 bare surface reflectance average Band 6 B7 bare surface reflectance average Band 7 B8 bare surface reflectance average Band 8 B8A bare surface reflectance average Band 9 B11 bare surface reflectance average Band 10 B12 bare surface reflectance average General use of the product The Bare Surface Reflectance Composite (Mean) contains reflectance values scaled between 0 and 10.000. It represents the mean reflectance of all Sentinel-2 bare surface occurrences between 2018 - 2022 excluding urban areas, water bodies, clouds, haze, cloud shadows and snow. This product is one of the main inputs for spectral as well as digital soil mapping approaches. Table 1: The characteristics of the Bare Surface Reflectance Composite - Mean (SRC) product (RGB - B4/B3/B2) 4 Bare Surface Reflectance Composite - Standard Deviation Product abbreviation [*] SRC-STD.tif Data type Int16 NoData value -10 Number of bands 10 Pixel size 20 m File Format Cloud optimized GeoTiff (COG) Compression LZW Formula sRbare =s1 |Rbare|X r∈Rbare (r−Rbare)2(2) Band 1 B2 bare surface reflectance standard deviation Band 2 B3 bare surface reflectance standard deviation Band 3 B4 bare surface reflectance standard deviation Band 4 B5 bare surface reflectance standard deviation Band 5 B6 bare surface reflectance standard deviation Band 6 B7 bare surface reflectance standard deviation Band 7 B8 bare surface reflectance standard deviation Band 8 B8A bare surface reflectance standard deviation Band 9 B11 bare surface reflectance standard deviation Band 10 B12 bare surface reflectance standard deviation General use of the product The Bare Surface Reflectance Composite (Std Dev) contains the standard deviation of all Sentinel-2 bare surface reflectance occurrences between 2018 - 2022. It is scaled between 0 and 10.000 for each band and excludes urban areas, water bodies, clouds, haze, cloud shadows and snow. This product quantifies e.g. the spectral dynamic of soils, which might differ according to their physical and chemical properties and thus, can be taken as an additional characteristic of the soil. Table 2: The characteristics of the Bare Surface Reflectance Composite - Standard Deviation (SRC-STD) product (RGB - B4/B3/B2) 5 Bare Surface Reflectance Composite - 95% Confidence Product abbreviation [*] SRC-CI95.tif Data type Int16 NoData value -10 Number of bands 10 Pixel size 20 m File Format Cloud optimized GeoTiff (COG) Compression LZW Formula c=t1−α 2,df SRbare p|Rbare|(3) where tis the Student’s t-distribution, α= 0.05, df =nb−1 and SRbare is the corrected sample standard deviation of Rbare. The 95% confidence interval of the bare surface reflectance is then given by CI95% =Rbare ±c. Band 1 B2 bare surface reflectance 95% confidence Band 2 B3 bare surface reflectance 95% confidence Band 3 B4 bare surface reflectance 95% confidence Band 4 B5 bare surface reflectance 95% confidence Band 5 B6 bare surface reflectance 95% confidence Band 6 B7 bare surface reflectance 95% confidence Band 7 B8 bare surface reflectance 95% confidence Band 8 B8A bare surface reflectance 95% confidence Band 9 B11 bare surface reflectance 95% confidence Band 10 B12 bare surface reflectance 95% confidence General use of the product The Bare Surface Reflectance Composite (95% Confidence) contains the half-width of the 95% confidence interval (CI) of the Bare Surface Reflectance Composite. A large confidence interval conveys large uncertainty in the SRC value and a small interval indicates a high degree of certainty. Thus, the visible stripes coincides with the lower number of valid pixels (see product SFREQ-VPC), which makes the SRC measurement less reliable. It can be used to filter out pixels with a very high uncertainty (large confidence interval) from the model generation and/or from soil parameter prediction. Very high uncertainty can be especially found at the edges of agricultural fields, where pixels are more affected by spectral mixtures and lower number of bare surface occurrences. Table 3: The characteristics of the Bare Surface Reflectance Composite - 95% confidence (SRC-CI95) product (RGB - B4/B3/B2) 6 Bare Surface Statistics Product abbreviation [*] SFREQ.tif Data type Float32 NoData value -10 Number of bands 3 Pixel size 20 m File Format Cloud optimized GeoTiff (COG) Compression LZW Band 1 Bare surface frequency (BSF) General use of the product The Bare Surface Frequency (BSF) is scaled between 0 and 1 and quantifies the ratio of bare surface occurrences over the total number of valid Sentinel-2 observations |Rbare|/|R|in the observed time period between 2018 - 2022. The BSF is comparable across the European continent. In agricultural areas, the BSF gives an indication about the intensity of use of agricultural soils. It also can show areas, where bare surfaces are not expected such as in eroded grassland areas or clear-cuts in forests. Band 2 Bare surface count |Rbare|(BSC) General use of the product The BSC is defined as the number of occurrences detected as bare surface between 2018 - 2022. It is not normalized to the total number of valid Sentinel-2 observations. The striping visible in the BSC product reveals the dependence of the pixel position on the Sentinel-2 ground track that can lead to almost double observations in overlapping orbits. Therefore, the resulting BSC is not comparable across the European continent and should just be taken as an additional per pixel statistical information. 7 Band 3 Valid pixel count |R|(VPC) General use of the product This VPC contains the total number of valid Sentinel-2 observations in the time period between 2018 - 2022 excluding cloud, haze, cloud shadow, snow, etc.. Table 4: The characteristics of the Bare Surface statistics collected in the SFREQ product 8 Mask Product abbreviation [*] MASK.tif Data type Byte NoData value 0 Number of bands 1 Pixel size 20 m File Format Cloud optimized GeoTiff (COG) Compression LZW Band 1 Three-class mask: (1) Bare surface occurrence, (2) Permanent vegetation, (3) Water bodies, urban areas, roads etc. General use of the product The MASK product aggregates simple landcover classes. The value 1 stands for the bare surface occurrence class with pixels that are characterized by an alternation of vegetation and bare soil/surface. Especially areas used for agriculture fall into this mask. It should be noted that a pixel only get 1, if bare surface was detected at least 3 times in the time period to reduce noise in the data. The value 2 describes pixels with permanent vegetation such as coniferous forests and permanent grasslands and excludes areas with bare surface occurrences. As an example, a clear-cut in a forest could appear as value 1 and not value 2, if bare surface is detected at least 3 times. All other pixels receive the value 3, which collects pixels with permanent non-vegetation surface such as water bodies, urban areas, roads, etc.). Table 5: The characteristics of the MASK product. 9 Figure 3: Selective SCMaP products for Europe Processing parameter Value Explanation S2 Level 2A reflectance processor MAJA (Hagolle et al., 2021) Atmospheric correction optimized for long time series. S2 Level 2A cloud mask MG2 (MAJA output) Time range (year) 2018 - 2022 Time range (month) 1 - 12 Cloud cover filtering <80% Only scenes with less than 80 percent cloud cover were used during compositing Sun elevation filtering >20°Only scenes with more than 20 degree sun elevation were used during compositing Bad scene filtering applied Detect scenes where the L2A processor produced an invalid cloud mask. NDSI value No filtering based on NDSI was applied. Minimum soil count 3 A pixel falls into the SRC mask, if it is detected three times as bare soil. This mitigates the effect of spurious outliers. Spectral Index PV+IR2 (Heiden et al., 2022) Blue outlier filter σ= 4 (mean reflectance), σ= 3 (soil reflectance) Cloud and haze pixels not detected by the L2A cloud mask are removed. NIR/SWIR filter applied SCMaP revision 8925e96c Product version version 1-13 Output coordinate system EPSG:3035 / LAEA Europe Table 9: Processing parameter settings 16 5. Application and limitation of the data The SoilSuite data for areas in Europe is primarily developed as input for spectral and digital soil parameter models. The products comprise spectral reflectance characteristics of the predominant land cover (*SRC*, *MREF*), their spectral dynamics (*SRC-STD*, *MREF-STD*) and also statistical information (*SFREQ*) about the changes of these land covers. Further, information is provided to evaluate the spectral quality of the data (*SRC-CI95*). In the following, the meaningfulness, value and limitations are shortly and exemplarily discussed. 5.1. MASK product The MASK product contains three classes and is basically a by-product of SCMaP to generate the SRC and related products. It allows to detect pixels that show bare surfaces in the observed time period (soil-mask) and it also highlights area, where bare surfaces are never been visible from space in the observed time period (vegetation-mask). In the ESA WorldSoils project, these mask are the basis for assigning pixels to one of the two implemented soil models (digital soil model for vegetated soils and spectral soil model for bare soils). In mid-Europe, the soil mask can be taken as approximation for areas that are intensively used for agriculture. However, care should be taken in Southern areas, where agricultural activity is completely different due to the different climate conditions and crop types such as orchards or agroforestry. Although these areas are intensively used for food production, they do not show a very distinct and for mid-Europe typical change from vegetation to bareness. 5.2. SFREQ products The 3-band SFREQ product comprise three statistical products and are shown in Figure 4 for the example of the Carpathians. Figure 4: SFREQ products for the Carpathians (1) The bare surface frequency (band 1) is a proxy for the temporal length a surface is bare and thus is very important product for soil erosion, soil degradation and soil carbon sequestration processes. However, this exclude the ”activity” of these surfaces, since it does not quantify how often the surface cover changes from bareness to vegetation. It is valid for all areas and can be compared among different regions since it is normalized by the valid pixel count and thus, considers the areas of the overlapping orbits. Preliminary analyses have shown that this product is very significant in digital soil mapping approaches. (2) The bare surface count (band 2) is in combination with the valid pixel count (see below) the basis for the calculation of the bare surface frequency. It should not be used 17 as covariate in digital soil mapping approaches because it shows the different amount of pixels due to the overlapping orbits in form of stripes (see Figure 4). For this purpose, the bare surface frequency should be used. (3) The valid pixel count (band 3) shows the general amount of pixels that are available for the SCMaP processing and excludes clouds, cloud shadows, haze, snow and other data artefacts. This information is the basis for all subsequent products (e.g. SRC and MREF products). It clearly highlights the overlapping orbits in which it happens that the valid pixel count is almost doubled. It is an important information for the data distribution across the complete region and thus, the comparability and reliability of the products. It also highlights areas of permanent higher cloud cover (Northern Europe) compared to lower permanent cloud cover (Southern Europe). 5.3. MREF products MREF comprise the mean of all valid pixels and MREF-STD the standard deviation of the mean product. Both products contain valuable information for digital soil mapping approaches. An example is given in Figure 5. The MREF-STD reveals information about the spectral dynamic of the landscape. The reddish and whitish areas are characterised by a very high dynamic compared to the greenish grassland areas along the rivers. Due to the different soil of the peatbog area (histosol), the crop type and the dynamic of the agricultural cycle is different from the whitish areas. There is a high potential of this product for soil mapping approaches and other land cover classification approaches. Figure 5: MREF products for a subset in Bavaria, Germany; especially the MREF-STD product shows a clear difference between the alluvial soils along the rivers and the peatbog area of the Koenigsmoos; RGB: B4/B3/B2 It should be noted that the MREF-STD product is affected by the overlapping orbits and thus show minor striping effects as shown for the SFREQ valid pixel cound (band 3). The standard deviation values are correct of the complete area and stripes are not errors. However, additional preprocessing is necessary in order to reduce these BRDF effects and produce a comparable product also for the orbit overlaps. This is work in progress and not solved yet. For AI or DL models it is expected that this effect can be learned by introducing both, the valid pixel count as well as the standard deviation product. The striping effects are very minor and barely visible. 5.4. SRC products One of the main products of the SCMaP processor is the bare surface reflectance composite (SRC) that collects all bare surface pixels in a given time period using the index listed in Table 9. The product is a very important for soil parameter prediction models that 18 directly correlate the spectral reflectance of the soil/surface with the soil parameter from ground data. It is therefore essential to collect just purely bare surfaces and sorting out pixels with disturbances from green and dry vegetation as well as from remaining clouds and haze. The spectral index threshold definition technique HISET is especially adapted to those disturbances by being very strict in the bare surface selection (low index threshold). Figure 6: The SRC products contain pixels that covering the bare appearance of soils, for all black areas (no data), the soils are always covered with vegetation (such as for forest, grasslands, etc.); the left image shows the mean of all valid surface/soil pixels, the right image (standard deviation of the mean) reveal the temporal behavior of the different soil types, visualised in different colors. Care should be taken for areas that are characterised by small agricultural fields with a mixture of trees and low vegetation smaller than the 3 times the spatial resolution (3 x 3 pixel window). Orchards and agroforestry areas are also affected by spectral mixtures and thus, are mainly not included in the SRC products. They are correctly excluded due to spectral mixtures that would distort the pure soil spectral reflectance signal. This might happen in several areas of the Mediterranean (e.g. Southern Spain and Greece), where fields are often small or there are olive groves. In these areas, only few pixels are available for the SRC product (Figure 6). Care should be taken when checking the completeness of the SRC with the usual Google Maps because Google may show just one single snapshot in time and can deviate from the real bareness in the analysed time period. The pixel number for the SRC could be increased by increasing the spectral index threshold. However, this would integrate spectral information from green and dry vegetation and can have a negative impact on the soil modeling and mapping. Our suggestion is to test and use the MREF and the MREF-STD as alternative products in these areas for any soil modeling task. The idea for generating a standard deviation product from SRC (SRC-STD) is to quantify the different dynamics of surfaces / soils due to their varying mineral composition. Very clay-rich soil holds moisture much longer than a sandy soil and therefore has a different spectral dynamic that is captured in the SRC-STD product. It is an experimental product that can be tested for spectral as well as digital soil mapping approaches. The coverage is the same as for the SRC product. We also provide the 95% confidence interval of the SRC product (SRC-CI95). It contains the half-width of the 95% confidence interval (CI) of the SRC that contains Sentinel-2 reflectance values of all bare surface occurrences collected between 2018 - 2022. A large confidence interval conveys large uncertainty in the SRC value and a small interval indicates a high degree of certainty. Thus, the visible stripes coincides with the lower number of 19 valid pixels (see product SFREQ-VPC), which makes the SRC measurement less reliable. Moreover, higher values (high uncertainty) appear at the edges of agricultural fields, where pixels are more affected by spectral mixtures and lower number of bare surface occurrences. As an application example, it could be explored, if the filtering of pixels with a very low uncertainty (high confidence interval) from soil parameter model generation leads to more robust models. References M. L. Garcia and V. Caselles. Mapping burns and natural reforestation using thematic mapper data. Geocarto International, 6(1):31–37, 1991. doi: 10.1080/10106049109354290. URL https://doi.org/10.1080/10106049109354290. O. Hagolle, M. Huc, C. Desjardins, S. Auer, and R. Richter. Maja algorithm theoretical basis document, Mar. 2018. URL https://doi.org/10.5281/zenodo.1209633. O. Hagolle, J. Colin, S. Coustance, P. Kettig, P. D’Angelo, S. Auer, G. Doxani, and C. Desjardins. Sentinel-2 surface reflectance products generated by cnes and dlr: Methods, validation and applications. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, V-1-2021:9–15, 2021. doi: 10.5194/isprs-annals-V-1-20219-2021. URL https://isprs-annals.copernicus.org/articles/V-1-2021/9/2021/. U. Heiden, P. d’Angelo, P. Schwind, P. Karlsh¨ofer, R. M¨uller, S. Zepp, M. Wiesmeier, and P. Reinartz. Soil reflectance composites—improved thresholding and performance evaluation. Remote Sensing, 14, 9 2022. ISSN 2072-4292. doi: 10.3390/rs14184526. D. Rogge, A. Bauer, J. Zeidler, A. Mueller, T. Esch, and U. Heiden. Building an exposed soil composite processor (scmap) for mapping spatial and temporal characteristics of soils with landsat imagery (1984ˆa€“2014). Remote Sensing of Environment, 205:1–17, 2018. ISSN 0034-4257. doi: https://doi.org/10.1016/j.rse.2017.11.004. URL https: //www.sciencedirect.com/science/article/pii/S003442571730514X. J. Rouse, R. Haas, J. Schell, and D. Deering. Monitoring vegetation systems in the great plains with erts. In Third Earth Resources Technology Satellite–1 Syposium. Volume I: Technical Presentations, pages 309–317, NASASP-351, NASA, Washington, D.C., 1974. S.C. Freden, E.P. Mercanti, and M. Becker (eds). B. van Wesemael, A. Abdelbaki, E. Ben-Dor, S. Chabrillat, P. d’Angelo, J. A. Demattˆe, G. Genova, A. Gholizadeh, U. Heiden, P. Karlshoefer, R. Milewski, L. Poggio, M. Sabetizade, A. Sanz, P. Schwind, N. Tsakiridis, N. Tziolas, J. Yag¨ue, and D. Zizala. A european soil organic carbon monitoring system leveraging sentinel 2 imagery and the lucas soil data base. Geoderma, 452:117113, 2024. ISSN 0016-7061. doi: https://doi.org/10.1016/j.geoderma.2024.117113. URL https://www.sciencedirect. com/science/article/pii/S0016706124003422. 20