Technical note describing the Quality assessment of AWS data and identified improvements for the AWS data during SIOV

Technical note describing the Quality assessment of AWS data and identified improvements for the AWS data during SIOV Deliverable 7, European Space Agency Project -Performance Evaluation of Arctic Weather Satellite Data (No. 4000136511/21/NL/IA) Per Dahlgren1, Adam Dybbroe2, Stephanie Guedj1 1 MET Norway 2 SMHI Introduction The Arctic Weather Satellite was successfully launched 16 August 2024 and a longer LEOP (Launch and Early Operation Phase) phase started with initial testing and adjustment of orbit, during which the project team had no access to any data. Then, in October and November 2024 a limited data set of five orbits were delivered to us with observations from mid September 2024. With this data we could technically test our NWP assimilation with real AWS data and do a first evaluation of its performance. These data were then re-processed with an updated version of the ground level processor, so we could also do an evaluation of the original data set versus the re-processed data. The results were presented to the Scientific Advisory Group on 3 December 2024. In this document, these results will be presented. We will also present the effect of introducing side lobe corrections into the ground level processor, which was done 14 March 2025. The NWP system A number of developments to the HARMONIE-AROME Data Assimilation (H-A DA) system were made in WP2 (see [1]) in preparation for active use of AWS. For this work we use a branch of H-A with selected developments from WP2. We use a branch of the H-A NWP system based on version cy43. In this configuration, AWS data is used in L1C NetCDF format. The L1C files used were produced from L1B using the ESA processor [1] in which footprints from feedhorns 1,2 and 4 were mapped to the grid of feedhorn 3. I.e., data is mapped to the grid of the 183GHz channels. The first thing that happens in H-A DA is that the NetCDF file is read by a tool called BATOR which reads the input file and writes the contents into Observation Data Base (ODB) format. ODB is the format which the DA system uses. Then the Screening is done which is where model equivalents are computed and quality control of observations is performed. In terms of AWS, model equivalents means calculating the AWS brightness temperatures from NWP model profiles via RTTOV. In these experiments, RTTOV version 10 is used and the assimilation is done in clear sky mode. After Screening comes Minim in which the actual analysis is calculated by minimizing a cost function. Experiment configuration and domains used We use a 3 dimensional variational data assimilation (3D-Var) technique to perform the analysis and an analysis is done every third hour starting from 00UTC. The 3 hour forecast from each analysis is used as first guess for the next DA cycle. The horizontal grid is at 2.5 km resolution and there are 65 levels in the vertical and the model top is at 10hPa. We run over two separate domains; Arome Arctic (AA) which is operational at MET Norway, and METCOOP which is operational at both MET Norway and SMHI, see Figure 1. It should be clarified that we do not run the operational NWP model(s), we are using our own H-A configuration. ECMWF operational forecasts are used as lateral boundary conditions in the same way as our operational NWP production. The experiments are set up so that they can warm start using data from the routine operational NWP production. Warm start means that the first guess, in our case a 3 hour forecast, is taken from the operational runs. It also means that we take the variational bias correction coefficients from the operational suite(s). The benefit of warm starting from operational data is that it gives flexibility. At whatever date AWS data is available, we can start assimilation experiments on that date directly, assimilating not only AWS but other heritage sensors used in operations as well e.g. AMSU-A, MHS and ATMS. Figure 1. The two domains used in these NWP experiments. Black is the Arome Arctic domain and red is the METCOOP domain. AWS Assimilation - quality control In these experiments, AWS data is assimilated in passive mode. This means that AWS will undergo quality control (Screening) and enter the minimization (Minim) but the data will be prevented from influencing the analysis. Data is set to passive in the blacklisting, which is part of the quality control procedure. Screening also does a first guess check in which the observation minus the model equivalent, (O-B), is checked to see if it exceeds a preset threshold. If the deviation is too large, the observation is rejected. Radiances also undergo what is called cloud clearing. For microwave radiances this means checking if the observation has been influenced by scattering from rain or very dense clouds. When setting up the H-A DA system to assimilate AWS radiances we used the same type of cloud clearing as MHS, AMSU-A and ATMS, which looks at (O-B) for window channel(s). For AWS channels 2 (AWS-12 at 52.8GHz) and 10 (AWS-31 at 165.5 GHz) are used. If (O-B) exceeds 5K for any of these two channels; we reject all channels from that FOV1. This is a first crude setup that can be improved and made more sophisticated. For an early evaluation of the quality of AWS radiances we think this works well as we will get rid of most cloud affected radiances. Further refinement of cloud clearing will make more sense when the actual impact of active assimilation of AWS is being studied. The technical implementation of passive mode assimilation poses some problems for the first guess check. When an observation is flagged as passive, the observation error becomes automatically inflated. This, in turn, has the effect that the preset threshold in the first guess check becomes larger which means that it will be difficult to exceed the threshold. In these tests presented here it does not pose a major problem. The way cloud clearing is done will anyway remove most of the larger departures. When we go on to active assimilation of AWS, this inflation of the observation error will not occur. The 5 orbit test data set The five orbits of data were from 13 to 14 of September 2024 and in terms of use in our H-A model we had AWS data to assimilate in the following data assimilation cycles: 2024-09-13 15 UTC 2024-09-13 18 UTC 2024-09-13 21 UTC 2024-09-14 00 UTC Figures 2, 3 and 4 show maps with AWS data from the Arome Arctic domain at 18 UTC on 13 September 2024. These maps give us a first view of AWS data from within the H-A DA system. Figure 2 is from AWS channel 16, or running number 6, which measures at 54.94GHz and is a temperature sounding channel. For comparison, this channel has a 1 As we use L1c data where all four feedhorns are mapped to one grid, the 183 GHz grid, FOV in this context is only strictly a true field of view for the 183 GHz channels, all others are constructed from nearby actual FOVs, see [2] for more details. weighting function very similar to AMSU-A channel 7 and ATMS channel 8. Figure 3 shows AWS channel 26 (running number 15) which is a humidity sounding channel with similar weighting function as MHS channel 3 and ATMS channel 22. Both these figures show that model and observation show similar large scale patterns. The modeled brightness temperature fields are a bit smoother, but these figures are mainly shown as a sanity check where we can get a visual overview that things are working correctly. Figure 4 shows two of the novel 325 GHz channels and it can again be seen that model and observation agree in terms of the large scale patterns, even though we are using clear sky settings in this H-A DA setup. Figure 2. AWS channel 16 (running number 6), a temperature sounding channel. Left: observed brightness temperature. Right: NWP model equivalence calculated with RTTOV. Figure 3. AWS channel 36 (running number 15), a humidity sounding channel. Left: observed brightness temperature. Right: NWP model equivalence calculated with RTTOV. Figure 4. AWS channels 41 on the top line and 43 on the bottom line (running numbers 16 and 18 respectively). From the novel 325 GHz set of channels. Left: observed brightness temperature. Right: NWP model equivalence calculated with RTTOV. 5 orbit statistics When running through the 4 DA cycles it turned out we got more data in the Arome Arctic domain and therefore results from that domain will be shown here. There were no dramatic differences in the results in the different domains. Figure 5 shows the gaussian distribution of (O-B) and the effect of quality control. First we note that channels seem to behave quite well in general, and also that the offsets, or biases, are quite large. For the temperature sounding channels plotted, the lowest peaking ones, ch 4 and 5, have a bi-modal shape (Figure 5a) which is removed after quality control is applied (Figure 5b). AWS channel 18 (running number 8) has a large standard deviation which can be seen in its wide shape. This channel was out of specification even before launch and it is a known error for which a solution is known. In a possible future constellation; STERNA, this problem will not occur. Figure 5. Frequency distribution of (O-B) for AWS. a: Temperature sounding channels before quality control (QC) is applied. b: Temperature sounding channels after QC is applied. c: 183GHz humidity sensing channels before QC. d: 183 GHz sounding channels after QC Reprocessed 5 orbit data Further updates to the ground level processor were made and the five orbits of data were reprocessed and delivered to us for evaluation. Here we will look into the effect of these updates. First, it is clear that more of the reprocessed data passes the H-A DA quality control. In Figure 6 the number of observations before and after QC for both data sets are plotted as a function of scan position. There is only a small difference in terms of the total number of observations that goes into the NWP system (thick red and blue lines in Figure 6). After QC there is more data left in the reprocessed data set, thin blue line in Figure 6. This indicates that there is more good quality data in the reprocessed data set. In terms of gaussian distribution, Figure 7, the reprocessed data has significantly improved the biases. For the temperature sensing channels, Figure 7 a and b, the biases shift from positive to negative and the reprocessed data are less biased, i.e. closer to zero. The humidity sensing channels are also shifted closer to zero which means that they are less biased compared to the NWP model. This is even more clear if the mean of (O-B) is plotted as a function of scan position, Figure 8. Here we see again that the temperature sensing channels shift sign and the humidity sensing channels are closer to the zero line. The limited sample presented here makes it difficult to conclude if the shape of the scan dependent biases is representative of any instrument characteristics. Figure 6. Number of observations in original and reprocessed data sets. Y-axis: number of observations. X-axis: scan position. Thick red: all observations in the original (first) data set. Thin red: number of observations in original data set after QC. Thick blue: all observations in the reprocessed data set. Thin blue: number of observations in the reprocessed data set after QC. Figure 7. Gaussian distribution for original (a and c) and reprocessed data set (b and d). Data shown has not undergone QC in H-A DA. a and b: temperature sensing channels. c and d: humidity sensing channels. Figure 8. Scan dependent bias of (O-B) for original (a and c) and reprocessed data set (b and d). Data shown has not undergone QC in H-A DA. a and b: temperature sensing channels. c and d: humidity sensing channels. Effect of sidelobe correction In December 2024 a data stream was set up in which global AWS from Tromsø was transferred to Eumetsat and made available to early evaluators. We then got the data from the Eumetsat data store in L1B format and locally ran the industry processor to obtain L1C. After that, the data could be used in near real time monitoring runs with H-A. On 14 March a sidelobe correction scheme was implemented in the Spacecraft Characteristics DataBase (SCDB) and introduced in the ground level processor. The effect of introducing sidelobe corrections is shown in Figure 9 in which time series of average/bias (O-B) deviations are shown. For all channels, the biases are clearly reduced. For the temperature sensing channels, 4-8, biases are around between -0.6 and -0.9K. The humidity sensing channels have biases between -0.8 to less than 0.2K. The drop in number of observations comes from the introduction of 3x3 averaging of 50 GHz channels. Here it is done in BATOR (explained above), which throws away observations in the averaging process causing the number of observations to drop.