Field Calibration of a Low-Cost Air Quality Monitoring Device in an Urban Background Site Using Machine Learning Models

Citation: Apostolopoulos, I.D.; Fouskas, G.; Pandis, S.N. Field Calibration of a Low-Cost Air Quality Monitoring Device in an Urban Background Site Using Machine Learning Models. Atmosphere 2023,14, 368. https:// doi.org/10.3390/atmos14020368 Academic Editor: Kenichi Tonokura Received: 28 December 2022 Revised: 1 February 2023 Accepted: 8 February 2023 Published: 13 February 2023 Copyright: © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). atmosphere Article Field Calibration of a Low-Cost Air Quality Monitoring Device in an Urban Background Site Using Machine Learning Models Ioannis D. Apostolopoulos 1, George Fouskas 1and Spyros N. Pandis 1,2,* 1Institute of Chemical Engineering Sciences (ICE-HT), Foundation for Research and Technology Hellas (FORTH), 265 04 Patras, Greece 2Department of Chemical Engineering, University of Patras, 265 04 Patras, Greece *Correspondence: spyr[email protected] Abstract: Field calibration of low-cost air quality (AQ) monitoring sensors is essential for their successful operation. Low-cost sensors often exhibit non-linear responses to air pollutants and their signals may be affected by the presence of multiple compounds making their calibration challenging. We investigate different approaches for the field calibration of an AQ monitoring device named ENSENSIA, developed in the Institute of Chemical Engineering Sciences in Greece. The present study focuses on the measurements of two of the most important pollutants measured by ENSENSIA: NO 2 and O 3 . The measurement site is located in the center of Patras, the third biggest city in Greece. Reference instrumentation used for regulatory purposes by the Region of Western Greece was used as the evaluation standard. The sensors were installed for two years at the same locations. Measurements from the first year (2021) from seven ENSENSIA sensors (NO 2 , NO, O 3 , CO, PM 2.5 , temperature and relative humidity) were used to train several Machine Learning (ML) and Deep Learning (DL) algorithms. The resulting calibration algorithms were assessed using data from the second year (2022). The Random Forest algorithm exhibited the best performance in correcting O 3 and NO 2 . For NO 2 the mean error was reduced from 9.4 ppb to 3 ppb, whilst R 2 improved from 0.22 to 0.86. Similar results were obtained for O 3 , wherein the mean error was reduced from 13 to 4.3 ppb and R 2 increased from 0.52 to 0.69. The Long-Short Term Memory Network (LSTM) also showed good performance in correcting the measurements of the two pollutants. Keywords: air quality monitoring; low-cost sensors; field calibration; machine learning 1. Introduction Monitoring outdoor and indoor air quality (AQ) is an urgent need in the developed and developing world [ 1 ]. Real-time monitoring can provide many societal benefits and help improve human health. Knowledge of the prevailing AQ conditions can reduce respiratory, cardiovascular and other problems [2,3], especially for sensitive groups. Two major gaseous pollutants affecting human health are ozone (O 3 ) and nitrogen dioxide (NO 2 ). Ozone has been found to trigger various health problems, such as chest pain, coughing, throat irritation, and congestion. Bronchitis, asthma, and emphysema can worsen due to exposure to high ozone levels. Long-time repeated exposure to ozone may permanently scar the lung tissues [ 4 ]. Nitrogen dioxide also has a range of harmful effects on our lungs, including (i) increased inflammation of the airways; (ii) cough and wheezing; (iii) reduced lung function; (iv) increased asthma attacks, and (v) greater likelihood of emergency department and hospital admissions [ 5 ]. Asthmatics, children, and older adults are especially sensitive to these air pollutants. Monitoring gaseous pollutant concentrations across wide areas is costly and requires sensitive instruments and the corresponding infrastructure for the monitoring stations. Deploying several monitoring stations becomes prohibitively expensive for large urban and industrial areas [ 6 ]. In addition, precision instruments can cost tens of thousands of euros [1]. Atmosphere 2023,14, 368. https://doi.org/10.3390/atmos14020368 https://www.mdpi.com/journal/atmosphere Atmosphere 2023,14, 368 2 of 14 Several low-cost sensors have recently been developed to record and monitor gaseous pollutants and aerosols. These sensors are characterized as low-cost, and their name suggests their main benefit. Low-cost sensors have attracted the interest of the research community, regulatory bodies, and companies [ 7 – 9 ]. Despite their disadvantages, such as low accuracy, frequent errors, and periods of instability [ 10 ], these sensors are certainly a major step in the effort to monitor AQ with high spatial resolution in large urban areas, but also to start air pollutant measurements in some parts of the developing world. Data analytics and low-cost sensor reading corrections are topics of current research. Popular low-cost sensors such as the Alphasense NO2-B43F and OX-B431, which measure NO 2 and O 3, respectively, occasionally exhibit a non-linear relationship with the reference measurements. Our hypothesis in this work is that the raw measurements of low-cost sensors can be corrected with the aid of Machine Learning (ML) and Deep Learning (DL) algorithms to attain an acceptable agreement with the ground truth. Several recent studies have assessed the performance of NO2-B43F and OX-B431 sensors. Zimmerman et al. [ 11 ] trained and tested Random Forest (RF) and Multiple Linear Regression (MLR) models for a six-month period in Pittsburgh. They reported that the RF algorithm was suitable for calibrating the two sensor types by integrating the raw sensor responses of all the deployed sensor types (CO-B4, COZIR-2000 CO 2 , NO2-B43F, and OX-B431), as well as the ambient temperature and relative humidity. The RF-calibrated NO2and O3concentrations exhibited an R2of 0.62 and 0.86, respectively. Ratingen et al. [ 12 ] employed MLR to calibrate the NO2-B43F sensor using the raw NO2-B43F, OX-B431 measurements, and ambient temperature as inputs in the Netherlands. The calibration methodology was evaluated based on a ten-fold cross-validation procedure. MLR improved the R 2 of hourly measurements from 0.54 to 0.69–0.84. Han et al. [ 13 ] collocated Alphasense B4 sensors for CO, NO 2 , O 3 , and SO 2 in Beijing. They used the working and auxiliary electrode signals in conjunction with the temperature and relative humidity to evaluate MLR, RF, and Long-Short Term Memory (LSTM) algorithms. RF and LSTM increased the R2from 0.3–0.5 to 0.7 for both O3and NO2. Christakis et al. [ 14 ] used linear corrections for an NO 2 (NO2-B43F) and an O 3 (OXB431) sensor in Athens. The linear calibration for each sensor integrated the sensor’s working and auxiliary electrode signals, along with a temperature-compensation factor. R 2 increased from 0.45 to 0.84 for the NO 2 sensor and from 0.61 to 0.87 for the O 3 sensor. Margaritis et al. [ 15 ] reported that the RF algorithm had better performance than other algorithms tested in Thessaloniki, Greece improving R 2 for both sensors (NO2-B43F, OXB431) to 0.92 and 0.96, respectively. The Institute of Chemical Engineering Sciences (ICE-HT) has developed a low-cost AQ monitoring device, ENSENSIA, equipped with low-cost sensors for several air pollutants including NO 2 and O 3 . The device is analytically described in [ 16 ]. In our previous work, the proposed device completed the preliminary tests, which included stability and continuous operation experiments, and reading agreement with equivalent low-cost commercially available devices. In this work, we present the results of the field calibration of ENSENSIA for NO 2 and O 3 . The device was co-located with reference instrumentation provided by the Region of Western Greece at the center of Patras, Greece. The period of the measurements (1/1/2021 to 30/10/2022) covers all seasons and meteorological conditions for a typical city in southeastern Europe. We present and discuss the operation, the advantages, and the limitations of several ML and DL algorithms that were used to correct the readings of our device. The conducted tests reveal that the low-cost sensor readings can indeed be improved with the use of such algorithms, especially with the development of ensemble methodologies. 2. Materials and Methods 2.1. Environment Sensing Appliance (ENSENSIA) The low-cost sensor device of the current study entitled “ENSENSIA” has been developed by FORTH/ICE-HT and is analytically described in [ 16 ]. ENSENSIA includes Atmosphere 2023,14, 368 3 of 14 aerosol, gas and meteorological sensors to measure fine particulate matter (PM 2.5 , mass of particle with diameter smaller than 2.5 µ m), ozone, nitrogen oxide and dioxide, and carbon monoxide. Table 1summarises the types of ENSENSIA sensors. Table 1. Description of the low-cost sensors installed in ENSENSIA in this study. Units Sensor Model Range Manufacturer Ozone ppb OX-B431 0 to 200 ppb Alphasense Nitrogen Dioxide ppb NO2-B43F 0 to 200 ppb Alphasense Nitric Oxide ppb NO-B4 0 to 200 ppb Alphasense Carbon Monoxide ppb CO-B4 0 to 2000 ppb Alphasense Fine Particle Matter µg m−3PMS5003 0 to 500 µg m−3Plantower Temperature Celsius BME680 −40 to 85 ◦C Bosch Sensortec Relative Humidity % BME680 0 to 100 % Bosch Sensortec It is a Raspberry PI-based device that offers several edge-computing opportunities, stability of use, and remote control. 2.2. Measurement Site The urban background measurement site is located in the center of Patras, Greece, at latitude 38 ◦ 14 0 45.976 00 and longitude 21 ◦ 44 0 8.036 00 (Figure 1). ENSENSIA was placed on the roof of a small structure used as a regulatory AQ monitoring station by the Region of Western Greece. The device was placed approximately 6 m above ground and next to the inlets of the reference monitors. The reference monitor is the model HORIBA 360, which measures CO, SO2, NOx, and O3. Atmosphere 2023, 14, x FOR PEER REVIEW 3 of 15 2. Materials and Methods 2.1. Environment Sensing Appliance (ENSENSIA) The low-cost sensor device of the current study entitled “ENSENSIA” has been developed by FORTH/ICE-HT and is analytically described in [16]. ENSENSIA includes aerosol, gas and meteorological sensors to measure fine particulate matter (PM2.5, mass of particle with diameter smaller than 2.5 μm), ozone, nitrogen oxide and dioxide, and carbon monoxide. Table 1 summarises the types of ENSENSIA sensors. Table 1. Description of the low-cost sensors installed in ENSENSIA in this study. Units Sensor Model Range Manufacturer Ozone ppb OX-B431 0 to 200 ppb Alphasense Nitrogen Dioxide ppb NO2-B43F 0 to 200 ppb Alphasense Nitric Oxide ppb NO-B4 0 to 200 ppb Alphasense Carbon Monoxide ppb CO-B4 0 to 2000 ppb Alphasense Fine Particle Matter μg m −3 PMS5003 0 to 500 μg m −3 Plantower Temperature Celsius BME680 −40 to 85 °C Bosch Sensortec Relative Humidity % BME680 0 to 100 % Bosch Sensortec It is a Raspberry PI-based device that offers several edge-computing opportunities, stability of use, and remote control. 2.2. Measurement Site The urban background measurement site is located in the center of Patras, Greece, at latitude 38°14′45.976″ and longitude 21°44′8.036″(Figure 1).Error! Reference source not found.ENSENSIA was placed on the roof of a small structure used as a regulatory AQ monitoring station by the Region of Western Greece. The device was placed approximately 6 m above ground and next to the inlets of the reference monitors. The reference monitor is the model HORIBA 360, which measures CO, SO2, NOx, and O3. Figure 1. Map and photos of the measurement site in Patras, Greece. Measurements by ENSENSIA and the regulatory monitors over a 22-month period starting on 1 January 2021 to 30 October 2022 are analyzed in the present work. 2.3. Data Preprocessing The device’s sensors measure every 10 s and report their average measurements every 2 min. ENSENSIA is configured to provide a collection of the sensors’ readings Figure 1. Map and photos of the measurement site in Patras, Greece. Measurements by ENSENSIA and the regulatory monitors over a 22-month period starting on 1 January 2021 to 30 October 2022 are analyzed in the present work. 2.3. Data Preprocessing The device’s sensors measure every 10 s and report their average measurements every 2 min. ENSENSIA is configured to provide a collection of the sensors’ readings every two minutes, and its operation is continuous. For the data analysis of the present study, the hourly averages of ENSENSIA readings are used. All gas-phase concentrations are reported as molecular mixing ratios in parts per billion (ppb) units. All sensor readings and the corresponding reference concentrations were stored in a MySQL database after processing the output files of the reference monitors. ENSENSIA reported its readings directly to the database. During the analysis, outlier values of both the regulatory instrumentation and ENSENSIA have been removed. Negative and missing values were excluded from the analysis. Atmosphere 2023,14, 368 4 of 14 The resulting data completeness over the 22-month period, was 91.8% for the ENSENSIA device and 91.2% for the reference instrumentation. The aggregation yielded a completeness of 88.5%. We considered all available sensor readings for applying ML and DL algorithms when correcting either NO 2 or O 3 . Subsequently, the input data of the algorithms include all the NO 2 , O 3 , NO, CO, PM 2.5 , T, and RH values, as reported by ENSENSIA. The measurements reported by the regulatory instrumentation are considered as the ground truth. 2.4. Machine Learning and Deep Learning The present study evaluates several ML and DL methods to correct the measured gas concentrations based on their reference counterparts. Conventional methods and ML and DL methods are employed to correct the readings of the low-cost device. In the following paragraphs, brief descriptions of these methods are provided. 2.4.1. Linear Correction A linear relation between the reference concentration and the multiple low-cost sensor readings was assumed in this approach: NO2,re f erence =aNO2,raw +bNOraw +cCO.raw +dO3,raw +eT+fRH +gPM2.5,raw O3,re f erence =aO3,raw +bNO2,raw +cNOraw +dCO.raw +eT+fRH +gPM2.5,raw where “raw” denotes the uncorrected low-cost sensor readings. 2.4.2. K-Nearest Neighbors The k-nearest neighbors (KNN) algorithm is one of the most naïve ML algorithms. KNN uses feature similarity to estimate the value of the new data points based on how closely they resemble the points in the training set [ 17 ]. KNN computes the distance between the new points and the training data points to achieve this. Several methods calculate the distance between data points, including Euclidean, Manhattan, and Hamming distances. The hyperparameter k, which is included in the name of the method, refers to the number of neighbors used to estimate any new point’s value. The corrected concentration value of NO 2 is expected to fall near the closest neighbor, i.e., the set of NO 2 , NO, CO, O 3 , T, RH, PM 2.5 which is most similar to the set under investigation. The Euclidean distance was used for the analysis. One of the main setbacks of KNN is that it is sensitive to outliers as it chooses the neighbors based on distance criteria. Table 2presents the KNN parameters tuned for this study. Table 2. Parameters and hyper-parameters of the ML and DL models. Method Parameters KNN Neighbors: 100 Leaf Size: 30 Distance Metric: Euclidean RF Min. split samples: 2 Bootstrap: Yes Estimators: 1000 Criterion: MSE NN Hidden layers: 2 Activation: Sigmoid Dropout: 50% Loss: MSE Optimizer: Stochastic Gradient Descent LSTM Steps: 24 LSTM units: 60/120 Dropout: 50% Loss: MAE Optimizer: Adam CNN Steps: 24 Layers: 3 Activation: RELU Filters (size): 24 (kernel = 3), 48 (kernel = 3), 120 (kernel = 3) Dropout: 50% Loss: MSE Optimizer: Adam Atmosphere 2023,14, 368 5 of 14 2.4.3. Random Forest Random Forest is an ensemble learning method for classification and regression [ 18 ]. It constructs several decision trees (estimators) that process the input data and provide their predictions independently. Each decision tree aims to reveal rules that match the observed concentrations (x) to the expected concentrations (y). For example, a tree may declare that when x is below a specific threshold (e.g., 5 ppb), it will be treated differently from the rest of the input data. This is called a split. The decision tree may perform many similar splits. In problems involving many attributes, the decision tree may perform such splits using those attributes. The algorithm receives a set of inputs to correct the O 3 readings. In our case, those inputs include all the ENSENSIA readings (CO, NO, NO 2 , O 3 , T, RH, PM 2.5 ). RF constructs several trees (estimators) and trains them to estimate the “correct” ozone concentration. The latter is an ensemble of decision trees, which constitute the “forest”. Usually, those trees are trained with the bagging method [ 19 ]. RF introduces randomness to its operation. Each estimator is trained on subsets of the initial data (bootstrap method) and becomes a specialized, but weak, learner. The general idea of the bagging method is that a combination of learning models improves the overall result. RF aggregates the predicted concentrations provided by the decision trees. The final prediction can be decided by several methods, such as majority voting or mean [ 20 ]. RF is expected to perform the necessary splits to predict the reference concentration, but also to determine the importance of each feature to the algorithm’s decision. In its current implementation, RF consists of 1000 estimators, which are selected by performing extensive tests using the grid-search algorithm. The importance of each feature is reported and used to verify the cross-sensitivity among the electrochemical sensors. 2.4.4. Artificial Neural Network NN or Artificial NN (ANN) is a network of neurons. Its architecture is inspired by the biological functions of the human brain. Each node or artificial neuron connects to another and has an associated weight and threshold [ 17 ]. If the output of any individual node is above a specified threshold value, that node is activated, sending its output to the next layer of the network. Once an input layer is determined, weights are assigned. Several hidden layers of neurons can process the input data. Neural Networks learn from the input data in batches, adjusting the neuron weights to minimize a specific error function. For air pollutant time-series, error functions usually are based on the mean absolute or the mean squared error of the predicted concentrations compared to the more accurate measurements. The proposed NN consists of an input layer of 7 nodes connected to the input variables (CO, NO, NO 2 , O 3 , T, RH, PM 2.5 ). Two hidden layers of 24 and 48 nodes follow, in which the input variables are combined to discover inner relations among them. The output layer is a single node representing the predicted concentration. The prediction is compared to the accurate concentration (the reference) and an error is computed. This error is back-propagated to the nodes of the network, forcing the weight values to be updated in a way that the network improves its performance. Since this is performed for several data points, the algorithm needs to converge to a solution. This is achieved by the use of optimization methods that update the learning parameters. The selection of a suitable optimizer is essential to efficiently train a NN. The current NN implementation used the Stochastic Gradient Descent optimizer. Our choice was made after testing several alternative optimizers, such as Adam, RMSprop, and Adagrad for the corresponding problem. NNs require large-scale datasets to learn effectively. The desired data size increases non-linearly with the depth of the network and the trainable parameters. Small datasets often cause the model to underfit, i.e., fail to learn, mandating the selection of a shallower network, or performing data augmentation. Limitations of the NNs include their black-box nature, their frequent failure to generalize and predict unseen data derived from external validation tests, and their timeconsuming training. Atmosphere 2023,14, 368 6 of 14 2.4.5. Long-Short Term Memory Network LSTM is a Recurrent Neural Network [ 21 , 22 ] used in several DL tasks. LSTM involves feedback connections among the units, allowing the processing of entire sequences of data, rather than single data points. The basis of a conventional LSTM cell includes four main gates: input gate, input modulation gate, forget gate and output gate. The input gate is responsible for processing incoming data points. The memory cell input gate receives the output of the LSTM cell during the last iterations. The Forget gate decides when to forget the output results and thus selects the optimal time lag for the input sequence. The output gate receives all the results from the previously mentioned gates and returns the output. LSTM operates similarly to a NN. The strongest feature of LSTM is its inherent ability to discover timerelated connections between data points, which is desirable when dealing with time-series regression problems. LSTM is implemented using two units that process batches of 24 data points. To this end, after the training is complete, the prediction of the current NO 2 or O 3 concentration requires the current raw ENSENSIA measurement of NO 2 , O 3 , NO, CO, PM 2.5 , T, and RH as well as the latest 23 raw ENSENSIA measurements of NO 2 , O 3 , NO, CO, PM 2.5 , T, and RH. The model uses two hidden layers of 60 and 120 nodes. The error function uses the mean absolute concentration error units to update the weights. The Adam optimization algorithm was selected after tests with the Stochastic Gradient Descent, and RMSprop. The parameters of the LSTM model are presented in Table 2. A potential weakness of this model is related to data completeness. The latest 23 ENSENSIA measurements may not correspond to uniform time intervals. As a result, unexpected data representations may appear in the trained model reducing its performance. In addition, LSTM shares the limitations of neural networks. 2.4.6. Convolutional Neural Network CNN [ 23 ] is a well-known deep learning architecture inspired by living creatures’ natural visual perception mechanism [ 24 ]. It is a special NN type named after the linear mathematical operation between matrices called convolution. Components of CNNs include several layers, such as the convolution layer, their cornerstone, pooling and activation layers, and densely connected layers. The convolution layer is built by several convolution kernels used to compute feature maps. Specifically, each neuron of a feature map is connected to a region of neighboring neurons in the previous layer. The feature map is generated by first convolving the input with a learned kernel and then applying an element-wise non-linear activation function on the convolved results. Modern CNNs consist of convolution layers that process the input data hierarchically and learn high-level features that map those data with the desired outputs. The proposed CNN makes use of 2D convolutions to process a sequence of measurements (24), similar to the LSTM model. As a result, the input data representation is similar to an image array. The input data is progressively filtered using 24, 48, and 120 filters with a 3 × 3 kernel. The resulting one-dimensional vector is used for the regression. The mean squared error function is utilized for the weights which are updated using the Adam optimization algorithm. CNN learning is similar to NN and shares the NN’s limitations. Table 2presents the parameters and hyper-parameters of the implemented CNN. 2.5. Experiment Setup and Performance Metrics Each ML and DL model is defined by several hyper-parameters, which describe its functionality and affect the performance given a specific regression task. We have performed offline parameter tuning to define the optimal values for the model’s parameters. Those are summarized Table 2. We divide the measurement period into two campaigns. The first campaign refers to 2021 and is used to train and evaluate the algorithms. The second campaign (2022) is utilized for the test set. Training and validation on the 2021 campaign are performed under Atmosphere 2023,14, 368 7 of 14 a five-fold cross-validation train-test scheme, as suggested in [ 10 ]. An overview of the experiment is illustrated in Figure 2. Atmosphere 2023, 14, x FOR PEER REVIEW 7 of 15 gressively filtered using 24, 48, and 120 filters with a 3 × 3 kernel. The resulting one-dimensional vector is used for the regression. The mean squared error function is utilized for the weights which are updated using the Adam optimization algorithm CNN learning is similar to NN and shares the NN’s limitations. Table 2 presents the parameters and hyper-parameters of the implemented CNN. 2.5. Experiment Setup and Performance Metrics Each ML and DL model is defined by several hyper-parameters, which describe its functionality and affect the performance given a specific regression task. We have performed offline parameter tuning to define the optimal values for the model’s parameters. Those are summarized Table 2. We divide the measurement period into two campaigns. The first campaign refers to 2021 and is used to train and evaluate the algorithms. The second campaign (2022) is utilized for the test set. Training and validation on the 2021 campaign are performed under a five-fold cross-validation train-test scheme, as suggested in [10]. An overview of the experiment is illustrated in Figure 2. Figure 2. Training and evaluation/validation overview. A series of evaluation metrics are used in our analysis. The Mean Error (ME) and Root Mean Squared Error (RMSE) are accuracy metrics that give the average errors concerning the reference measurements. R (correlation coefficient) and R2 (coefficient of determination) are correlation parameters that describe the strength of the relationship between the sensor readings and the reference measurements. The normalized ME (nME) is a relative error metric that also provides useful information about the performance of our sensors and correction algorithms. Finally, mean bias (MB) quantifies the bias of the sensor. Studies suggest that at least ME, RMSE, R, R2, and nME should be reported [10]. The above metrics are calculated using the following equations: ME = ∑|Ei −Oi| n i=1 n RMSE = �∑(Ei −Oi)2 n i=1 n MB = ∑(Oi −Ei) n i=1 n nME = ∑|𝑂𝑂𝑂𝑂− 𝐸𝐸𝑂𝑂| 𝑟𝑟 𝑖𝑖=1 ∑𝑂𝑂𝑂𝑂 𝑟𝑟 𝑖𝑖=1 where the n measurements of the reference instruments (observed values) are denoted by Oi and the corrected sensor values (expected values) by Ei). Figure 2. Training and evaluation/validation overview. A series of evaluation metrics are used in our analysis. The Mean Error (ME) and Root Mean Squared Error (RMSE) are accuracy metrics that give the average errors concerning the reference measurements. R (correlation coefficient) and R 2 (coefficient of determination) are correlation parameters that describe the strength of the relationship between the sensor readings and the reference measurements. The normalized ME (nME) is a relative error metric that also provides useful information about the performance of our sensors and correction algorithms. Finally, mean bias (MB) quantifies the bias of the sensor. Studies suggest that at least ME, RMSE, R, R2, and nME should be reported [10]. The above metrics are calculated using the following equations: ME =∑n i=1|Ei−Oi| n RMSE =s∑n i=1(Ei−Oi)2 n MB =∑n i=1(Oi−Ei) n nME =∑n i=1|Oi−Ei| ∑n i=1Oi where the nmeasurements of the reference instruments (observed values) are denoted by Oiand the corrected sensor values (expected values) by Ei). 3. Results 3.1. Nitrogen Dioxide During the 2021 campaign, the observed (reference) mean and standard deviation of NO 2 concentration was 12.5 ± 8.4 ppb. The minimum and maximum observed values were 1 and 44 ppb, respectively. The ML algorithms were trained under a 5-fold cross-validation procedure in the 2021 campaign. The trained models were evaluated on the unseen data of the 2022 campaign. During this period, the observed mean and standard deviation of NO 2 concentration was 12.8 ±8.5 ppb . The minimum and maximum observed values were 0.5 and 45 ppb, respectively. 3.1.1. Evaluation of Uncorrected Sensor Readings Considering the uncorrected measurements of ENSENSIA, the ME of the two periods were 8.6 and 9.4 ppb, respectively. Table 3summarizes the results. Atmosphere 2023,14, 368 8 of 14 Table 3. Evaluation metrics for the uncorrected ENSENSIA measurements of ozone against the reference. The metrics were calculated based on hourly-averaged values. RR2ME (ppb) RMSE (ppb) MB (ppb) nME Train (2021 Campaign) 0.47 0.22 8.6 10.7 6.2 0.65 Test (2022 Campaign) 0.47 0.22 9.4 11.6 7.3 0.67 R2for both years remained at 0.22 and the nME was also similar (65% and 67%) in the two periods. 3.1.2. Evaluation of Machine Learning Algorithms The ML algorithms were trained and validated on the 2021 Campaign under a 5fold cross-validation procedure. The 2022 Campaign was used to assess the algorithms’ efficiency in predicting unseen data. The results are presented in Table 4. Table 4. Agreement metrics between the reference NO 2 concentrations and the corrected ENSENSIA readings. Data R R2ME (ppb) RMSE (ppb) MB (ppb) nME LR. Training (2021) 0.76 0.58 4.1 6.0 0.7 0.36 Test (2022) 0.85 0.72 3.7 4.8 1.3 0.33 KNN Training (2021) 0.82 0.67 3.6 4.8 0.0 0.34 Test (2022) 0.86 0.74 3.3 4.4 0.0 0.32 RF Training (2021) 0.87 0.75 3.1 4.2 0.4 0.31 Test (2022) 0.91 0.86 3.0 3.9 1.7 0.30 NN Training (2021) 0.81 0.68 3.9 5.3 0.0 0.34 Test (2022) 0.83 0.69 4.0 4.9 1.0 0.34 CNN Training (2021) 0.82 0.68 3.5 5.0 0.0 0.32 Test (2022) 0.85 0.72 3.5 4.6 1.0 0.33 LSTM Training (2021) 0.89 0.78 2.8 3.9 0.0 0.26 Test (2022) 0.9 0.82 3.0 3.9 1.8 0.31 Figure 3displays the uncorrected NO 2 sensor readings versus the observed reference NO 2 concentration and the RF-corrected NO 2 sensor readings versus the observed reference NO2concentration. Atmosphere 2023, 14, x FOR PEER REVIEW 9 of 15 Figure 3. (a) Uncorrected hourly NO2 sensor readings versus the observed reference NO2 concentration, (b) RF-corrected NO2 sensor readings versus the observed reference NO2 concentration. Both diagrams refer to the test set (2022). The RF algorithm demonstrated very good performance. The uncorrected NO2 sensor reading for the test period showed a ME, R2, and nME of 9.4 ppb, 0.22, and 0.67, respectively (Table 3). After the correction with the RF algorithm, the ME, R2, and nME were 3 ppb, 0.86, and 0.3, respectively (Table 4). The LSTM algorithm also had good performance in correcting the NO2 concentrations of the test set. After the correction with the LSTM algorithm, the ME was 3 ppb, the R2 was 0.82 and nME was 0.31. Figure 4 shows the time series of NO2 concentrations for two characteristic 7-day periods. Figure 4. Time series of NO2 concentrations for two 7-day periods of the test set (2022). Uncorrected NO2 low-cost sensor reading is displayed with blue lines, RF-corrected NO2 values are displayed with black lines, and reference NO2 concentrations are displayed with red lines. The RF algorithm also calculates the importance of each input attribute to its prediction. RF selects the raw reading of CO as its most important measurement (48% importance). The NO measurement is rated at 30%, whilst the raw NO2 reading’s importance is valued at 8%. These relations are summarized in Figure 5. Figure 3. ( a ) Uncorrected hourly NO 2 sensor readings versus the observed reference NO 2 concentration, ( b ) RF-corrected NO 2 sensor readings versus the observed reference NO 2 concentration. Both diagrams refer to the test set (2022). The RF algorithm demonstrated very good performance. The uncorrected NO 2 sensor reading for the test period showed a ME, R 2 , and nME of 9.4 ppb, 0.22, and 0.67, respectively (Table 3). After the correction with the RF algorithm, the ME, R 2 , and nME were 3 ppb, 0.86, and 0.3, respectively (Table 4). The LSTM algorithm also had good performance Atmosphere 2023,14, 368 9 of 14 in correcting the NO 2 concentrations of the test set. After the correction with the LSTM algorithm, the ME was 3 ppb, the R2was 0.82 and nME was 0.31. Figure 4shows the time series of NO 2 concentrations for two characteristic 7-day periods. Atmosphere 2023, 14, x FOR PEER REVIEW 9 of 15 Figure 3. (a) Uncorrected hourly NO2 sensor readings versus the observed reference NO2 concentration, (b) RF-corrected NO2 sensor readings versus the observed reference NO2 concentration. Both diagrams refer to the test set (2022). The RF algorithm demonstrated very good performance. The uncorrected NO2 sensor reading for the test period showed a ME, R2, and nME of 9.4 ppb, 0.22, and 0.67, respectively (Table 3). After the correction with the RF algorithm, the ME, R2, and nME were 3 ppb, 0.86, and 0.3, respectively (Table 4). The LSTM algorithm also had good performance in correcting the NO2 concentrations of the test set. After the correction with the LSTM algorithm, the ME was 3 ppb, the R2 was 0.82 and nME was 0.31. Figure 4 shows the time series of NO2 concentrations for two characteristic 7-day periods. Figure 4. Time series of NO2 concentrations for two 7-day periods of the test set (2022). Uncorrected NO2 low-cost sensor reading is displayed with blue lines, RF-corrected NO2 values are displayed with black lines, and reference NO2 concentrations are displayed with red lines. The RF algorithm also calculates the importance of each input attribute to its prediction. RF selects the raw reading of CO as its most important measurement (48% importance). The NO measurement is rated at 30%, whilst the raw NO2 reading’s importance is valued at 8%. These relations are summarized in Figure 5. Figure 4. Time series of NO 2 concentrations for two 7-day periods of the test set (2022). Uncorrected NO 2 low-cost sensor reading is displayed with blue lines, RF-corrected NO 2 values are displayed with black lines, and reference NO2concentrations are displayed with red lines. The RF algorithm also calculates the importance of each input attribute to its prediction. RF selects the raw reading of CO as its most important measurement (48% importance). The NO measurement is rated at 30%, whilst the raw NO 2 reading’s importance is valued at 8%. These relations are summarized in Figure 5. Atmosphere 2023, 14, x FOR PEER REVIEW 10 of 15 Figure 5. The importance of the ENSENSIA readings for correcting NO2 using the RF algorithm. The importance is suggested by RF based on the training set (2021). From this observation, we conclude that the sensors complement each other. For this reason, the correction of their measurements can be more accurate when the measurements from the desired gas and the measurements of the other sensors are taken into account. 3.2. Ozone During the 2021 campaign, the observed mean and standard deviation of O3 concentration was 34 ± 11.8 ppb. The minimum and maximum observed values were 1 and 97 ppb, respectively. The ML algorithms were trained under a 5-fold cross-validation procedure in the 2021 campaign. The trained models were evaluated on the unseen data of the 2022 campaign. During this period, the observed mean and standard deviation of O3 concentration was 31 ± 9.6 ppb. The minimum and maximum observed values were 1 and 96 ppb, respectively. 3.2.1. Evaluation of Uncorrected Sensor Readings Considering the uncorrected measurements of ENSENSIA, the ME for the two periods were 13.9 and 13 ppb, respectively. Table 5 summarizes the results. Table 5. Evaluation metrics for the uncorrected ENSENSIA measurements of ozone against the reference. The metrics were calculated based on hourly-averaged values. R R2 ME (ppb) RMSE (ppb) MB (ppb) nME Train (2021 Campaign) 0.62 0.39 13.9 16.6 −7.4 0.57 Test (2022 Campaign) 0.72 0.52 13.0 15.5 −5.9 0.55 R2 was better during the 2022 Campaign (0.52). There was a decrease in the nME from 57 to 55%. 3.2.2. Evaluation of Machine Learning Algorithms The ML algorithms were trained and validated on the 2021 Campaign under a 5-fold cross-validation procedure. The 2022 campaign was used to assess the algorithms’ efficiency in predicting unseen data. Figure 6 shows the uncorrected O3 and the RF-corrected O3 sensor readings versus the measured reference O3 concentration. Figure 7 presents time series of O3 concentrations for two 7-day periods. Figure 5. The importance of the ENSENSIA readings for correcting NO 2 using the RF algorithm. The importance is suggested by RF based on the training set (2021). From this observation, we conclude that the sensors complement each other. For this reason, the correction of their measurements can be more accurate when the measurements from the desired gas and the measurements of the other sensors are taken into account.