A Deep Learning Model of Radio Wave Propagation for Precision Agriculture and Sensor System in Greenhouses
Abstract
The production of crops in greenhouses will ensure the demand for food for the world’s population in the coming decades. Precision agriculture is an important tool for this purpose, supported among other things, by the technology of wireless sensor networks (WSN) in the monitoring of agronomic parameters. Therefore, prior planning of the deployment of WSN nodes is relevant because their coverage decreases when the radio waves are attenuated by the foliage of the plantation. In that sense, the method proposed in this study applies Deep Learning to develop an empirical model of radio wave attenuation when it crosses vegetation that includes height and distance between the transceivers of the WSN nodes. The model quality is expressed via the parameters cross-validation, R2 of 0.966, while its generalized error is 0.920 verifying the reliability of the empirical model.
Full text
Citation: Cama-Pinto, D.; Damas, M.; Holgado-Terriza, J.A.; Arrabal-Campos, F.M.; Martínez-Lao, J.A.; Cama-Pinto, A.; Manzano-Agugliaro, F. A Deep Learning Model of Radio Wave Propagation for Precision Agriculture and Sensor System in Greenhouses. Agronomy 2023,13, 244. https:// doi.org/10.3390/agronomy13010244 Academic Editor: Roberto Marani Received: 1 November 2022 Revised: 9 January 2023 Accepted: 10 January 2023 Published: 13 January 2023 Copyright: © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). agronomy Article A Deep Learning Model of Radio Wave Propagation for Precision Agriculture and Sensor System in Greenhouses Dora Cama-Pinto 1, Miguel Damas 1, Juan Antonio Holgado-Terriza 2, Francisco Manuel Arrabal-Campos 3,4 , Juan Antonio Martínez-Lao 3, Alejandro Cama-Pinto 5 and Francisco Manzano-Agugliaro 3,4,* 1Department of Computer Architecture and Technology, University of Granada, 18071 Granada, Spain 2Software Engineering Department, University of Granada, 18071 Granada, Spain 3Department Engineering, University of Almeria, Carretera Sacramento, s/n, La Cañada de San Urbano, 04120 Almería, Spain 4 CIAIMBITAL Research Center, CeiA3, University of Almería, Carretera Sacramento s/n, 04120 Almería, Spain 5Faculty of Engineering, Universidad de la Costa, Calle 58 # 55-66, Barranquilla 080002, Colombia *Correspondence: [email protected] Abstract: The production of crops in greenhouses will ensure the demand for food for the world’s population in the coming decades. Precision agriculture is an important tool for this purpose, supported among other things, by the technology of wireless sensor networks (WSN) in the monitoring of agronomic parameters. Therefore, prior planning of the deployment of WSN nodes is relevant because their coverage decreases when the radio waves are attenuated by the foliage of the plantation. In that sense, the method proposed in this study applies Deep Learning to develop an empirical model of radio wave attenuation when it crosses vegetation that includes height and distance between the transceivers of the WSN nodes. The model quality is expressed via the parameters cross-validation, R2of 0.966, while its generalized error is 0.920 verifying the reliability of the empirical model. Keywords: deep learning; neural network; precision agriculture; propagation model; wireless sensor networks 1. Introduction The increase in demand for crops and food production is associated with the growth of the world population, which according to data from the Food and Agriculture Organization (FAO) of the United Nations, is currently 7.7 billion humans, projected to be 9.4 billion in 2030 and 10.1 billion in 2050, when the world population will need 70% more food, 42% more arable land and 120% more water for food-related purposes [ 1 – 4 ]. Since traditional outdoor agriculture does not satisfy food production, coupled with the reduction of limited agricultural land for civil works construction, an optimal solution is protected crops called greenhouses that increase the number of harvests. Better yet, when transformed to smart greenhouses using information technology and sensors, can contribute to the increase of agricultural production [5]. In relation to the technological advances of Industry 4.0, cloud computing and the IoT (Internet of Things) contribute to making traditional systems smart [ 6 – 8 ]. An example of this process is smart farming (SF) that improves productivity and reduces surplus elements used in crops [9]. On the other hand, within the IoT concept, the role of wireless sensor networks (WSN) is paramount [ 10 , 11 ] because several IoT applications are based on wireless data transmission allowing sensor/actuator nodes to communicate with each other through a wireless network connection, even potentialized within the mMTC (massive machine-type communications) scenario of 5G [12–15]. Its sensors record variable data in crop fields and transfer it wirelessly to the base station for agricultural decision-making and monitoring [ 16 ]. Proper planning of the Agronomy 2023,13, 244. https://doi.org/10.3390/agronomy13010244 https://www.mdpi.com/journal/agronomy
Agronomy 2023,13, 244 2 of 16 arrangement of the number of wireless nodes within a greenhouse is a major challenge. Maximum coverage in wireless communication is a research objective to establish a model to determine the attenuation curves of the radio signal when deployed inside the greenhouse. Several empirical models, such as Weissbeerger’s or ITU-R’s model for radiowave attenuation, have significant error rates when compared to results obtained in greenhouse field tests because they ignore the antenna height variable in their equations [ 17 , 18 ]. Efforts have been made to improve the predictions through novel models that introduce variable antenna height because foliage in crops has a different density at different spans. Among these, we highlight some that employ linear and polynomial [ 19 – 21 ] regressions. However, the best prediction was performed by regularized non-linear regression in [22]. There are several reasons why deep learning models may be useful, even in cases where there is a small amount of data available. First, deep learning models are particularly well-suited for tasks that involve learning from complex high-dimensional data. These types of tasks can be challenging to model using traditional machine learning approaches, but deep learning models are able to learn useful features and patterns directly from the data. Second, deep learning models are able to learn hierarchical representations of the data, with different layers of the model learning to represent different levels of abstraction. This allows the model to learn complex relationships in the data and make more accurate predictions. Third, deep learning models are able to handle large amounts of noise and variability in the data, which can be especially useful in real-world applications where data is often messy and incomplete. This research aims to improve prediction by means of deep learning, a sub-field of machine learning, a branch of artificial intelligence, to find a new empirical model of attenuation and contrast it with the previous model (regularised regression) to determine whether it offers greater accuracy in its prediction. Until now, with respect to the literature reviewed, we found that this is the first time that, using deep learning, an empirical propagation model has been developed for application to any greenhouse plantation. 2. Background Based on the paradigms of Industry 4.0 (Fourth Industrial Revolution), the PA (Precision Agriculture, Third Agricultural Revolution) evolved into Agriculture 4.0 (A4.0) and is also called smart farming (SF) [ 23 ]. It integrates information and communication technologies (ICT) into traditional farming practices to monitor a wide range of agricultural parameters that improve crop yields [ 24 ]. Both terms (SF and A4.0) related to digital agriculture (DA) are driving change in revolution, sustainability, efficiency, productivity, and food security. This novel paradigm is based on technologies such as IoT, artificial intelligence, big data, cloud computing, and other related smart systems and devices for crop and farm management [25–27]. Within this technological scenario, the wireless sensor networks (WSN) provide a local crop monitoring system that enables appropriate decisions to be made in a controlled production system affected by climate change [ 28 , 29 ]. Through wireless data transmission, WSN supports the collection of information in agriculture due to their low cost, minimal power consumption, self-organizing capability, wide area coverage by multi-hop links, and deployment in environments changed by plant growth, with limited power grid [ 27 ], contributing to improved agricultural productivity in an environmentally sustainable way [ 30 , 31 ]. The types of sensors for agriculture are set according to the characteristics of each plantation [32,33]. The Received Signal Strength Indicator (RSSI) reveals power values in radio wave propagation. The environment, crop growth, and antenna heights determine RSSI values [ 34 , 35 ]. The models used to predict the RSSI between two transceivers are called propagation models [36]. The Friis model of free space propagation was used to obtain the line-of-sight (LOS) path loss incurred in a free space environment from a transmitter to a receiver, as a relation
Agronomy 2023,13, 244 3 of 16 between the received power to the transmitted power, in terms of effective areas of the receiving (Rx) and transmitting (Tx) antenna through free space [37–44]. In greenhouses, the effects of the vegetation impact in the radio-wave propagation, which occurs with NLOS (non-line of sight). Signals at microwave (1–30 GHz) [ 45 ] and millimeter (30–300 GHz) frequencies [ 14 ] experience scattering and absorption caused by randomly distributed vegetation leaves and branches [ 46 ]. The total path losses are formulated by combining the PL fs model losses with the PL veg vegetation losses predicted by the different vegetation models [19,47,48]. The second category, the empirical model of path loss, was chosen for the present study because of the simplicity with which its equations are formulated, notably those listed in [ 21 , 22 ] based on the EDM (exponential decay model). However, its estimates have a considerable margin of error compared to those taken in field tests prompting us to focus our work to improve them. Among empirical models, the authors developed an empirical multi-parametric equation model based on non-linear regularised regressions using experimental measurements of the RSSI signal obtained from field test measurements of four greenhouses. In that study, the evaluation of the model with 5th degree polynomials yielded 0.948 for R 2 , 0.946 in R 2adj (20-parameter solution), and 0.942 for R 2 , y 0.940 en R 2adj when the equation was reduced to 15 parameters by applying cross-validation [22]. The attenuation of the radio wave inside the greenhouse depends on the signal frequency, antenna height, and distance between antennas, exhibiting a non-linearity behavior. Therefore, an interesting approach can also be applied, taking advantage of machine learning (ML) [ 49 ] in order to find the relationship between these non-independent variables. ML builds a model automatically by deducing meaningful ideas (known as features) from the dataset, with feature extraction being the most critical step in a model generation [ 50 ]. Then the non-linear features of the input data establish interactions and relationships with the output predictor variables [ 51 ]. Analogously, humans use a model of the world as a simulator in our brain, which is obtained by learning from large amounts of data collected by our senses interacting with the surrounding environment [52]. ML collects input and output data to subsequently predict future values [ 53 – 55 ]. For the implementation of machine learning algorithms ANNs (artificial neural networks) [56–60] . Based on this architecture, ANNs can be classified into CNNs (convolutional neural networks) [61–63] and recurrent neural networks (RNNs) [62,64]. DL (Deep learning) is a form or subfield of ML [ 65 , 66 ]. ANNs are the core algorithms of DL. If the depth or number of layers of the ANN is greater than three, it will cease to be a simple ANN and become a DL algorithm [ 67 ], called a deep neural network (DLL), allowing it to successfully interpret more complex non-linear inputs [ 68 – 71 ]. As mentioned before, although there has been no research using ML in the estimation of radio propagation loss in the presence of vegetation, there are some works related to radio propagation, such as DNN-based, employing CNN for radio propagation loss estimation using spatial information, such as building occupancy maps for input data [ 72 ], path loss prediction in rural areas using 3.7 GHz band, combines different ML models, for the base learning stage uses ANN, DT (decision trees), SVR (support vector regression), kNN (k-nearest neighbors), GLM (generalized linear model) and a custom DNN with three hidden layers as meta-learner [ 73 ]. The paper by Bogdándy et al. [ 74 ] used the log of WiFi RSSI values as input data to determine the indoor positioning of nodes with an ANN. In addition, [ 75 ] used ML to obtain an ANN-based model that predicts radio propagation loss characteristics inside tunnels. 3. Materials and Methods 3.1. Source of Data All data were collected by Cama-Pinto et al. [ 21 , 22 ]. The experiment was performed in greenhouses located in Almería, southeastern Spain [ 76 – 79 ]. Vegetable and fruit production is exported mainly to the EU [ 80 – 87 ]. RSSI data are from trials in four greenhouse fields
Agronomy 2023,13, 244 4 of 16 during February 2020, each with areas of 10,000 m 2 in the Almeria localities of La Cañada, Retamar, El Alquian, Níjar, and greenhouse test data from La Cañada in 2018. The total number of data collected were 345. Each experiment was repeated 10 times in 2020 and 60 times in 2018. The data used was the average of the experiments. The outline of the measurement system hardware configuration is detailed by the authors in [88]. As shown in Figure 1, during the measurement phase, the antennas of the T x node and the sink node (R x ) were placed at the same height. The signal arrived at the receiver attenuated after passing through the tomato plant walls (1 m thick) every 5 min, repeating the process 10 times: (1) For the measurement, both the T x node and the sink are located at equal distances from the ground. Every 5 min the R x node records the signal from the T x node, which arrives attenuated. The measurement is repeated 10 times, then the distance between the nodes is increased by adding one more tomato wall and doubling the previous procedure. After the separation increases by adding more tomato plant walls, there comes a point where there is no communication, ending this stage. (2) The T x and R x nodes are moved two meters next to the tomato wall into the side corridor and step 1 is repeated. (3) Steps 1 and 2 are followed with different heights (the heights in centimeters are 30, 50, 70, 90, 100, 150, and 200). Agronomy2023,13,xFORPEERREVIEW4of17 3.MaterialsandMethods 3.1.SourceofData AlldatawerecollectedbyCama‐Pintoetal.[21,22].Theexperimentwasperformed ingreenhouseslocatedinAlmería,southeasternSpain[76–79].Vegetableandfruitpro‐ ductionisexportedmainlytotheEU[80–87].RSSIdataarefromtrialsinfourgreenhouse fieldsduringFebruary2020,eachwithareasof10,000m 2 intheAlmerialocalitiesofLa Cañada,Retamar,ElAlquian,Níjar,andgreenhousetestdatafromLaCañadain2018. Thetotalnumberofdatacollectedwere345.Eachexperimentwasrepeated10timesin 2020and60timesin2018.Thedatausedwastheaverageoftheexperiments.Theoutline ofthemeasurementsystemhardwareconfigurationisdetailedbytheauthorsin[88]. AsshowninFigure1,duringthemeasurementphase,theantennasoftheT x node andthesinknode(Rx)wereplacedatthesameheight.Thesignalarrivedatthereceiver attenuatedafterpassingthroughthetomatoplantwalls(1mthick)every5min,repeating theprocess10times: (1) Forthemeasurement,boththeT x nodeandthesinkarelocatedatequaldistances fromtheground.Every5mintheRxnoderecordsthesignalfromtheTxnode,which arrivesattenuated.Themeasurementisrepeated10times,thenthedistancebetween thenodesisincreasedbyaddingonemoretomatowallanddoublingtheprevious procedure.Aftertheseparationincreasesbyaddingmoretomatoplantwalls,there comesapointwherethereisnocommunication,endingthisstage. (2) TheT x andR x nodesaremovedtwometersnexttothetomatowallintothesidecor‐ ridorandstep1isrepeated. (3) Steps1and2arefollowedwithdifferentheights(theheightsincentimetersare30, 50,70,90,100,150,and200). Figure1.Locationofnodesinsidethegreenhouseduringfieldtests. TheschematicofthetopviewofthedeploymentoftheTxandRxnodesinsidethe greenhouseisshowninFigure2. Figure 1. Location of nodes inside the greenhouse during field tests. The schematic of the top view of the deployment of the T x and R x nodes inside the greenhouse is shown in Figure 2. 3.2. Evaluation of the Model’s Performance The predictive performance of the model was assessed using seven (07) criteria. The mean square error (MSE), root mean square error (RMSE), mean absolute percentage error (MAPE), coefficient of determination (R 2 ), adjusted coefficient of determination (R 2adj ), Akaike information criterion (AIC), and the Bayesian information criterion BIC, also called the Schwarz information criterion—SBC [89–98]. The accuracy or assessment of model performance can be verified by the R 2 , its variant, the R 2adj and Q 2 [ 99 – 105 ]. On the other hand, AIC and SBC are widely used for model selection [106–117].
Agronomy 2023,13, 244 5 of 16 Agronomy2023,13,xFORPEERREVIEW5of17 Figure2.TopviewofthedeploymentoftheTxandRxnodesinsidethegreenhouse. 3.2.EvaluationoftheModel’sPerformance Thepredictiveperformanceofthemodelwasassessedusingseven(07)criteria.The meansquareerror(MSE),rootmeansquareerror(RMSE),meanabsolutepercentageerror (MAPE),coefficientofdetermination(R 2 ),adjustedcoefficientofdetermination(R 2 adj), Akaikeinformationcriterion(AIC),andtheBayesianinformationcriterionBIC,also calledtheSchwarzinformationcriterion—SBC[89–98]. TheaccuracyorassessmentofmodelperformancecanbeverifiedbytheR 2 ,itsvari‐ ant,theR 2adj andQ 2 [99–105].Ontheotherhand,AICandSBCarewidelyusedformodel selection[106–117]. 4.ADeepLearningModelofRadioWavePropagation Anoveldeeplearningmodelisproposedinthisworkbasedonbinaryfeedforward neuralnetwork.Itiscomposedoftwolayers,anencodingandadecodinglayer.Theen‐ codinglayerconvertsthedistanceandtheheightatwhichtheattenuationistobeknown intobinary.Sincetherangeofdataislimited.Thenumberofbitstodeterminetheinteger partandthedecimalpartwillbesmall.Thedistancevariesfrom1to35m,andtheheight variesfrom30cmto200cm.Theencodingisdoneusing14bitsforthedistance7forthe integerpartandtheother7forthedecimalpart.Forheight,weused11bits,4ofthemfor theintegerpartandtherestforthedecimalpart,givingtworealnumberswithtwodeci‐ malsusing25bitsintotal.Thedecodingconvertsfrombinarytorealnumberwithan accuracyof3decimalplaces,using17bitstoperformthisconversiontotherealnumber, soitused7bitsfortheintegerpartandtheother10forthedecimalpart.Theneuralnet‐ workiscomposedof7layers.Thefirstandthelastaretheinputandoutputlayers,re‐ spectively.Therestofthelayersarehidden.Figure3belowshowsthestructureofthe deepneuralnetwork.Theactivationfunctionfortheperceptronsisthesigmoidfunction. Theinputlayerhas25perceptronscorrespondingtothe25inputbits,whiletheoutput layerhas17perceptronscorrespondingtothe17outputbits. Figure 2. Top view of the deployment of the Txand Rxnodes inside the greenhouse. 4. A Deep Learning Model of Radio Wave Propagation A novel deep learning model is proposed in this work based on binary feedforward neural network. It is composed of two layers, an encoding and a decoding layer. The encoding layer converts the distance and the height at which the attenuation is to be known into binary. Since the range of data is limited. The number of bits to determine the integer part and the decimal part will be small. The distance varies from 1 to 35 m, and the height varies from 30 cm to 200 cm. The encoding is done using 14 bits for the distance 7 for the integer part and the other 7 for the decimal part. For height, we used 11 bits, 4 of them for the integer part and the rest for the decimal part, giving two real numbers with two decimals using 25 bits in total. The decoding converts from binary to real number with an accuracy of 3 decimal places, using 17 bits to perform this conversion to the real number, so it used 7 bits for the integer part and the other 10 for the decimal part. The neural network is composed of 7 layers. The first and the last are the input and output layers, respectively. The rest of the layers are hidden. Figure 3below shows the structure of the deep neural network. The activation function for the perceptrons is the sigmoid function. The input layer has 25 perceptrons corresponding to the 25 input bits, while the output layer has 17 perceptrons corresponding to the 17 output bits. Agronomy 2023, 13, x FOR PEER REVIEW 6 of 17 Figure 3. Schematic of the Deep Learning Model for the estimation of attenuation from distance and height. The distance and height values compose the vector X while the estimated attenuation values 𝑳𝑳𝒇𝒇𝒇𝒇𝒇𝒇𝒇𝒇𝒇𝒇𝒇𝒇𝒇𝒇(𝒅𝒅,𝒉𝒉) or measured dB are Y. The representation of the two vectors X and Y is shown in Figure 4 below, Figure 4. Representation of distance and height, which is composed the vector X, while the estimated attenuation values 𝐿𝐿𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓(𝑑𝑑,ℎ) or measured dB are composed of Y. The B values are bits that can take the values either 0 or 1. Then, for creating the real number, the distance, height, and Lfoliage(d,h) have a integer part and decimal part. The parametric adjustment of the deep neural network is performed by minimizing the following cost function is, 𝐽𝐽(𝜃𝜃)=1 𝑚𝑚���−𝑦𝑦 𝑓𝑓 log �ℎ 𝜃𝜃 (𝑥𝑥 𝑓𝑓 )�−(1−𝑦𝑦 𝑓𝑓 )log �1−ℎ 𝜃𝜃 (𝑥𝑥 𝑓𝑓 )�� 𝐾𝐾 𝑘𝑘=1 𝑚𝑚 𝑓𝑓=1 (1 ) where m is the number of experiment performed in the greenhouse, where for a distance and height given, we obtain a signal attenuation, and K is the total number of bits in the output layer. The logistic function is defined as, ℎ𝜃𝜃=𝑔𝑔(𝜃𝜃𝑇𝑇𝑥𝑥) (2 ) where g is the sigmoid function, 𝑔𝑔(𝑧𝑧)=1 1 + 𝑒𝑒−𝑧𝑧 (3 ) To avoid deviations and overfitting of the cost function parameters of Equation (1), the regularization function called Tikhonov regularization [118,119] is added as follows, Figure 3. Schematic of the Deep Learning Model for the estimation of attenuation from distance and height.
Agronomy 2023,13, 244 6 of 16 The distance and height values compose the vector X while the estimated attenuation values Lf oliage(d,h) or measured dB are Y. The representation of the two vectors X and Y is shown in Figure 4below, Agronomy2023,13,xFORPEERREVIEW6of17 Figure3.SchematicoftheDeepLearningModelfortheestimationofattenuationfromdistanceand height. ThedistanceandheightvaluescomposethevectorXwhiletheestimatedattenuation values𝑳𝒇𝒐𝒍𝒊𝒂𝒈𝒆𝒅,𝒉ormeasureddBareY.TherepresentationofthetwovectorsXand YisshowninFigure4below, Figure4.Representationofdistanceandheight,whichiscomposedthevectorX,whilethe estimatedattenuationvalues𝐿 𝑑,ℎormeasureddBarecomposedofY.TheBvaluesare bitsthatcantakethevalueseither0or1.Then,forcreatingtherealnumber,thedistance,height, andL foliage (d,h)haveaintegerpartanddecimalpart. Theparametricadjustmentofthedeepneuralnetworkisperformedbyminimizing thefollowingcostfunctionis, 𝐽𝜃1 𝑚𝑦logℎ𝑥1𝑦log1ℎ𝑥 (1) wheremisthenumberofexperimentperformedinthegreenhouse,whereforadistance andheightgiven,weobtainasignalattenuation,andKisthetotalnumberofbitsinthe outputlayer.Thelogisticfunctionisdefinedas, ℎ𝑔𝜃𝑥(2) wheregisthesigmoidfunction, 𝑔𝑧1 1𝑒 (3) Figure 4. Representation of distance and height, which is composed the vector X, while the estimated attenuation values Lf oliage(d,h) or measured dB are composed of Y. The B values are bits that can take the values either 0 or 1. Then, for creating the real number, the distance, height, and L foliage (d,h) have a integer part and decimal part. The parametric adjustment of the deep neural network is performed by minimizing the following cost function is, J(θ) = 1 m m ∑ i=1 K ∑ k=1h−yiloghθ(xi)−1−yilog1−hθ(xi)i (1) where mis the number of experiment performed in the greenhouse, where for a distance and height given, we obtain a signal attenuation, and Kis the total number of bits in the output layer. The logistic function is defined as, hθ=gθTx(2) where gis the sigmoid function, g(z) = 1 1+e−z(3) To avoid deviations and overfitting of the cost function parameters of Equation (1), the regularization function called Tikhonov regularization [118,119] is added as follows, J(θ) = 1 m m ∑ i=1 K ∑ k=1h−yiloghθ(xi)−1−yilog1−hθ(xi)i+λ 2m"N−1 ∑ n=1 Jn ∑ j=1 Sn ∑ s=1θ(n) j,s2#(4) The network parameters are represented by θ(i) j,k , where Nis the number of layers, Jn is the number of total incoming connections at the n-th layer, and Sn is the number of total incoming connections at the n-th layer. λ is the regularising term and establishes the weight that the parameters should have in the cost function, avoiding overfitting and variability in the parameterized functions. In this optimization problem, it is mandatory to determine the gradients in each direction. The gradients can be calculated using the backpropagation algorithm (see Algorithm 1). Once the gradients have been calculated, the Polac–Ribiere method [ 120 ] is used to calculate the conjugate gradients to estimate the search direction. The approximation is performed using quadratic polynomial functions. The stopping criterion used is the so-called Wolfe-Powel conditions [121,122].
Agronomy 2023,13, 244 7 of 16 Algorithm 1 Backtracking Algorithm Applied in the Deep Learning 1Training set nx(1), y(1),x(2), y(2), . . . , x(m), y(m)o 2 For the entire training package 3It establishes ∆(n) ij =0 4 Compute forward propagation 5 Compute regularized cost function J(θ) 6Set a(1)=x(i) 7Perform forward propagation to compute a(n)for n=2, 3, . . . , N 8Using y(i), compute δ(N)=a(N)−y(i) 9Compute δ(N−1),δ(N−2),δ(N−3), . . . , δ(2) 10 ∆(n) ij :=∆(n) ij +an jδ(n+1) i 11 D(l) ij := 1 m∆(n) ij +λθ(n) ij if j6=0 12 D(l) ij := 1 m∆(n) ij if j=0 13 ∂ ∂θ(n) ij J(θ) = D(n) ij The total number of parameters conditions both the training time and the density of perceptrons in the neural network. A study is made of the number of parameters for a given value of λ . Figure 5below relates the error in the prediction of the attenuation value to the number of perceptrons in the network. Agronomy2023,13,xFORPEERREVIEW8of17 Figure5.RMSEcross‐validationvaluesforsettingthenumberofperceptrons.Itissetupto15,575 distributedin7layers. Theoptimalnumberofperceptronsoftheneuralnetworkis15,575.Followingthis architecture,thevalueofλisoptimizedbychoosingvaluesof0.1,0.01,0.001,and0.0001, resultinginthefollowing(Figure6). Figure6.RMSEcross‐validation.Theoptimalvalueof λ issetnearto10 −3 . Theresultsuggeststhebestvalueofλ.Therootmeansumsquareerror(RMSE)re‐ mainsconstantwhenλisnearto0.001.Then,theoptimalvalueissetupto0.001.The valueofλisusedinthebackpropagationalgorithmtoavoidbiasandoverfitting[118,119]. Figure7showsthelossfunctionversusthenumberofepoch.Itisnecessaryfor10,000 epochtoobtainalosscostvalueequala0.0332. Figure 5. RMSE cross-validation values for setting the number of perceptrons. It is set up to 15,575 distributed in 7 layers. The optimal number of perceptrons of the neural network is 15,575. Following this architecture, the value of λ is optimized by choosing values of 0.1, 0.01, 0.001, and 0.0001, resulting in the following (Figure 6). The result suggests the best value of λ . The root mean sum square error (RMSE) remains constant when λ is near to 0.001. Then, the optimal value is set up to 0.001. The value of λ is used in the backpropagation algorithm to avoid bias and overfitting [ 118 , 119 ]. Figure 7shows the loss function versus the number of epoch. It is necessary for 10,000 epoch to obtain a loss cost value equal a 0.0332.
Agronomy 2023,13, 244 8 of 16 Agronomy2023,13,xFORPEERREVIEW8of17 Figure5.RMSEcross‐validationvaluesforsettingthenumberofperceptrons.Itissetupto15,575 distributedin7layers. Theoptimalnumberofperceptronsoftheneuralnetworkis15,575.Followingthis architecture,thevalueofλisoptimizedbychoosingvaluesof0.1,0.01,0.001,and0.0001, resultinginthefollowing(Figure6). Figure6.RMSEcross‐validation.Theoptimalvalueof λ issetnearto10 −3 . Theresultsuggeststhebestvalueofλ.Therootmeansumsquareerror(RMSE)re‐ mainsconstantwhenλisnearto0.001.Then,theoptimalvalueissetupto0.001.The valueofλisusedinthebackpropagationalgorithmtoavoidbiasandoverfitting[118,119]. Figure7showsthelossfunctionversusthenumberofepoch.Itisnecessaryfor10,000 epochtoobtainalosscostvalueequala0.0332. Figure 6. RMSE cross-validation. The optimal value of λis set near to 10−3. Agronomy2023,13,xFORPEERREVIEW9of17 Figure7.Lossfunctionversusthenumberofepoch.Theoptimalvalueofepochis10,000givinga losscostvalue0.0332. 5.Results Figure8showsthesolutionobtainedfortheproposeddeeplearningmodel.Thisis the3Dviewoftheneuralnetwork,ascanbeseeninFigure3(Figure8a),wherethevalues takeninthegreenhouseappearasbluedots.Thex‐axisandy‐axisaredistance(d)and height(h),respectively,inmeters.Thez‐axisisvalueswhenevaluatingthedeepneural network𝐿 𝑑,ℎforthedistanceandheightdata.Figure8bshowstheresidual valuesbetweenmeasureddataandthosecalculatedwiththedeepneuralnetwork. Figure 7. Loss function versus the number of epoch. The optimal value of epoch is 10,000 giving a loss cost value 0.0332. 5. Results Figure 8shows the solution obtained for the proposed deep learning model. This is the 3D view of the neural network, as can be seen in Figure 3(Figure 8a), where the values taken in the greenhouse appear as blue dots. The x-axis and y-axis are distance (d) and height (h), respectively, in meters. The z-axis is values when evaluating the deep neural network Lf oliage (d,h) for the distance and height data. Figure 8b shows the residual values between measured data and those calculated with the deep neural network.
Agronomy 2023,13, 244 9 of 16 Agronomy2023,13,xFORPEERREVIEW10of17 (a) (b) Figure8.(a)Theblackdotsrepresentthevaluestakeninfieldtests,togetherwiththe𝐿 𝑑,ℎ optimizedforthedeeplearningmodel.(b)Differencebetweenmeasuredandpredicteddataarethe Residualvaluesof𝐿 𝑑,ℎ. Thecross‐validationofparametersrevealsthequalityofthenewmodel.R2andQ2, thesevalueswere0.966and0.957.TheRMSECVwas1.98.Thedeepneuralnetworkwas alsovalidatedbypermutationtesting. Thevaluesfortheevaluationofthemulti‐parametricoptimisedfunctionarepre‐ sentedinTable1.The0.966valuewastheadjustedR2. Table1.StatisticalQualityAssessmentoftheproposeddeeplearningmodel. R2R2AdjMSERMSEMAPEAICSBC 𝑳𝒇𝒐𝒍𝒊𝒂𝒈𝒆𝒅,𝒉0.9660.9643.391.980.113432221 Figure 8. ( a ) The black dots represent the values taken in field tests, together with the Lf oliage (d,h) optimized for the deep learning model. ( b ) Difference between measured and predicted data are the Residual values of Lf oliage (d,h). The cross-validation of parameters reveals the quality of the new model. R 2 and Q 2 , these values were 0.966 and 0.957. The RMSECV was 1.98. The deep neural network was also validated by permutation testing. The values for the evaluation of the multi-parametric optimised function are presented in Table 1. The 0.966 value was the adjusted R2. Table 1. Statistical Quality Assessment of the proposed deep learning model. R2R2Adj MSE RMSE MAPE AIC SBC Lfoliage(d,h)0.966 0.964 3.39 1.98 0.113 432 221
Agronomy 2023,13, 244 16 of 16 104. Oussama, C.; Abdellah, E.A.; Youssef, E.O.; Mohammed, B.; Abdelkrim, O. In silico Prediction of Novel SARS-CoV 3CL pro Inhibitors: A Combination of 3D-QSAR, Molecular Docking, ADMET Prediction, and Molecular Dynamics Simulation. Biointerface Res. Appl. Chem. 2022,12, 5100–5115. [CrossRef] 105. Gramatica, P. On the Development and Validation of QSAR Models. Methods Mol. Biol. 2013 ,930, 499–526. [CrossRef] [PubMed] 106. Gohain, P.B.; Jansson, M. Scale-Invariant and consistent Bayesian information criterion for order selection in linear regression models. Signal Process. 2022,196, 108499. [CrossRef] 107. Banteng, L.; Yang, H.; Chen, Q.; Wang, Z. Research on the subtractive clustering algorithm for mobile ad hoc network based on the Akaike information criterion. Int. J. Distrib. Sens. Netw. 2019,15, 1550147719877612. [CrossRef] 108. Banks, H.T.; Joyner, M.L. Adaption of Akaike information criterion under least squares frameworks for comparison of stochastic models. Q. Appl. Math. 2019,77, 831–859. [CrossRef] 109. Tozzi, R.; Masci, F.; Pezzopane, M. A stress test to evaluate the usefulness of Akaike information criterion in short-term earthquake prediction. Sci. Rep. 2020,10, 21153. [CrossRef] 110. Benedet, A.L.; Brum, W.S.; Hansson, O.; Karikari, T.K.; Zimmer, E.R.; Zetterberg, H.; Blennow, K.; Ashton, N.J. Alzheimer’s Disease Neuroimaging Initiative the accuracy and robustness of plasma biomarker models for amyloid PET positivity. Alzheimer’s Res. Ther. 2022,14, 26. [CrossRef] 111. Bhattacharya, S.; Biswas, A.; Nandi, S.; Patra, S.K. Exhaustive model selection in b → s `` decays: Pitting cross-validation against the Akaike information criterion. Phys. Rev. D 2020,101, 055025. [CrossRef] 112. Pham, H. A New Criterion for Model Selection. Mathematics 2019,7, 1215. [CrossRef] 113. Tran, D.A.; Tsujimura, M.; Ha, N.T.; Nguyen, V.T.; van Binh, D.; Dang, T.D.; Doan, Q.-V.; Bui, D.T.; Ngoc, T.A.; Phu, L.V.; et al. Evaluating the predictive power of different machine learning algorithms for groundwater salinity prediction of multi-layer coastal aquifers in the Mekong Delta, Vietnam. Ecol. Indic. 2021,127, 107790. [CrossRef] 114. Saez, L.M.A.; Campos, F.M.A.; Callejón-Ferre, J.; Medina, M.D.S.; Fernández, I. Use of multivariate NMR analysis in the content prediction of hemicellulose, cellulose and lignin in greenhouse crop residues. Phytochemistry 2019,158, 110–119. [CrossRef] 115. Ezuma, M.; Anjinappa, C.K.; Funderburk, M.; Guvenc, I. Radar Cross Section Based Statistical Recognition of UAVs at Microwave Frequencies. IEEE Trans. Aerosp. Electron. Syst. 2022,58, 27–46. [CrossRef] 116. Ikbal, N.A.M.; Halim, S.A.; Ali, N. Estimating Weibull Parameters Using Maximum Likelihood Estimation and Ordinary Least Squares: Simulation Study and Application on Meteorological Data. Math. Stat. 2022,10, 269–292. [CrossRef] 117. Xu, Q.; Li, R.; Liu, Y.; Luo, C.; Xu, A.; Xue, F.; Xu, Q.; Li, X. Forecasting the Incidence of Mumps in Zibo City Based on a SARIMA Model. Int. J. Environ. Res. Public Health 2017,14, 925. [CrossRef] 118. Tikhonov, A.N. On the solution of ill-posed problems and the method of regularization. In Doklady Akademii Nauk; Russian Academy of Sciences: Moscow, Russia, 1963; pp. 501–504. [CrossRef] 119. Calvetti, D.; Morigi, S.; Reichel, L.; Sgallari, F. Tikhonov regularization and the L-curve for large discrete ill-posed problems. J. Comput. Appl. Math. 2000,123, 423–446. [CrossRef] 120. Rasmussen, C.E.; Williams, C.K.I. Gaussian Processes for Machine Learning; the MIT Press: Cambridge, MA, USA, 2006; ISBN 026218253X. 121. Wolfe, P. Convergence Conditions for Ascent Methods. SIAM Rev. 1969,11, 226–235. [CrossRef] 122. Wolfe, P. Convergence Conditions for Ascent Methods. II: Some Corrections. SIAM Rev. 1971,13, 185–188. [CrossRef] Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.