Ornithopter Trajectory Optimization with Neural Networks and Random Forest

https://doi.org/10.1007/s10846-022-01612-5 REGULAR PAPER Ornithopter Trajectory Optimization with Neural Networks and Random Forest M. A. P´ erez-Cuti˜ no1,2 ·F. Rodr´ ıguez1·L. D. Pascual3·J. M. D´ ıaz-B´ a˜ nez1 Received: 1 October 2021 / Accepted: 5 March 2022 ©The Author(s) 2022 Abstract Trajectory optimization has recently been addressed to compute energy-efficient routes for ornithopter navigation, but its online application remains a challenge. To overcome the high computation time of traditional approaches, this paper proposes algorithms that recursively generate trajectories based on the output of neural networks and random forest. To this end, we create a large data set composed by energy-efficient trajectories obtained by running a competitive planner. To the best of our knowledge our proposed data set is the first one with a high number of pseudo-optimal paths for ornithopter trajectory optimization. We compare the performance of three methods to compute low-cost trajectories: two classification approaches to learn maneuvers and an alternative regression method that predicts new states. The algorithms are tested in several scenarios, including the landing case. The effectiveness and efficiency of the proposed algorithms are demonstrated through simulation, which show that the machine learning techniques can be used to compute the flight path of the ornithopter in real time, even under uncertainties such as wrong sensor readings or re-positioning of the target. Random Forest obtains the higher performance with more than 99% and 97% of accuracy in a landing and a mid-range scenario, respectively. Keywords Trajectory optimization ·Neural networks ·Random forest ·Ornithopter ·Dataset Multimedia Material The Ornithopter Trajectory Optimization (OTO) data set and evaluation code can be found at https://github.com/ mpcutino/OTO dataset. 1 Introduction An ornithopter is a flapping wing drone [5] and it is normally designed to imitate the flight of a bird, a bat or an insect. These prototypes seek to overcome the battery limitation and the human-safety problems of the M. A. P´ erez-Cuti˜ no [email protected];[email protected] Extended author information available on the last page of the article. multi-rotor drones. Over the past few decades, there have been remarkable advances in the design of flapping wing vehicles [23,31]. However, autonomous navigation remains a challenging task, among other reasons, due to lack of adequate motion planning algorithms. In robotics, motion planning is the problem of computing a path that connects an initial and a target robot configuration, also known as a trajectory. Motion planning for avoiding collisions has been the most common goal [12], but other objectives such as smooth paths or low battery consumption can be defined. Trajectory optimization algorithms aim to find the best trajectory for an object in a space when a set of kinematics and dynamics rules are considered [13]. An optimal trajectory is the one that minimizes (or maximizes) a given cost function and satisfies a set of constraints, usually described in a mathematical differential model. The complexity of computing an optimal trajectory is closely related to the number of maneuvers that the object can perform at each point of the trajectory (i.e. the number of paths that can be generated). Thus, planning optimal trajectories for autonomous ornithopters is a complicated problem. First, / Published online: 7 May 2022 Journal of Intelligent & Robotic Systems (2022) 105: 17 these types of vehicles present nonlinear and complex dynamics which must be taken into account when computing feasible trajectories. Second, the state space is large as it must include the position, velocities and altitude of the drone, which are relevant for gliding and perching operations. Due to these complexities, there is a need for efficient model-based methods that allow the computation of pseudooptimal trajectories with real-time performance. To address this gap in the research, in this paper we propose supervised Machine Learning (ML) algorithms to learn to fly an ornithopter. Due to the lack of data from real bird flights, our idea is to generate artificial bio-inspired flight data sets consisting of pseudo-optimal trajectories computed by a competitive planner. A recent approach for the problem of optimizing ornithopters trajectories is [27]. The authors proposed the planner OSPA (which stands for Ornithopter Segmentation-based Planning Approach), a trajectory optimization algorithm for a real ornithopter prototype. This algorithm can be applied to any dynamic model and different flight types. The authors demonstrated that OSPA outperforms alternative probabilistic kinodynamic planners both in cost (total energy) and accuracy (distance to the target). Unfortunately, OSPA is only valid for online application in short distances scenarios. Furthermore, in these scenarios OSPA only minimizes the positional distance, without taking into account the ornithopter velocity and pitch values at the target configuration. Here, we present new approaches to plan online energy-efficient trajectories for an ornithopter, both for long and short distances scenarios. The goal is improving the efficiency of the algorithms without degrading the quality of its solutions. Our method uses neural networks (NNs) and Random Forest (RF) to significantly improve the computational time of the OSPA planner. To train the supervised methods, we build a data set of energy-optimized trajectories based on the OSPA algorithm; see Fig. 1(a). We design ML algorithms that can learn an optimal policy from the data set, and predict the sequence of states of an energyefficient trajectory that considers transition between flapping and gliding, in order to save as much energy as possible, as illustrated with the example of Fig. 1(b)). In our previous work [25], we provide the first data set containing energy-efficient trajectories for ornithopter flights. In this paper, four main additional extensions are developed: 1) we increase the existing data set with a large number of trajectories, considering a higher number of maneuvers, as detailed in Section 4; 2) we incorporate an efficient Random Forest to accurately predict the next maneuver; 3) we add complex strategies to the networks architecture and the training process, such as residual connections [8], and a weighted loss function to consider class imbalance; 4) we study the performance of the proposed ML algorithms in more complex scenarios, by introducing random noise during flights and forcing replanning strategies. In general, our main contributions are the following: – We create a novel data set for ornithopter trajectory optimization termed as OTO. To the best of our knowledge, this is the first dataset with more than 100 000 pseudo-optimal trajectories for ornithopter motion planning, surpassing by a wide margin the previous existing one (containing less than 1 000 trajectories). The data set is benchmarked with several algorithms, and we provide a training-test split to ensure fair comparisons. – We design two classifiers: an artificial neural network (ANN) and a Random Forest. The algorithms learn maneuvers and can be queried to predict energyoptimized trajectories that can be computed online. These algorithms provide a strong benckmark for our novel data set. – We use an alternative recurrent neural network (RNN) to learn states instead of maneuvers, avoiding the need of knowing the model of the drone dynamics. – We show that the proposed ML-based algorithms can be used for real-time trajectory optimization, both for medium and short distances. Indeed, our methods significantly reduce computational costs while resulting in trajectories comparable to those produced by OSPA. The remainder of the paper is organized as follows: Section 2outlines related articles and Section 3states the problem description. Afterwards, the created data set is introduced in Section 4. The ML-based algorithms are presented in Section 5, while computational analysis and results take place in Section 6. Finally, conclusions are outlined in Section 7. 2 Related Work Trajectory optimization is a crucial task in autonomous navigation. Over the past decade, the optimization problem has been vastly studied, specially for multirotors and fixedwing vehicles; see [4] for a comprehensive survey. However, optimizing trajectories with real-time algorithms remains a challenge, particularly for systems with complex dynamics. Among the existing techniques for trajectory optimization [3], we are interested on developing efficient ML algorithms capable of overcome the existing limitations of traditional methods. In the existing literature, some types of path planners based on NNs have been proposed for Unmanned Aerial Vehicles (UAVs). Most of the methods utilize NNs to approximate the system dynamics, objective functions, 17 Page 2 of 16 J Intell Robot Syst (2022) 105: 17 Fig. 1 The proposed ML algorithms learn from experience by observing optimal trajectories starting at a given state (red dot) and targeting a point in space (red crosses), as in Figure (a). Data is obtained by implementing OSPA [27], which generates pseudo-optimal trajectories ending (blue dots) near the target states (red stars). Figure (b) is an example of a desired trajectory with three maneuvers from 15 meters before landing. The landing operation is achieved by a first maneuver with flapping, a second one performing gliding, and the last one using tail deflection and gradients, which removes the requirement for collocation [10,15]. Another promising approach is the use of deep NNs. For instance, in the paper [16], a Convolutional NN (CNN) model has been introduced to formulate path planning as a supervised classification problem. On the other hand, recurrent NNs (RNNs) has been proposed for solving 2-D maze navigation in [11], and algorithms combining RNNs with other learning techniques have been recently studied in [7,32]. Finally, other learning approaches, as Q-learning [34], cooperative learning [37], random forest [22] and reinforcement learning algorithms [38] have been proposed for UAV path planning. The aforementioned algorithms have been designed for the problem of path planning, which is usually the first step before obtaining an optimal trajectory. Our proposal is capable of generating an optimal trajectory without using an initial trajectory as reference. Trajectory optimization algorithms using neural networks have been proposed in [17,18]. In [18], an studentteacher strategy is used to couple an Artificial Neural Network and a trajectory optimization method based on contact-invariant optimization [19]. On the other hand, [17] trained a Multilayer Perceptron on trajectories obtained with the use of differential game theory. Both methods evidences the benefits of using NNs as part of the optimization process. Moreover, when properly controlling the parameters of these algorithms, the NNs can produce real-time responses. Despite the development of the aforementioned works, trajectory optimization using ML algorithms for complex systems, such as ornithopters, is an under researched area. In this paper, a dataset with a large number of pseudo-optimal trajectories is provided, and several ML-based methods are proposed to achieve ornithopter real-time path planning. The algorithms are tested in complex environments, addressing a trade-off between path quality and efficiency. 3 Problem Description Suppose that we have an autonomous ornithopter with a known model of its dynamics (see [27] for more details about the specific dynamic model assumed in this paper). The problem is to plan online energy-efficient trajectories to navigate the ornithopter from a starting point to a target location. Those trajectories have to comply with the ornithopter dynamics, therefore they are flyable. Then, we assume the existence of lower-level algorithms to control the ornithopter through the computed trajectory. A trajectory is a sequence of flight states, where a state describes the ornithopter configuration at a given instant of time. A flight state is given by a tuple s=(x,z,u,w,,q), where xand zare the positional values, uand ware velocity components in the body reference frame, is the pitch angle and qis the pitch angular velocity. Our trajectories are constrained to the XZ plane, as we assume a longitudinal motion model for the ornithopter. The ornithopter reaches a state (dynamically feasible) performing a specific control maneuver. A control maneuver is a tuple of values m=(δ, f ),whereδis the tail deflection, determined by the deflection angle (up and down), and f is the wing flapping, determined by the flapping frequency (including zero value for gliding). Thus an energy-efficient trajectory can be described by combining flapping, gliding and tail flick. An example of an ornithopter trajectory is illustrated in Fig. 1(b). In this paper, we establish a maneuver set Mcomposed by all combinations of flapping frequencies Page 3 of 16 17J Intell Robot Syst (2022) 105: 17 Table 1 The considered maneuvers with their corresponding labels class maneuver class maneuver 0(−4◦,0H/z) 13 (−2◦,3H/z) 1(−4◦,1H/z) 14 (−2◦,4H/z) 2(−4◦,2H/z) 15 (−1◦,0H/z) 3(−4◦,3H/z) 16 (−1◦,1H/z) 4(−4◦,4H/z) 17 (−1◦,2H/z) 5(−3◦,0H/z) 18 (−1◦,3H/z) 6(−3◦,1H/z) 19 (−1◦,4H/z) 7(−3◦,2H/z) 20 (0◦,0H/z) 8(−3◦,3H/z) 21 (0◦,1H/z) 9(−3◦,4H/z) 22 (0◦,2H/z) 10 (−2◦,0H/z) 23 (0◦,3H/z) 11 (−2◦,1H/z) 24 (0◦,4H/z) 12 (−2◦,2H/z) F={0Hz,1Hz,2Hz,3Hz,4Hz}and tail deflection angles T={−4◦,−3◦,−2◦,−1◦,0◦},seeTable1. We assume an open space where the drone flies without obstacles, and we consider two scenarios: the landing problem, where the X-distance is within a range of 15 to 25 meters, and the mid-range problem, where the X-distance between starting and target points is within a range of 25 to 100 meters. The optimization function is the total energy cost (battery) consumed by the ornithopter. Two main metrics will be used in the experiments: precision and cost. The precision measures how far the final state is from the target in the XZ-plane. The cost is given by the energy consumption determined by maneuvers performed by the ornithopter. The energy consumed by the ornithopter performing a certain maneuver for a time step tsis given by the following formula: E=ts(Kaerof3+cr).(1) The first term represents the energy consumption in the flapping wings maneuver. It has been empirically proven that this consumption is proportional to the cube of the flapping frequency [20]. The constant coefficient Kaero is estimated empirically based on the physical characteristics of the ornithopter1. The second term models the residual energy consumption crwhen the ornithopter is not flapping, mainly due to the onboard electronics. As the cost of moving the tail is negligible compared to the average consumption of the electronics, we consider the cost cr constant2. As expected, Equation 1shows that the energy cost is dominated by the flapping maneuvers. 1All the experiments in this paper used a value Kaero =2.5 W/Hz3, obtained empirically for the ornithopter prototype used in [27]. 2The empirically estimated upper bound of cris 5W. There are some approaches to address this optimization problem but unfortunately, solving the highly non-linear system that describes the dynamics of the ornithopter is time consuming and the methods from the literature are no valid to be used online. Thus, the main goal in this paper is to use supervised learning algorithms to compute short-term trajectory optimization for an ornithopter. 4 OTO Data Set To our knowledge, the first data set available for ornithopter trajectory optimization was given by [25].Thedatasetwas created to train NN’s using the planner OSPA [27] with scenarios involving flapping, gliding and landing maneuvers. In this paper, we improve the data set provided in [25] by adding a great number of new flight trajectories. For the sake of reproducibility, we make our data set publicly available3. 4.1 Data Generation Our intention is to make a large and representative ornithopter flight data. To this end we create a set of initial states for the ornithopter describing many possible starting positions in terms of velocity and pitch. A simple translation is applied over the initial and target state to simulate that every trajectory starts at the origin of coordinates. Thus, we assume that the ornithopter always starts at (0,0)position in the XZ plane with null angular velocity. We pick 9 initial states by taking the combinations of pitch and speed values P={−30◦,0◦,30◦}and S={0.5 m/s, 1m/s, 1.5 m/s}, respectively. From each of these initial states, we mimic the technique proposed in [27] to expand a tree of states by applying each maneuver in the set Mconsecutively taking a timestep of 0.1sand 1sbetween maneuvers for the landing and midrange problems, respectively. We consider a sequence of maneuvers increasing by 1Hz in flapping frequency and 1◦ in tail deflection angle (see Table 1). In order to create a large number of trajectories while avoiding exponential grow, we expand the tree of states and prune some vertices using a clustering approach at each level. This technique consists in grouping nearby states in space and keeping only the ones with lower cost in each cluster for the next iteration. With a big enough number of clusters by iteration, we get wide trees of low-cost trajectories scattered in space. We randomly generate a large number of target states in the space within the spatial ranges shown in Table 2. Target states have fixed pitch and speed values of 30◦and 3https://github.com/mpcutino/OTO dataset/tree/master/data 17 Page 4 of 16 J Intell Robot Syst (2022) 105: 17 Table 2 Ranges of target state position used in the experiments. MR: mid-range, L: landing Problem X range (m) Z range (m) L[0,25][0,−25] MR [25,100][−50,50] 0m/s, respectively. This represents a suitable posture for the ornithopter to reach the target. For each target point and the expanded tree, we select all states within a ratio of 0.01 mand 1 mfrom the target for the landing and the mid-range problem, respectively. From the selected states, we take as optimal ending state the one with better pitch and speed compared to the target in the landing problem, and the one with lower energy consumption in the midrange problem. We use these metrics because precision is more important than energy in the landing problem. For each of the optimal ending states we get its trajectory from the corresponding tree and include it into the data sets. Following the above procedure we create over 27.026 different trajectories (a total of 345.676 states) for the landing problem, and 95.153 different trajectories (a total of 876.333 states) for the mid-range problem. From each trajectory, we store the state-control waypoints, the initial and target states, the timestep and the energy cost. Finally, we use the 80% of the data to train the models and the rest for testing. 4.2 Data Analysis The ML-based algorithms proposed in this study use information about flight states and maneuvers to predict efficient trajectories between initial and target states. In this section we made a brief study over the maneuvers and over all variables in flight states in the OTO data set. With this study we pretend to observe the difficulty of our problem and to obtain some insights to improve the accuracy of predictions. Recall that the tree of states in the OSPA algorithm is expanded using the set Mof maneuvers in Table 1. Figure 2shows the percent of use of each maneuver in the landing and middle range data sets. We can observe that all maneuvers are used in the mid-range data set, but there are some of them with very low use rate. In this cases, the difficulty of predicting the correct maneuvers increases. On the other hand, for the landing scenario, the percentage value of each maneuver used decreases. As a result, the maneuver corresponding to class 24 is not used by OSPA during the landing flights. Figure 3illustrates the data distribution of the landing dataset, using the difference between the target and current state of the ornithopter. The analysis using these values is Fig. 2 (a) and (b) show the percent of use of the maneuvers in landing and mid-range problems, respectively important as it highlights the distribution of each variable in the data set. In addition, the difference of the states is the input of our algorithms. As each variable represent a different distribution, in the training phase we scale its values to have zero mean and unit standard deviation. On the other hand, the box-plots across the action classes highlights the complexity of our problem. For each individual variable, there are several outliers, which increase the variability in the data, but decreases the statistical power. We also performed a scaling operation on the training phase for the data corresponding to the mid-range problem. The distribution of the variables and the box-plots of the mid-range data set, result in a similar picture to the one generated for the landing scenario. 5 Machine Learning Algorithms The OSPA planner performance depends on the maneuvers it takes at a given state. Our proposal relies on trajectories generated by OSPA, using different supervised ML algorithms. The goal is to obtain energy-efficient trajectories and reduce computational times compared to exploratory algorithms. Thus, computational cost is a key point in the design of the architectures of our models. We propose two types of Artificial Neural Networks: a Multi-Layer Perceptron (MLP), to predict actions; and Page 5 of 16 17J Intell Robot Syst (2022) 105: 17 Fig. 3 Distribution of the landing problem dataset. The variables (u, v, omega, theta, x,z) represents the difference between the target and initial state. The distributions are depicted in the first and third plot with a Kernel Density Estimation (KDE) function. The box-plots in the second and fourth column shown the values of the variables across the different action codes, from 0 to 23 a Recurrent Neural Network (RNN), to predict states. For the MLP, we consider a discrete set of actions or maneuvers in a continuous space of configurations. This derives in two different kinds of problems, classification and regression, respectively. The advantage of the maneuver classification network is that it can generate feasible paths for the ornithopter according to the predefined dynamic model and, it is possible to compute the cost in terms of energy. However, this network only consider a discrete set of available maneuvers. Therefore, we propose a second type of network to directly predict a set of future states given the past states in the flight. In this case we tackle the continuous problem, even if the training data is generated in a discrete manner. For the classification problem, we also propose the use of Random Forest (RF) [2], as an alternative to the MLP. Random Forest has proved to outperform classifiers based on neural networks in terms of recognition accuracy, stability, and robustness to features in several scenarios [6,21,36]. Moreover, RF produces fast responses when properly controlling the number of trees in the forest. 5.1 Multi-Layer Perceptron MLP is a feed-forward artificial neural network that is usually trained in a supervised manner and contains several hidden layers. For our problem, we model a trajectory from the data as a discrete set of state-maneuver pairs. Since the set of maneuvers is fixed, we train the network to learn the next control maneuver depending on the current and target states. The goal is to build a MLP to learn the best maneuver in each scenario. For this, we label the maneuvers with a class tag from C={0,1,···,N}where Nrepresents the maximum integer identifier of the proposed maneuvers, as in Table 1, depending on the evaluated problem. For the landing task, N=23, and N=24 for the mid-range flights. The input of the network is the difference between the current and target states, and the output is the maneuver to be performed. 17 Page 6 of 16 J Intell Robot Syst (2022) 105: 17 Fig. 4 Definition of a Basic Block for the MultiLayer Perceptron. It is composed by a Linear layer with input of size nand output of size m,a Normalization layer, a ReLU activation function, and a Dropout layer with probability p 5.1.1 MLP Architecture The architecture of the network is designed as the application of consecutive Basic Blocks. The definition of a Basic Block is depicted on Fig. 4. First, a Linear layer transform a vector of size ninto a vector of size m. The result is normalized applying the Layer Normalization technique described in [1]. We introduce non-linearities with the use of the ReLU activation function, and finally we add a Dropout layer to prevent complex co-adaptations between neurons [9]. The bells and whistles of the proposed MLP are depicted on Fig. 5. We use a Deep Neural Network with 8 modules, containing a total of 777.792 neurons. Residuals connections are applied between the second and the fourth Basic Block, and between the fourth and the sixth. This type of connection reduce the complexity of the learning task, making the network easier to optimize [8]. 5.1.2 Training The MLP was trained using the RMSprop optimizer with a loss function based on Cross-Entropy. The loss values are weighted to consider the different aprioriprobabilities of the classes adopted. The function is defined as: LCE(x) =− i wipi(x) log qi(x), (2) where pi(x) is the probability of sample xof belonging to the class i,qi(x) is the probability prediction of the MLP, and wiis a weight function to take into account the different probabilities of belonging to class i. The weight value per class is computed as: wi=(1−Pi)αi,(3) Fig. 5 Architecture of the proposed MLP. The arrows between nonconsecutive blocks represents residual connections. The parameter C in the output layer is the number of classes depending on the problem: C=24 for landing, and C=25 for mid-range flights where Piis the aprioriprobability of class iand it is estimated as the percentage of samples of class iin the training dataset, and αiis a parameter that highlights the class relevance. wiis useful to compensate the differences in the representation of each class and, at the same time, it allows to push the network prediction in a specific direction to reduce (increase) the false negatives (positives). 5.1.3 Path Generation Once trained, the MLP can predict the control maneuver m from a flight state s. Also, we implement the kinodynamic model introduced in [27], K(s,m), to compute the next state given an initial state and a control maneuver. This allows us to compute the full trajectory by using the next formula in a loop starting at the initial state: si+1=K(si,N(si)), where N(si)represents the action predicted by the network for the state si. The loop is stopped when si+1is further than sifrom the target state. 5.2 Recurrent Neural Network (RNN) The goal of the proposed MLP is to predict the next maneuver at each time step using the information of the current state. Note that this strategy is dependent from the ornithopter model. We propose a RNN to predict the next Page 7 of 16 17J Intell Robot Syst (2022) 105: 17 flight state based on previous states. Recurrent architectures have been widely adopted for solving forecasting problems, with the use of Gated Recurrent Units (GRU) [28,29] or Long Short-Term Memory (LSTM) neurons to handle sequential relationships [33]. In this case, we use a decoder architecture so only the initial distance is needed to develop a complete trajectory from it. As can be seen in the Fig. 6, a decoder has a single input, and produces a sequence starting from this input. For our specific case, the input is the initial distance, and the output is a set containing the next states in the trajectory. 5.2.1 RNN Architecture We build a simple RNN with one recurrent layer and one output layer with a linear activation function. The output layer has as many units as flight states considered for prediction (two states for the mid-range problem and six states for the landing problem). The input is the initial distance from the current to the target state, and the output is the set of distances between each element in the trajectory and the target state. The RNN is executed recursively to get the complete trajectory starting from the initial distance. All input variables are normalized following the same strategy as in the MLP scenario. This represents an efficient strategy to push the network to predict low values while keeping the ability to compute the next state. The next state can be obtained by simply subtracting the output of the network and the target state. The recurrent layer has 11 LSTM neurons. We select LSTM neurons to deal with the vanishing gradient problem encountered by traditional recurrent neurons. In addition, we decide to keep a small architecture to obtain a fast predictions rate. Fig. 6 Overview of the decoder RNN architecture 5.2.2 Training The network is trained with one trajectory at a time, by comparing its output with the expected values. The goal of the training is to obtain the same input sequence shifted by one step ahead. The optimization process is carried out by the Adam optimizer using the values recommended in [14]: a learning rate of 0.001, a β1value of 0.9, a β2value of 0.999, an epsilon value of 1e-8 and no weight decay. The learning is stopped after 100 epochs. The loss function used is the MSE (Mean Squared Error): LMSE =1 N N  i=1 d(yi,ˆyi)2, where Nis the number of considered flight states, yiis the icomponent of the OSPA state and ˆyiis the i-component of the RNN predicted state. In our specific case, minimizing the MSE implies that the likelihood function between the predicted states distribution and the OSPA states is maximized. 5.2.3 Path Computation At every time step t, the information to predict the next state consists on the previous prediction of the distance (yt−1) at time step t−1, and the recurrent layer hidden states (ht−1) at the same time step. The hidden states condenses all previous information regarding the trajectory; see Fig. 7 for a representation of the proposed RNN, unrolled until time step t−1. The algorithm ends when an output ytis close enough to zero, i.e. the target has been reached and the trajectory is over. For t=0, we use as input the difference between the target and the initial state. 5.3 Random Forest The combination of different models over the same classification problem is usually known as ensemble learning. Fig. 7 Decoder RNN neuron. The input neuron (red box) contains six units, one for each variable of the ornithopter state 17 Page 8 of 16 J Intell Robot Syst (2022) 105: 17 In the case of RF, the predictions are obtained by a voting process over the output of different Decision Tress [26]. To produce different distributions in the models, every tree selects randomly a subset of the features present in the data, and random samples from the dataset. To evaluate the importance of the attributes before creating a new node in the trees, we selected the Gini Index (GI) as it results in higher precision on the conducted experiments. GI is calculated by subtracting the sum of the squared probabilities of each class from one. GI =1− C  i=1 p2 i,(4) where Crepresent the number of classes and piis the probability of class i, obtained as the frequency of appearance on the considered split. The RF architecture for training is obtained by applying a Grid Search strategy. For each task, i.e. landing and midrange flights, we define a set of possible parameters, and we create different forest using all possible combinations. Each forest is evaluated on a small portion of the training data, i.e. a validation set, and the best configuration of parameters is maintained for full training. The specific values of the parameters are depicted on Section 6. In addition, the path generation follows the same guidelines detailed in the MLP section. 6 Experiments In this section we compare the proposed ML methods against the OSPA planner4. The metrics used are the average values of energy consumption (Cost), execution time (Time), the Euclidean distance between the last state and the target (Precision) and the mean distance between the calculated and OSPA trajectories (Error). The precision is calculated by using the xand zvalues of the states. The Error is defined as the average distance between a fixed number of points in trajectories generated by OSPA and the ML algorithms. The points are obtained by sampling 10 random states for each trajectory and computing the correspondent positional distance. This metric gives us an idea about the similarity between two given trajectories. Metrics are computed over the testing dataset. 4It is important to notice that our problem is very specific from an engineering point of view, and we need to integrate complex and non-linear dynamics. This makes it hard to find heuristics that works properly. This is why we decided to only compare with OSPA, which is particularly suited for the problem at hand. The classifiers are compared using the metrics Accuracy, average Recall, and Macro average Recall. In a binary setting, Recall and Accuracy are defined as: Recall =TP TP +FN,(5) Accuracy =TP +TN N,(6) where TP,FN,TN represent True Positive, False Negative and True Negatives samples, respectively. Nis the cardinality of the test set. As our problem is not binary, we compute the Accuracy as the fraction of correctly classified samples. On the other hand, Recall is computed per class, thus transforming our problem in Cbinary tasks. The average Recall is computed as the weighted average of the recall per class. The weight is defined by the number of samples of the class in the test set. In addition, the Macro average metric does not take into account the number of samples; therefore, it assumes equal importance on every class. This value is a representation of the behaviour on classes less populated on the dataset. To counteract the class imbalance, the αparameter selected in the loss function of the MLP is 5. In both scenarios, mid-range and landing, the 20% of the training dataset is used for validation. Validation data is used differently depending on the algorithms. For the neural networks, we keep the model with lower loss value on the validation data; for Random Forest, we perform a gird search to optimize the model parameters. The ML algorithms were tested on a Intel Core i5-8250U CPU, with a clock frequency of 1.60GHz. The NNs were optimized using the nvidia graphic card GeForce MX150. We use Pytorch [24] for the implementation of the MLP, and Keras for implementing the RNN. 6.1 Landing Problem Landing phase remains to be one of the most crucial and difficult tasks to achieve among the flight envelope of an aerial robot. Landing requires complex maneuvers in order to prepare the ornithopter to perform a perching operation. Moreover, there is a need for a fast computation because the ornithopter has limited time for reaction. Also, for the landing problem it is more relevant to end the trajectory closer to the target rather than reducing the energy cost. In the learning phase for this task, RNN and MLP are trained by a maximum of 100 and 300 epochs, respectively. The validation dataset is used to keep the model with the best loss value across all the epochs. In this manner, we ensure a better generalization when predicting over unseen data. For the RF algorithm, we perform a grid search over the validation data, resulting in a model that uses 100 trees, the Gini Index as the quality function, and a maximum depth Page 9 of 16 17J Intell Robot Syst (2022) 105: 17 37. Zhang, B., Liu, W., Mao, Z., Liu, J., Shen, L.: Cooperative and geometric learning algorithm (cgla) for path planning of uavs with limited information. Automatica 50(3), 809–820 (2014) 38. Zhang, B., Mao, Z., Liu, W., Liu, J.: Geometric reinforcement learning for path planning of uavs. Journal of Intelligent & Robotic Systems 77(2), 391–409 (2015) Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Miguel Angel P´ erez-Cuti˜ no is a PhD student in Mathematics at the University of Seville, Spain. He has received a bachelor’s degree in Computer Science from University of Havana in 2018, and a master’s degree in Logic, Computing and Artificial Intelligence from University of Seville in 2021. He is currently working at VirtualMech developing computer vision systems for fault detection. His research interests include computational geometry, computer vision and deep learning. Fabio Rodr´ıguez is in his second year of the Ph.D. program in Mathematics at the University of Seville. His work is focus on the areas of algorithms, computational theory, and mathematics and recently, data science and machine learning. He is passionate about the algorithms design and analysis and currently, he is applying computational techniques to the resolution of problems with drones in industry. He has participated in many international conferences related to algorithms and machine learning. Fabio’s research is supported by the DENiM European Grant Agreement (https://cordis.europa. eu/project/id/958339). He holds a Master degree in Mathematicsin University of Seville and a Bachelor of Computer Science from the University of Havana. See more info at https://www.linkedin.com/in/ fabio25-rodriguez. Luis David Pascual Callejo is an aeronautical engineer at Airbus, working currently as product owner of an Airbus developed web application. During his professional career he has gained experience in aircraft production, web development and project management. He has a double Master of Sc.in aeronautical engineering between the University of Madrid and Supareo (Toulouse), an MBA at College des Ingenieurs (Paris) and a master’s in mathematics at the University of Seville. His research interests are mainly focused on machine learning applied to aeronautics. See full bio at https://www.dpascual.space. Jos´ e-Miguel D´ıaz-B´ a˜ nez is a Full Professor of Applied Mathematics at the University of Seville, Spain. His research interests include computational geometry, decentralized algorithms for cooperative UAVs and computational ethnomusicology, within the common theme of geometric algorithms. He has coauthored 75 research papers in indexed journals by the Journal Citation Reports, 3 books, 15 book chapters and more than 80 conference papers. He has organized several international conferences related to geometric algorithms. Prof. D´ ıaz- B´ a˜ nez has supervised 8 PhD’s thesis and 12 Master’s thesis. He has led 15 research projects, most of them funded by the European Commission and Spanish Government. He is currently the coordinator of the Optidrone project, as well as other projects on geometric algorithms applied to path planning with drones. See more info at http://alojamientos.us.es/galgo/ and https://diazbanez.wordpress.com/. Affiliations M. A. P´ erez-Cuti˜ no1,2 ·F. Rodr´ ıguez1·L. D. Pascual3·J. M. D´ ıaz-B´ a˜ nez1 F. Rodr´ ıguez [email protected] L. D. Pascual [email protected] J. M. D´ ıaz-B´ a˜ nez [email protected] 1Department of Applied Mathematics II, University of Seville, Seville, Spain 2VirtualMech, Seville, Spain 3Airbus, Seville, Spain 17 Page 16 of 16 J Intell Robot Syst (2022) 105: 17