Statistical Evaluation of Pilot’s Behavior Models Parameters Connected to Military Flight Training

energies Article Statistical Evaluation of Pilot’s Behavior Models Parameters Connected to Military Flight Training Miroslav Jirgl 1, Jan Boril 2and Rudolf Jalovecky 2,* 1Department of Control and Instrumentation, Faculty of Electrical Engineering and Communication, Brno University of Technology, Brno 61600, Czech Republic; [email protected].cz 2Faculty of Military Technology, University of Defense, Brno 66210, Czech Republic; [email protected] *Correspondence: r[email protected] Received: 7 July 2020; Accepted: 26 August 2020; Published: 28 August 2020   Abstract: The paper discusses the possibilities of objective assessment of military flight training quality based on statistical evaluation of pilot’s behavior models parameters. For these purposes, the pilots’ responses to non-standard flight situations were measured by using a fixed-base and a moving-base engineering flight simulator. Tens of military pilots at different training stages were tested. By exploiting real-life tests, we established that the given pilot models provide sufficiently accurate approximation of realistic human responses. Importantly, the models are relatively easy to use, and the individual parameters can be unambiguously interpreted, i.e., the time constants of the pilot behavior model are obtainable, representing the pilot’s current psychological and physiological state of mind. The parameters lay in the defined ranges, and they characterize the ability of the human/pilot to adapt to a controlled dynamic system. Consequently, a fundamental statistical analysis based on pilot’s behavioral model parameters was conducted, using the acquired test data representing the pilot’s behavior during repeated measuring. The initial results indicate the possibility to use the results for objective assessment the military flight training level. Keywords: McRuer models; reaction delay; MATLAB; flight simulator; pilot’s behavior 1. Introduction Contemporary aviation places increasingly more emphasis on the mental stability of both the pilots and the supporting personnel. In addition to the traditional factors that complicate their work performance, including noise, vibration, overload, and changing air pressure, these professionals are required to operate new technologies whose complexity and control configuration features (despite the high degree of automation) often put whole crews under significant stress [ 1 ]. The problem is exacerbated by the fact that most of the skills utilized by pilots for fast and reliable operation of the systems deteriorate during flight downtimes. Piloting, in its pure form, together with the development of technologies, has receded into the background to be replaced by the need to operate navigation systems and autopilots. Airplanes are presently capable of performing autonomous take-offand landing; the pilot, however, still plays a crucial, irreplaceable role in the decision-making process [2]. Aviation generally needs to recruit specialists with qualities that contribute to flight safety; these comprise, above all, spatial orientation, movement coordination, motivation, resistance to stress, cooperative mind, responsibility, and the ability to lead. However, there is no uniform pattern of what qualities should a pilot have. The requirements placed on groups of pilots then form another specific domain, where communication, coordination, and resistance to monotony and stress are the know-how to be adopted by each transport pilot. In contrast, fighter pilots need to exhibit high perceptual speed, responsiveness, and resistance to g-force and time stress. By further extension, helicopter pilots should excel in coordinating their hand and leg movements and concentration capabilities. Energies 2020,13, 4452; doi:10.3390/en13174452 www.mdpi.com/journal/energies Energies 2020,13, 4452 2 of 13 In aviation, the human factor plays an important role, and a potential human failure could lead to poor flight performance, an accident, or even a disaster. The study of the relationships between diverse human factors and flight performance or activities related to flight safety thus embodies an important research domain where aircraft design, pilot selection, and operation environment constitute major sub-problems. The effect of such an investigation is defined by links within the SHELL (Software, Hardware, Environment, Liveware/human being to Liveware) conceptual model [ 3 ], in which the most important link is human–human. Historically, it was human skills that facilitated the birth and progressive development of aviation technologies; gradually, however, people have become the weakest element in the entire system. This is also why only selecting the best individuals does not suffice for the reliable performance of the pilot profession. Significant aspects consist in effective management and use of the training time, with adequate practice needed to refine the potential of future pilots. Thus, aviation personnel training is a lengthy and demanding process that often involves the most advanced knowledge from psychology, physiology, and IT. Pilot behavior modeling remains an open topic, wherein the tasks are usually solved by structuring the active (for example, voluntary, behavioral, and low-frequency) and passive (involuntary, biodynamic, and mid-frequency) activities, according to the fixed wing aircraft classification. Based on this approach, the active pilot activities relate to phenomena, and the pilot response is connected with flight dynamics, whereas the corresponding passive activities are related to aero-elastic responses. The initial hypothesis for our experiments is that the pilot’s ability to control the aircraft reliably and safely increases with the training. Therefore, there is a certain opportunity to try to measure, test, or evaluate the state of pilot’s ability to control the aircraft appropriately and objectively. The goal of our research focuses especially on experiments to support pilot behavior modeling analysis in association with military flight training, using flight simulators. The nature of this approach lies in repeating measuring the pilots’ responses to a visual stimulus and consequent identification and evaluation of the pilot’s behavioral model parameters, such as reaction delay or lead/lag constants as the so-called equalization part responsible for pilot’s ability to adjust to the controlled dynamics and his/her attitude to control. The resulting parameters can then be used for the following: - Finding common features in the pilots’ behavior during flight control, - Finding the behavioral changes during military flight training and under different influencing factors, - Comparing the control quality of young, less trained pilots with pilots who have completed 400, 500, or more flying hours on real airplanes, not just simulators. In terms of the contents, Section 2provides the mathematical background to pilot modeling techniques; Section 3describes the experiments and example of identification of the presented model parameters; and Section 4delivers the experimental results with statistical analysis and assessment of the pilot training based on an evaluation of the behavioral model parameters. Section 5then provides relevant conclusions. 2. Mathematical Background to Pilot Modeling The ability of a human being to fly an airplane implies processing diverse information, including the transfer of such information to corresponding movements of the airplane control units. Thus, during this process, the pilot performs the function of an adaptive biological controller. Real-life human behavior, especially when “control tasks” are being executed, is hardly stationary in time, mainly due to fatigue or loss of attention. In addition, there are several such tasks where the operators must adjust their strategy due to changes of the vehicle (aircraft) dynamics in time [4,5]. Based on the information presented in the relevant studies [ 1 , 6 ], the human behavior during a control task can be described by using Rasmussen’s general model. It consists of several levels for different types of control. These levels are as follows [7]: - Control level (compensatory feedback control), - Coordination level (control based on rules), Energies 2020,13, 4452 3 of 13 - Organization or cognitive level (control based on knowledge). When modeling pilot maneuvers, it is important to identify the central factors of the simplest possible model that still maintains the main behavior and complexity of the pilot modeling [ 8 ]. Most of the approaches are focused on the modeling of the lowest level of human behavior model (control level), because it is responsible for fast reaction to changes and contingencies, and it is one of the most important parts influencing human control and his/her dynamic properties. Compensatory feedback control was carefully researched for controlling activities, where the human factor could perceive only one controlled variable and other influencing parameters were unpredictable. Much of our current knowledge originates from research of human dynamics during compensation control tasks with one feedback loop [ 6 , 9 ]; this investigation also showed the complexity of studying the human control process and the ability to adapt to various tasks, namely environmental, control, and procedural variables, as they are summarized in the comprehensive overview proposed in Reference [9]. For the purposes of the research presented herein, we used the simplest system with manual operation, namely a single loop compensatory setup with visual stimulus (Figure 1). Energies 2020, 13, x FOR PEER REVIEW 3 of 13 - Control level (compensatory feedback control), - Coordination level (control based on rules), - Organization or cognitive level (control based on knowledge). When modeling pilot maneuvers, it is important to identify the central factors of the simplest possible model that still maintains the main behavior and complexity of the pilot modeling [8]. Most of the approaches are focused on the modeling of the lowest level of human behavior model (control level), because it is responsible for fast reaction to changes and contingencies, and it is one of the most important parts influencing human control and his/her dynamic properties. Compensatory feedback control was carefully researched for controlling activities, where the human factor could perceive only one controlled variable and other influencing parameters were unpredictable. Much of our current knowledge originates from research of human dynamics during compensation control tasks with one feedback loop [6,9]; this investigation also showed the complexity of studying the human control process and the ability to adapt to various tasks, namely environmental, control, and procedural variables, as they are summarized in the comprehensive overview proposed in Reference [9]. For the purposes of the research presented herein, we used the simplest system with manual operation, namely a single loop compensatory setup with visual stimulus (Figure 1). During the compensation phase (using compensatory feedback control), the human regulator works only with control error between the system output (e.g., the flight altitude) and so-called “forcing function”. This function is a random function having stationary or quasi-stationary features (e.g., requirement for the step change of flight altitude) and represents an input signal. Usually, the pilot is expected to minimize the deviation, meaning that he or she is trying to maintain the flight parameters at the required levels defined in the task. The error is minimized via the pilot’s control activity. The error signal, showing the difference between the “target forcing function” and the output (response of an aircraft dynamics), is shown on the display, or the altitude meter in this case. During the control activity, the pilot operates the aircraft dynamics visually, while actively controlling the aircraft movement, in order to minimize the difference between the input and the output signals. This task allows frequency response identification, which defines the pilot behavior and control activity. Figure 1. Single-loop compensatory tracking tasks (based on Reference [1]). Mathematical models of human behavior as a controller at the control level have been described in several dedicated studies [10–14]. Among the outstanding specialists in this area is Prof. D.T. McRuer, whose “Crossover Law” [6] became the starting point for many follow-up experiments and Figure 1. Single-loop compensatory tracking tasks (based on Reference [1]). During the compensation phase (using compensatory feedback control), the human regulator works only with control error between the system output (e.g., the flight altitude) and so-called “forcing function”. This function is a random function having stationary or quasi-stationary features (e.g., requirement for the step change of flight altitude) and represents an input signal. Usually, the pilot is expected to minimize the deviation, meaning that he or she is trying to maintain the flight parameters at the required levels defined in the task. The error is minimized via the pilot’s control activity. The error signal, showing the difference between the “target forcing function” and the output (response of an aircraft dynamics), is shown on the display, or the altitude meter in this case. During the control activity, the pilot operates the aircraft dynamics visually, while actively controlling the aircraft movement, in order to minimize the difference between the input and the output signals. This task allows frequency response identification, which defines the pilot behavior and control activity. Mathematical models of human behavior as a controller at the control level have been described in several dedicated studies [ 10 – 14 ]. Among the outstanding specialists in this area is Prof. D.T. McRuer, whose “Crossover Law” [ 6 ] became the starting point for many follow-up experiments and papers. McRuer’s theory showed that, after accepting certain simplifications, human behavior (his/her dynamic Energies 2020,13, 4452 4 of 13 properties) can be described by using the theory of dynamic systems and theory of automatic control. Based on this theory, researchers characterized and verified many types of human behavioral patterns for simple controlled dynamics [15,16]. By generalizing the theory of Crossover Law and the associated models, human (a pilot’s) behavior can be simplistically described via a transfer function, as shown in Equation (1). The equation describes the dynamic model of human behavior at the control level as a ratio of Laplace transform of an output signal (pilot’s control action)—Y(s) and human pilot’s input signal (perceived input)—U(s) and is valid for the SISO (single-input single-output) generic controlling dynamic systems. F(s)=Y(s) U(s)=K·Qq k=1(TLks+1) hQm i=1(TIis+1)i·(Qn j=1"1 ω2 j s2+2ξj ωjs+1#) ·exp(−τs)(1) The first of the model parameters, K, is referred to as the amplification or gain, which represents the pilot’s habits for the given control action and relates, to some extent, also to the ratio of the output and input signal. Another part of the equation, the “equalization part,” defines the way the pilot adapts to the controlled system and its dynamics. The basis for the pilot’s adaptation consists in the ratio and form of the numerator and the denominator polynomials provided by the values m,n, and q, defining the general structure of the human-pilot behavior model. There are several rules leading to choice of these parameters defining the model structure. Considering the causality of the model, the numerator order must be higher or equal to the denominator order (relation between qand m, nvalues). The dynamic properties of a controlled element also influence the choice of these parameters, as the Crossover Law defines [ 6 ]. Moreover, increasing values of m,n, and qparameters bring more precise approximation of human behavior. According to the abovementioned information, the choice of model structure is especially based on the character of the pilot’s response during control of a certain dynamic system, e.g., an oscillating pilot response requires inclusion of oscillating component ( n>0 ) into the model, or, vice versa, the absence of oscillations defines the model structure as an inertial system (m>0, n=0). Exploiting the multiple experiments characterized in References [ 6 , 17 ] and other sources, we established that, to approximate the control actions (namely, to describe human behavior), the structure is usually sufficient where the values of the m,n, and qparameters range most often as follows: q=0–1, m=0–2, and n=0–1. There are, of course, even more complex models, which imply higher orders of numerator and denominator polynomials (see, e.g., Reference [ 10 ]). These models, however, are used in particular to refine the model at high values of the frequency characteristics or in rough approximation of some nonlinearities. Apart from the structure of the model itself, the individual parameters T L ,T I , ω , and ξ are also very important. These parameters—the parameter ratio, in particular—define, specifically, the resulting control action. For example, the ratio of the time constants T L and T I describes, primarily, the method and approach to control. For the derivative, the lead time constant, T L , reflects the pilot’s ability to predict the given situation, whereas the T I constant, conversely, reflects a certain inertia or lag. The values of the frequency (ω) and damping (ξ) indicate the presence of an oscillating component. The last part of the equation embodies the reaction delay, τ , which is related to the registration and subsequent processing of the visual information. The range where the value of this parameter is usually found was experimentally set to (0.2–1.5 s) [ 11 , 17 ]. The reaction delay is a rather critical parameter, as a situation with an excessively high reaction delay could result in a failure of the pilot’s control abilities and the subsequent destabilization of the controlled system. Such a precondition could lead to catastrophic consequences, or, more narrowly, pilot-induced oscillation. A non-negligible fact contributing to the problem lies in that an increase of this parameter can be caused by, for example, fatigue [18]. Energies 2020,13, 4452 5 of 13 3. Assessing Pilots’ Training Phase and Experience Various researchers are using aviation simulation technologies, in relation to pilot modeling, to increase aviation safety [ 19 , 20 ]. One of the methods to assess objectively a pilot’s current state or training level is based on monitoring alterations of his or her dynamic behavior in time, i.e., at different training stages. The change of dynamic behavior is represented by values of individual behavioral model parameters such as K,T I ,T L , ω , ξ , or τ . This assessment involves the measurement and analysis of the pilot’s responses by using a flight simulator. To acquire pilot responses to mainly visual stimuli, two available simulators were used— a fixed-base engineering flight simulator equipped with X-PLANE 10 Pro software (see Figure 2a) and a moving-base engineering flight simulator based on Flight Gear software solution (see Figure 2b). Energies 2020, 13, x FOR PEER REVIEW 5 of 13 Various researchers are using aviation simulation technologies, in relation to pilot modeling, to increase aviation safety [19,20]. One of the methods to assess objectively a pilot’s current state or training level is based on monitoring alterations of his or her dynamic behavior in time, i.e., at different training stages. The change of dynamic behavior is represented by values of individual behavioral model parameters such as K, TI, TL, ω, ξ, or τ. This assessment involves the measurement and analysis of the pilot’s responses by using a flight simulator. To acquire pilot responses to mainly visual stimuli, two available simulators were used—a fixedbase engineering flight simulator equipped with X-PLANE 10 Pro software (see Figure 2a) and a moving-base engineering flight simulator based on Flight Gear software solution (see Figure 2b). (a) (b) Figure 2. The flight simulators in use—(a) the fixed-base flight simulator and (b) the moving-base flight simulator. Both simulators use software that enables the data recording of flight parameters during simulated flight, such as stick deflection, engine thrusts, flight velocity, flight altitude, angle of attack, etc., with sampling frequency up to 20 Hz. The data acquired from these mathematical–physical calculations in real time are extremely precise, with minimum deviation from the real preset situation. The basic principle of the applied approach (in case of both flight simulators in use) rests in repeated measurement of the response, namely, the pilot’s reaction to a visual stimulus. In our research, the visual stimulus was a simulated step change of the flight altitude, H, indicated by an altimeter. The pilot’s task consisted in reaching the original flight level (flight altitude) as quickly as possible, using solely the joystick deflection (dv). The selected aircraft was King Air C90B and the initial flight mode was defined as altitude 2900 ft and speed 170 mph, and the angle of attack and pitch angle, including their change, was approximately 0 (note 1 ft. = 0.305 m and 1 mph = 0.447 m·s−1). At a certain time, the altitude was stepchanged to 2600 ft., and the task of the pilot was to correct the altitude back to the original 2900 ft., where the current altitude was indicated by the altimeter. The repeated measurements with the same pilots under the same or similar conditions, using the aforementioned flight simulators, were performed. Data representing mainly altitude, H (perceived input signal), and stick (joystick) deflection, dv (pilot’s control action), were acquired as a result of the measurement for the following data processing and parameters identification of the pilot’s behavioral model. An example of the measured data is in Figure 3. Figure 2. The flight simulators in use—( a ) the fixed-base flight simulator and ( b ) the moving-base flight simulator. Both simulators use software that enables the data recording of flight parameters during simulated flight, such as stick deflection, engine thrusts, flight velocity, flight altitude, angle of attack, etc., with sampling frequency up to 20 Hz. The data acquired from these mathematical–physical calculations in real time are extremely precise, with minimum deviation from the real preset situation. The basic principle of the applied approach (in case of both flight simulators in use) rests in repeated measurement of the response, namely, the pilot’s reaction to a visual stimulus. In our research, the visual stimulus was a simulated step change of the flight altitude, H, indicated by an altimeter. The pilot’s task consisted in reaching the original flight level (flight altitude) as quickly as possible, using solely the joystick deflection (dv). The selected aircraft was King Air C90B and the initial flight mode was defined as altitude 2900 ft and speed 170 mph, and the angle of attack and pitch angle, including their change, was approximately 0 (note 1 ft. =0.305 m and 1 mph =0.447 m · s −1 ). At a certain time, the altitude was step-changed to 2600 ft., and the task of the pilot was to correct the altitude back to the original 2900 ft., where the current altitude was indicated by the altimeter. The repeated measurements with the same pilots under the same or similar conditions, using the aforementioned flight simulators, were performed. Data representing mainly altitude, H(perceived input signal), and stick (joystick) deflection, dv (pilot’s control action), were acquired as a result of the measurement for the following data processing and parameters identification of the pilot’s behavioral model. An example of the measured data is in Figure 3. The measured data and character of the pilot’s response enabled us to select the structure of the pilot’s behavior model (see the description in Section 2) and to identify the model parameters. Energies 2020,13, 4452 6 of 13 Energies 2020, 13, x FOR PEER REVIEW 6 of 13 (a) (b) Figure 3. Data representing pilot’s control action (stick deflection) (a) and as a response to the initial step change of altitude (perceived input signal) (b). The measured data and character of the pilot’s response enabled us to select the structure of the pilot’s behavior model (see the description in Section 2) and to identify the model parameters. The model structure had to be chosen with respect to the character of the measured pilot’s response, i.e., the dynamic behavior of the controlled system. One of the most extensively used models is the transfer function in the form of Equation (1), with parameters q = 1, m = 2, and n = 0. The resulting form is then described by Equation (2): 𝐹(𝑠)=𝑌(𝑠) 𝑈(𝑠)=𝐾∙ (𝑇𝑠+1) (𝑇𝑠+1)(𝑇𝑠+1)∙exp (−𝜏𝑠) (2) The illustrated model constitutes a general model with parameters K, TL, TI, and τ, described in the previous chapter, and can be utilized in a wide range of controlling or piloting activities; its properties and application possibilities are discussed and analyzed in References [17,21]. Based on real-life measurements, it was further determined that this model provides a sufficiently accurate approximation of most real human responses, [17,21]. The indisputable advantages of this model are its ease of use and the option to define a physiological interpretation of the individual parameters. The time constant, TN, is a parameter describing the dynamic properties of a human-pilot neuromuscular system, and its value tends to be relatively small (0.05–0.2 s). The TL parameter relates to the prediction ability, while TI expresses a certain inertia or lag and dampening of rapid control actions. The ratio of these constants describes the character of the control action— slow and damped if TI > TL, or vice versa, and fast with derivative features if TI < TL. The values of these parameters are most often in the order of seconds. The remaining parameters, i.e., the gain, K, and the reaction delay, τ, were, together with their properties, discussed in the previous section. Utilizing the experience from the experimental measurements and evaluations of the pilots’ responses, we also used other model structures, especially when oscillations in the response were observed. For these purposes, the model in the form of Equation (1), with q = 1, m = 0, and n = 1, proved to be appropriate. The resulting model is then represented by Equation (3). 𝐹(𝑠)=𝑌(𝑠) 𝑈(𝑠)=𝐾∙ (𝑇𝑠+1) 𝑠 𝜔+2𝜉 𝜔𝑠+1∙exp (−𝜏𝑠) (3) The meaning of the parameters gain (K), derivative/lead constant (TL), and reaction delay (τ) of Model (3) is similar to the previous model, Model (2). Frequency (ω) and damping (ξ) characterize the oscillating component of pilot’s control action. Model (3) is expected to find application in cases of more complex dynamics of the controlled element, such as dynamic, non-minimal phase systems. As an example of a system with non-minimal phase, a dynamic model of flight can be presented. By identifying the parameters of the presented models, it is possible to evaluate the actual state of the pilot in terms of his/her dynamic properties; if these parameters are then monitored in time or at different stages of training, we can use this information to assess the achieved training level [17]. 010 20 30 -0.5 0 0.5 1 t [s] dv [-] 010 20 30 2400 2600 2800 3000 t [s] H [ft] Figure 3. Data representing pilot’s control action (stick deflection) ( a ) and as a response to the initial step change of altitude (perceived input signal) (b). The model structure had to be chosen with respect to the character of the measured pilot’s response, i.e., the dynamic behavior of the controlled system. One of the most extensively used models is the transfer function in the form of Equation (1), with parameters q=1, m=2, and n=0. The resulting form is then described by Equation (2): F(s)=Y(s) U(s)=K· (TLs+1) (TIs+1)(TNs+1) ·exp(−τs)(2) The illustrated model constitutes a general model with parameters K,T L ,T I , and τ ,described in the previous chapter, and can be utilized in a wide range of controlling or piloting activities; its properties and application possibilities are discussed and analyzed in References [17,21]. Based on real-life measurements, it was further determined that this model provides a sufficiently accurate approximation of most real human responses, [ 17 , 21 ]. The indisputable advantages of this model are its ease of use and the option to define a physiological interpretation of the individual parameters. The time constant, T N , is a parameter describing the dynamic properties of a human-pilot neuromuscular system, and its value tends to be relatively small (0.05–0.2 s). The T L parameter relates to the prediction ability, while T I expresses a certain inertia or lag and dampening of rapid control actions. The ratio of these constants describes the character of the control action—slow and damped if T I >T L , or vice versa, and fast with derivative features if T I <T L . The values of these parameters are most often in the order of seconds. The remaining parameters, i.e., the gain, K, and the reaction delay, τ, were, together with their properties, discussed in the previous section. Utilizing the experience from the experimental measurements and evaluations of the pilots’ responses, we also used other model structures, especially when oscillations in the response were observed. For these purposes, the model in the form of Equation (1), with q=1, m =0, and n=1, proved to be appropriate. The resulting model is then represented by Equation (3). F(s)=Y(s) U(s)=K· (TLs+1) s2 ω2+2ξ ωs+1 ·exp(−τs)(3) The meaning of the parameters gain (K), derivative/lead constant (T L ), and reaction delay ( τ ) of Model (3) is similar to the previous model, Model (2). Frequency ( ω ) and damping ( ξ ) characterize the oscillating component of pilot’s control action. Model (3) is expected to find application in cases of more complex dynamics of the controlled element, such as dynamic, non-minimal phase systems. As an example of a system with non-minimal phase, a dynamic model of flight can be presented. By identifying the parameters of the presented models, it is possible to evaluate the actual state of the pilot in terms of his/her dynamic properties; if these parameters are then monitored in time or at different stages of training, we can use this information to assess the achieved training level [17]. Energies 2020,13, 4452 7 of 13 Various types of tools implemented in MATLAB were employed to identify the parameters of Models (2) and (3). One of the options was the identification algorithm based on the fminsearch simplex function, using the criteria function calculated as a sum of deviations squared; another one consisted in System Identification Toolbox. The properties of these approaches were compared in Reference [ 21 ]. A detailed description of the experiment can be found in Reference [17]. 4. Evaluation and Results The authors’ research focuses on assessment of the human factor in aviation, with relevant aspects of interest being pilots’ flight control abilities, military flight training quality, and long-term data acquisition and analysis to facilitate statistical description of the parameter ranges of the behavioral models. The aforementioned parameters of the behavioral models, such as K,T I ,T L , ω , ξ , or τ , describe the dynamic properties of pilot’s behavior and provide information about a pilot’s attitude to control. The applied approach is based on measuring the pilots’ responses to a step change of an altitude using flight simulator(s), as described in the previous section. 4.1. Statistical Evaluation of the Selected Parameters of the Behavioral Model Based on the described principle, a group of 10 military pilots at different training stages was tested on the moving-base flight simulator (see Figure 2b) equipped by Flight Gear software. Each of the individuals underwent 10 repeating tests. An example of the measured data for Pilot No. 1 is in Figure 4. Energies 2020, 13, x FOR PEER REVIEW 7 of 13 Various types of tools implemented in MATLAB were employed to identify the parameters of Models (2) and (3). One of the options was the identification algorithm based on the fminsearch simplex function, using the criteria function calculated as a sum of deviations squared; another one consisted in System Identification Toolbox. The properties of these approaches were compared in Reference [21]. A detailed description of the experiment can be found in Reference [17]. 4. Evaluation and Results The authors’ research focuses on assessment of the human factor in aviation, with relevant aspects of interest being pilots’ flight control abilities, military flight training quality, and long-term data acquisition and analysis to facilitate statistical description of the parameter ranges of the behavioral models. The aforementioned parameters of the behavioral models, such as K, TI, TL, ω, ξ, or τ, describe the dynamic properties of pilot’s behavior and provide information about a pilot’s attitude to control. The applied approach is based on measuring the pilots’ responses to a step change of an altitude using flight simulator(s), as described in the previous section. 4.1. Statistical Evaluation of the Selected Parameters of the Behavioral Model Based on the described principle, a group of 10 military pilots at different training stages was tested on the moving-base flight simulator (see Figure 2b) equipped by Flight Gear software. Each of the individuals underwent 10 repeating tests. An example of the measured data for Pilot No. 1 is in Figure 4. The controlled element dynamics exhibit a relatively strong non-minimum phase; this property is observable in the graphs representing altitude H (see Figure 4) in the form of “undershoot” as a response to sudden change of the pilot’s control actions. The related pilots’ control actions correspond to this effect, i.e., the presence of oscillations is observed in most cases. This fact led to the actual use of Model (3). The applied identification tool was based on our own approach, utilizing the fminsearch simplex algorithm [21]. Figure 4. Data representing altitude, H—perceived input signal (upper) and control action (bottom) of Pilot No. 1 (10 repeated measurements). Figure 4. Data representing altitude, H—perceived input signal ( upper ) and control action ( bottom ) of Pilot No. 1 (10 repeated measurements). The controlled element dynamics exhibit a relatively strong non-minimum phase; this property is observable in the graphs representing altitude H(see Figure 4) in the form of “undershoot” as a response to sudden change of the pilot’s control actions. The related pilots’ control actions correspond to this effect, i.e., the presence of oscillations is observed in most cases. This fact led to the actual use of Model (3). The applied identification tool was based on our own approach, utilizing the fminsearch simplex algorithm [21]. Assuming that the histogram graphically represents the distribution of the selected data in the form of a bar graph, with all columns exhibiting the same width, we can employ it to express the Energies 2020,13, 4452 8 of 13 variance of the observed model parameters of the pilots’ analyzed behavior during the compensatory feedback control phase represented by pilots’ response to the step change of flight altitude, H. Various mathematical expressions were defined for several interval widths (classes), and all of these had been derived from the statistical processing of the input data [ 22 , 23 ]. In the actual pilot testing, we applied the number of analyzes carried out in each flight task with identical flight parameters and the same analyzed parameters (the necessary precondition is the same simulator). From the performed mathematical analyses and information reported in the literature, it can be assumed that the histogram data files have a normal or another standard distribution with a probability of appearance of some outliers. The results yielded statistical information about the possibilities and limitations of the pilots, in general [24,25]. Figures 5and 6present an evaluation of the individual parameters of the pilot’s behavior Model (3) for the aforementioned group of military pilots. All of these histograms exploit the statistical dataset, including approximately 100 values (10 test measurements per 10 pilots). The outliers are neglected. The distribution of the individual parameters is, in most cases, similar to the log-normal distribution, corresponding to the expectations, as the log-normal distribution of probability is often connected with the human factor. The first and, at the same time, one of the most important parameters is the reaction delay, τ . The evaluation of this parameter is included in the histogram in Figure 5. The most frequently occurring value of this parameter reaches about 0.5 s, a level typical of such human (pilot) activities [26]. As mentioned in the preceding section, the reaction delay, τ , is one of the most important parameters. The quantity relates to the visual information processing time, and its value has an invariably negative influence on the controlled process, or flight control in the given case. Other significant information about the human/pilot behavior or dynamic properties related to the described task is outlined in Figure 6. The frequency, ω , with a most frequently occurred value of about 1 rad/s, describes—together with damping, ξ —an oscillating part of the pilot’s responses. The most frequently occurring value of damping, ξ (about 0.3), indicates the presence of oscillations that are only slightly damped. The gain, K, and the lead time constant, T L (with the most probable, and relatively high, value of about 6 s), define the character of the pilots’ control action as a derivative— fast and more aggressive. Energies 2020, 13, x FOR PEER REVIEW 8 of 13 Assuming that the histogram graphically represents the distribution of the selected data in the form of a bar graph, with all columns exhibiting the same width, we can employ it to express the variance of the observed model parameters of the pilots’ analyzed behavior during the compensatory feedback control phase represented by pilots’ response to the step change of flight altitude, H. Various mathematical expressions were defined for several interval widths (classes), and all of these had been derived from the statistical processing of the input data [22,23]. In the actual pilot testing, we applied the number of analyzes carried out in each flight task with identical flight parameters and the same analyzed parameters (the necessary precondition is the same simulator). From the performed mathematical analyses and information reported in the literature, it can be assumed that the histogram data files have a normal or another standard distribution with a probability of appearance of some outliers. The results yielded statistical information about the possibilities and limitations of the pilots, in general [24,25]. Figures 5 and 6 present an evaluation of the individual parameters of the pilot’s behavior Model (3) for the aforementioned group of military pilots. All of these histograms exploit the statistical dataset, including approximately 100 values (10 test measurements per 10 pilots). The outliers are neglected. The distribution of the individual parameters is, in most cases, similar to the log-normal distribution, corresponding to the expectations, as the log-normal distribution of probability is often connected with the human factor. The first and, at the same time, one of the most important parameters is the reaction delay, τ. The evaluation of this parameter is included in the histogram in Figure 5. The most frequently occurring value of this parameter reaches about 0.5 s, a level typical of such human (pilot) activities [26]. As mentioned in the preceding section, the reaction delay, τ, is one of the most important parameters. The quantity relates to the visual information processing time, and its value has an invariably negative influence on the controlled process, or flight control in the given case. Other significant information about the human/pilot behavior or dynamic properties related to the described task is outlined in Figure 6. The frequency, ω, with a most frequently occurred value of about 1 rad/s, describes—together with damping, ξ—an oscillating part of the pilot’s responses. The most frequently occurring value of damping, ξ (about 0.3), indicates the presence of oscillations that are only slightly damped. The gain, K, and the lead time constant, TL (with the most probable, and relatively high, value of about 6 s), define the character of the pilots’ control action as a derivative— fast and more aggressive. Figure 5. Histogram depicting the distribution of reaction delay parameter for the group of 10 military pilots. 0 5 10 15 20 25 30 35 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 Frequency of occurence Reaction delay τ(s) Figure 5. Histogram depicting the distribution of reaction delay parameter for the group of 10 military pilots. Energies 2020,13, 4452 9 of 13 Energies 2020, 13, x FOR PEER REVIEW 9 of 13 Figure 6. Histograms depicting the distribution of parameters of Model (3) for the group of 10 military pilots. 4.2. Assessing the Pilot Training Based on the Evaluation of Behavioral Model Parameters As an example, two independent sets of test results are presented, both performed with seven military pilots completing 10 flight tasks. The participants were students (pilots) in the fourth year of the bachelor’s cycle of study at the Air Force and Aircraft Technology Departments (Faculty of Military Technology, UOD), meaning that they shared the same or similar levels of training. Both sets of tests were carried out under identical conditions on the fixed-base engineering flight simulator (see Figure 2a) equipped with the professional X-PLANE 10 simulation software [21]; the unit is operated by the University of Defense in Brno. The time interval between the first and the second measurement sets was approximately six months. During this period, the pilots also underwent the required military flight training, which included, inter alia, the flight practice of approximately 20 flying hours. Due to those, pilots were only in initial flight training, meaning that they had about 100 flying hours, in total, and those 20 hours are approximately 1/5 of their overall practice, i.e., the impact of the training will be greater than for an experienced pilot with hundreds or thousands of flying hours. For this study, the Model Equation (2) was employed in the modeling of the pilots’ behavior. Based on the previous experience, the model provided sufficient approximation of the pilots’ responses during testing on this type of simulator. The applied identification tool again exploited our own approach, using the fminsearch simplex algorithm. The identified parameters K, TL, TI, TN, and τ (see (2)) for each pilot and flight task can be viewed as statistical datasets, and their basic characteristics can be specified. As mentioned above, seven pilots participated in each measurement set, with each of the participants subjected to 10 repetitive flight tasks; in total, 70 cycles were thus carried out for each set. Although the entire group is not large in terms of statistical significance, the results show that the repeatability of the measurements with respect to the variance of the parameters is satisfactory. 0 5 10 15 20 25 30 35 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 Frequency of occurence Frequency ω (rad/s) 0 5 10 15 20 25 30 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Frequency of occurence Damping ξ (-) 0 5 10 15 20 25 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 Frequency of occurence Gain K (-) 0 5 10 15 20 25 1234567891011 Frequency of occurence The lead constant TL(s) Figure 6. Histograms depicting the distribution of parameters of Model (3) for the group of 10 military pilots. 4.2. Assessing the Pilot Training Based on the Evaluation of Behavioral Model Parameters As an example, two independent sets of test results are presented, both performed with seven military pilots completing 10 flight tasks. The participants were students (pilots) in the fourth year of the bachelor’s cycle of study at the Air Force and Aircraft Technology Departments (Faculty of Military Technology, UOD), meaning that they shared the same or similar levels of training. Both sets of tests were carried out under identical conditions on the fixed-base engineering flight simulator (see Figure 2a) equipped with the professional X-PLANE 10 simulation software [ 21 ]; the unit is operated by the University of Defense in Brno. The time interval between the first and the second measurement sets was approximately six months. During this period, the pilots also underwent the required military flight training, which included, inter alia, the flight practice of approximately 20 flying hours. Due to those, pilots were only in initial flight training, meaning that they had about 100 flying hours, in total, and those 20 h are approximately 1/5 of their overall practice, i.e., the impact of the training will be greater than for an experienced pilot with hundreds or thousands of flying hours. For this study, the Model Equation (2) was employed in the modeling of the pilots’ behavior. Based on the previous experience, the model provided sufficient approximation of the pilots’ responses during testing on this type of simulator. The applied identification tool again exploited our own approach, using the fminsearch simplex algorithm. The identified parameters K,T L ,T I ,T N , and τ (see (2)) for each pilot and flight task can be viewed as statistical datasets, and their basic characteristics can be specified. As mentioned above, seven pilots participated in each measurement set, with each of the participants subjected to 10 repetitive flight tasks; in total, 70 cycles were thus carried out for each set. Although the entire group is not large in terms of statistical significance, the results show that the repeatability of the measurements with respect to the variance of the parameters is satisfactory.