Tailored simulator training concept

Tailored simulator training concept

-

Assessment & training for general aviation

Assessment & training for general aviation

Project management: Dr Klaus Mehl, University of Oldenburg School I & Research Centre Safety Critical Systems (contact: klaus.mehl@uol.de) in co-operation with: Ashampoo Air Service GmbH & Co KG, AirShampoo Flight Academy Ganderkesee Airfield

When aircraft have an accident, the actions and behaviour of the crew play a central role in over 50% of incidents, as the following distribution shows:

Figure 1: Boeing report on the causes of commercial aviation accidents

As far as the specific hazard is concerned, statistics such as those of the US National Transportation Safety Board (NTSB, 2003) show that the risk associated with a flight in general aviation is very low (see Figure 2). Statistically speaking, for example, a person would have to fly from Ganderkesee or Oldenburg to the island of Juist and back 114,000 times to be involved in an accident. This would mean that a 57-year-old person would have to make this journey twice every day of their life from the day they were born. At the same time, the figures for civil commercial aviation are lower - certainly not least due to the regular simulator-based training programmes for pilots.

Figure 2: Statistical analysis of accidents in general aviation by the NTSB (2003): Accident rate based on 100,000 flight hours

As shown, the actions and behaviour of the pilots responsible are the main cause of accidents that occur. If we look further into the underlying causes, the following picture emerges (see Figure 3): In a broad-based study, Wiegmann & Shappell (2003) analysed a total of 14,200 accidents in so-called general aviation in more detail and found that over 70% must be attributed to inadequate skills! Decision errors (descision errors) follow in second place with a share of approx. 35%. The results thus very clearly point to a deficient level of training and presumably above all training of general aviation pilots - a finding that should worry all pilots in this aircraft class.

Figure 3: Distribution of the underlying causes of accidents in general aviation
(from: Wiegmann & Shappell 2003, 120)

However, while the majority of pilots are careful to check the engine oil level before take-off, and some can also name the pressure values of each individual cylinder of their engine, there is more optimistic confidence in their own competences and skills than reliable knowledge. A widespread attitude is pointedly: "So far, I've always ended up where I wanted to; why shouldn't I be able to do it now?"

The project to be described here provides general aviation pilots with an assessment and training tool that can be used to recognise and rectify their own deficits. Standardised flight tasks are available for this purpose, which are completed in a modified ELITE visual and instrument flight procedure trainer. In principle, the concept is quite simple: first, the handling and performance data of the largest possible number of pilots with a wide range of skills and qualifications are collected in standardised flight tasks that are always the same for all participants. Figure 4 shows the idea and the steps involved in the research project:

Figure 4: Schematic representation of the steps involved in the research project.

With the help of a database system and a sampling rate of 60Hz, important handling values (a total of approx. 130 individual parameters, such as altitude, course, inputs into the aircraft, etc.) are recorded: Altitude, course, rudder inputs, power setting etc.) are recorded. This is followed by a statistical analysis which, in an elementary first step, analyses the range between high and low performance. As a result, the distribution of performance is visualised at a first stage, as shown in Figure 5. In this figure, the vertical y-axis shows the deviations from a given heading (taking into account the data of approximately 500 pilots). The horizontal x-axis shows the deviations from the specified flight altitude. Based on the performance of a representative number of pilots, the following questions can be answered:

• Where are the achievable performance ranges?
• Where is the individual pilot within this performance spectrum?

The point marked with an arrow emphasises an individual performance as an example. It shows: The performance achieved is in an "average" range and can be improved and certainly needs to be improved. However, questions with a different focus can also be answered on this basis. For example, the proportion of pilots who were able to achieve this performance at a certain level of training, i.e. the question of the suitability and "talent" of individual people to learn to fly an aircraft.

<v:shape id="_x0000_i1029" type="#_x0000_t75" style="width: 453pt; height: 339.75pt;"> </v:shape>

Figure 5: Sog. Scatter plot of the distribution of the performances of approx. 500 pilots

If we now no longer look at all the performances, but at those of individual people in the course of each completed training session, we see images like those in the following Figure 6: Again, as in Figure 5, the course and altitude deviations are plotted, but in each case for a single person (vp.). The numbering of the individual performance points indicates the number and sequence of training sessions completed. In the case of vp.26, for example, it can be seen that she was slightly less able to maintain altitude in the first two runs 1 and 2, but then showed consistently good performance values. With vp. 30 shows a rather reversed picture: the first runs show consistently high precision in maintaining altitude and heading, and only the last runs 9, 14 and 15 show slightly increased deviations - presumably due to fatigue effects.

<v:shape id="_x0000_i1029" type="#_x0000_t75" style="width: 453pt; height: 339.75pt;"> </v:shape>

Figure 6: Sog. Scatter diagram like Fig. 5, but here the individual performances are shown over time

Training participants 6 and 7, on the other hand, are completely different: hardly any progress or improvements in performance are recognisable here. The trainees stagnate at a rather low performance level, which raises the question of the causes.

In order to investigate this question, the direction of the analyses must be changed. Up to now, only different performances have been analysed without investigating the question of why or what actually distinguishes a good performance from a less good one. In addition to this information about their level of ability, coaches and students want to know what they need to change in order to improve their performance. Complex statistical analyses are necessary to obtain this information. Data mining methods are primarily used for this purpose. In simple terms, the principle of these analyses can be described as follows: Two groups of data sets are formed, one with "good" performance and another with "less good" performance. A wide variety of handling features are then analysed to determine how well they can be used to differentiate between these two data sets. In other words, the aim is to find the handling characteristics that are as different as possible in the two separate data groups. According to the theoretical assumption - which then needs to be critically scrutinised - these are the decisive characteristics that result in good or poor mastery of the task. Consequently, the aim of the training sessions is to reduce the behavioural characteristics that cause poor performance and to increase those that are associated with good performance. Figure 7 illustrates this relationship using one such characteristic as an example. In reality, however, the differences in performance are generally based on a whole range of characteristics - an aspect that cannot be discussed further here. In the example shown, we are dealing with the reaction times of the pilots, whereby the time that elapses between the start of the deviation from a reference variable and the end of this deviation, i.e. the start of the correction, is measured. As recognisable, the small squares mark the median

<v:shape id="_x0000_i1029" type="#_x0000_t75" style="width: 453pt;"> </v:shape>

Figure 7: Sog. Box plots of the reaction times of individual training runs for four test participants (vp.)

of these times for the flights already considered in Figures 5 and 6. The small rectangles show the value ranges between 25% and 75%. It is initially recognisable that the two "good" pilots 26 and 30 show significantly faster reactions. To pick out just one of the conspicuous points here: The reaction times of flight 9 of vp. 30 clearly fall outside the range otherwise shown. A comparative look at Figure 6 shows that this flight also stands out in terms of its quality, meaning that the speed of reaction is important in this case. At the same time, the aforementioned fact that more than just one action characteristic is responsible for the realisation of "good" or "bad" performances becomes apparent: The reaction times of flight 8 of vp. 26 are also out of the ordinary, without this having any recognisable effect on the quality of the flight guidance (see Figures 7 and 6).

However, an intermediate goal for pilots 6 and 7 is to practise faster reactions!

Another aspect of the project:

The question of the causes of pilot errors and accidents

The approach described is not only intended to contribute to a "rational use of simulation" (Salas, Bowers & Rhodenizer 1998) for education and training purposes, but also to open up a special approach to analysing handling errors and accidents in the field of aviation. If one moves away from the supposed explanation of "human error" and asks what was actually done instead of what was required and desired in the event of an error, then training simulators are an excellent instrument for analysing the situation. Errors in behaviour and virtual accidents also occur in training operations. However, unlike under real conditions, the deviating actions can be analysed very precisely in the simulator.

tion and compared with the documented handling data (as described above) during the education and training phase. This makes it possible to investigate some central questions from the perspective of behavioural psychology, such as

• In a strict sense, do flawed actions even exist, understood as actions that lack something, that exhibit "damage", or do action errors reveal wrong actions, actions that are merely performed in the wrong place, are thus misplaced and are in themselves completely error-free?
• Which systematically created situational conditions can be used to deliberately induce the occurrence of action errors in the sense of a synthetic approach?

<v:shape id="_x0000_i1032" type="#_x0000_t75" style="width: 453pt; height: 339.75pt;"> <v:imagedata src="file:///C:\DOKUME~1\KLAUSM~1\LOKALE~1\Temp\msohtml1\01\clip_image017.jpg" o:title="Flight photo"> </v:imagedata></v:shape>