Klaus Mehl Research focus Example
(page under construction)
Tailored simulator training
Tailored simulator training concept
—
Assessment & training for general aviation
Assessment & training for general aviation
Project leader: Dr Klaus Mehl, University of Oldenburg School I & Research Centre for Safety-Critical Systems (Contact: klaus.mehl@uol.de) in co-operation with: Ashampoo Air Service GmbH & Co. KG, AirShampoo Flight Academy, Ganderkesee Airfield
When aircraft accidents occur, the actions and behaviour of the crew play a central role in over 50 per cent of incidents, as the following breakdown shows:
Figure 1: Boeing report on the causes of aircraft accidents in commercial aviation
As regards the specific risk, statistics such as those from the US National Transportation Safety Board (NTSB, 2003) show that the risk involved in a flight (or co-pilot flight) in general aviation is very low (see Figure 2). Statistically speaking, a person would, for example, have to fly 114,000 times from Ganderkesee or Oldenburg to the island of Juist and back to be involved in an accident. This would mean that a 57-year-old person would have to make this journey twice every day of their life, starting from the very first day of their birth. At the same time, the figures for civil commercial aviation are lower – no doubt not least due to the regular simulator-based training and practice sessions undertaken by pilots in that sector.
Figure 2: Statistical analysis of accidents in general aviation by the NTSB (2003): accident rate per 100,000 flight hours
As shown, the actions and behaviour of the pilots responsible are the main cause of accidents. If one looks further into the underlying causes, the following picture emerges (see Figure 3): In a wide-ranging study, Wiegmann & Shappell (2003) analysed a total of 14,200 accidents in so-called general aviation in greater detail and found that over 70 per cent can be attributed to inadequate skills! In second place are decision errors, accounting for approximately 35 per cent. The results thus point very clearly to a deficiency in the training and, presumably above all, the practical proficiency of general aviation pilots – a finding that should be a cause for concern for all pilots in this category.
Figure 3: Distribution of the root causes of accidents in general aviation
(from: Wiegmann & Shappell 2003, 120)
However, whilst the majority of pilots take care to check the engine oil level carefully before take-off, and some can even quote the pressure readings for each individual cylinder of their engine, there is more optimistic confidence in their own competences and skills than there is sound knowledge. A widespread attitude can be summed up succinctly as: “So far, I’ve always landed where I wanted to; why shouldn’t it work this time too?”
The project described here provides pilots in general aviation with an assessment and training tool that enables them to identify and rectify their own shortcomings. To this end, standardised flight tasks are available, which are carried out in a modified ELITE visual and instrument flight procedure trainer. The concept is, in principle, quite simple: First, the handling and performance data of as many pilots as possible, with a wide range of skills and qualifications, are collected during standardised flight tasks – that is, tasks that are always the same for all participants. Figure 4 illustrates the concept and the steps to be carried out in the research project:
Figure 4: Schematic representation of the steps in the research project
Using a database system, key manoeuvring values (a total of approximately 130 individual parameters, such as altitude, heading, control inputs, power setting, etc.) are recorded at a sampling rate of 60 Hz. This is followed by a statistical analysis which, as a basic first step, examines the range between high and low performance levels. As a result, the distribution of performance becomes apparent at this initial stage, as illustrated in Figure 5. This figure shows, on the vertical y-axis, the deviations from a specified heading (data from approximately 500 pilots is taken into account). The horizontal x-axis shows the deviations from the specified flight altitude. Based on the performance data of a number of pilots considered to be representative, the following questions can thus be answered:
- What are the performance ranges that can actually be achieved?
- Where does the individual pilot fall within this performance spectrum?
The point marked by an arrow highlights an example of an individual performance. It shows that the performance achieved lies within a ‘middle’ range and has room for improvement – indeed, it certainly needs to be improved. However, questions with a different focus can also be answered on this basis. For example, the question of the proportion of pilots who were able to achieve this performance at a specific stage of education or training – in other words, the question of the aptitude and ‘talent’ of individual persons to learn how to fly an aeroplane.
<v:shape id="_x0000_i1029" type="#_x0000_t75" style="width: 453pt; height: 339.75pt;"> </v:shape>
Figure 5: A so-called scatter plot showing the distribution of performance data for approximately 500 pilots
If we now consider not all performance data, but that of individual pilots over the course of each completed training session, we see patterns such as those in Figure 6 below: Once again, as in Figure 5, the course and altitude deviations are plotted, but in each case for a single pilot (vp.). The numbering of the individual performance points indicates the number and sequence of the training sessions completed. In the case of vp.26, for example, it can be seen that she had slightly more difficulty maintaining altitude during the first two runs, 1 and 2; thereafter, however, she consistently achieved quite good performance figures. In the case of vp. 30, the picture is rather the reverse: the first runs consistently demonstrate a high degree of precision in maintaining altitude and heading, and it is only the final runs 9, 14 and 15 that show slightly increased deviations – presumably due to fatigue.
<v:shape id="_x0000_i1029" type="#_x0000_t75" style="width: 453pt; height: 339.75pt;"> </v:shape>
Figure 6: A so-called scatter plot as in Fig. 5, but here the individual performance data are shown over time
In contrast, training participants 6 and 7 present a very different picture. Here, hardly any progress or improvements in performance are discernible. The trainees are stagnating at a rather low level of performance, which raises the question of the underlying causes.
To investigate this, the focus of the analysis must be shifted. Up to now, the analysis has merely looked at different performance levels without exploring why, or by what means, a good performance actually differs from a less good one. However, in addition to this information about the current state of their skills, coaches and pupils want, above all, to know exactly what they need to change in order to improve their performance. Complex statistical analyses are necessary to obtain this information. The methods used for this are primarily those of so-called data mining. Put simply, the principle behind these analyses can be described as follows: two groups of data sets are formed – one comprising ‘good’ performances and the other ‘less good’ performances. A wide variety of performance characteristics are then examined to determine how effectively they can be used to distinguish between these two data sets. In other words, the aim is to identify the features that are as distinct as possible between the two data groups that have been distinguished from one another. According to the theoretical assumption – which still needs to be critically examined – these represent the decisive characteristics that lead to good or poor performance of the task. Consequently, the aim in training sessions is to reduce the behavioural characteristics that lead to poor performance and to enhance those associated with good performance. Figure 7 illustrates this relationship using such a characteristic as an example. In reality, however, differences in performance are generally based on a whole range of characteristics – an aspect which cannot be explored further here. In the example shown, the focus is on pilots’ reaction times, whereby the time is measured that elapses between the start of a deviation from a reference value and the end of this deviation – that is, the start of the correction. As can be seen, the small squares mark the median
<v:shape id="_x0000_i1029" type="#_x0000_t75" style="width: 453pt;"> </v:shape>
Figure 7: So-called box plots of the reaction times for individual training runs across four test participants (tp.)
of these times for the flights already examined in Figures 5 and 6. The small rectangles show the ranges of values between 25% and 75%. It is immediately apparent that the two ‘good’ pilots, 26 and 30, demonstrate significantly faster reaction times. To highlight just one of the striking points here: the reaction times for flight 9 of participant 30 fall well outside the range otherwise shown. A comparative look at Figure 6 shows that this flight also stands out in terms of its quality; in this case, therefore, reaction speed is a significant factor. At the same time, the circumstance already mentioned becomes apparent: that more than just one behavioural characteristic is responsible for the emergence of ‘good’ or ‘poor’ performance: The reaction times for Flight 8 of vp. 26 also fall outside the norm, without this having any discernible impact on the quality of the flight handling (cf. Figures 7 and 6).
For pilots 6 and 7, however, one interim objective is to practise faster reactions!
Another aspect of the project:
The question of the causes of pilot errors and accidents
The approach described is intended not only to contribute to a ‘rational use of simulation’ (Salas, Bowers & Rhodenizer 1998) for educational and training purposes, but also to open up a specific approach to the analysis of operational errors and accidents in the field of aviation. If one moves away from the supposed explanation of ‘human error’ and instead asks what was actually done in the event of an error instead of what was required and desired, then training simulators are an excellent analytical tool. Errors in action also occur during instruction and training, and virtual accidents take place. Unlike under real-world conditions, however, the deviant actions can be recorded very precisely in the simulator
and compared with the documented handling data (as described above) during the training phase. This makes it possible to explore certain questions that are central from a psychological perspective on human behaviour, such as:
- In the strictest sense, are there even such things as erroneous actions – understood as actions in which something is missing, which exhibit ‘damage’ – or do errors in action reveal incorrect behaviour, actions that are merely carried out in the wrong place – and are therefore misplaced, whilst being entirely error-free in themselves?
- Through which systematically created situational conditions can the occurrence of errors in action be deliberately induced in the sense of a synthetic approach?
<v:shape id="_x0000_i1032" type="#_x0000_t75" style="width: 453pt; height: 339.75pt;"> <v:imagedata src="file:///C:\DOKUME~1\KLAUSM~1\LOKALE~1\Temp\msohtml1\01\clip_image017.jpg" o:title="Aerial photograph"> </v:imagedata></v:shape>