By measuring eye movements using a camera, it is possible to determine where a person is looking at a certain point in time. This eye movement can be used in speech intelligibility studies to obtain information about the exact time of comprehension due to the close temporal interaction between the eyes and hearing. In an ongoing study in the DFG-funded AULIN project, the eye movements of test subjects are tracked during the presentation of speech using an eye tracker system. In contrast to the audiological tests previously used to investigate speech intelligibility, the time of comprehension can be measured by simultaneously presenting speech and the images matching the speech material - and created specifically for this purpose. The participants have to decide during the measurement which of the images shown on the screen matches the sentence they have heard. For example, whether the clever professor is kissing the wet mermaid or the wet mermaid is kissing the clever professor. [With the help of a corresponding data analysis, the time of comprehension, i.e. the time at which the participant looks at the matching picture and "the penny drops", can then be determined. Based on the measured eye movements, it has already been shown that it takes different lengths of time for the sentence to be understood, depending on its complexity.

The eye tracker system can also be used to measure pupil size. The size of the pupil can provide an indication of how strenuous a listening situation is for the listener. If speech has to be understood in an acoustically complex situation, such as in background noise, a larger pupil diameter can be an indication of increased effort on the part of the listener.
[24.04.2012]