Complex temporal amplitude modulation with several modulation frequencies as well as spectro-temporal modulations are stimulus characteristics that may aid the auditory system in identifying sources in acoustic scenes. Many studies on temporal modulations have shown that our auditory system is selective for modulation frequency. However, some recent psychoacoustical experiments indicate that our auditory system is also sensitive to second-order temporal modulations which may interact with first-order modulations (e.g. Verhey et al. 2003). The exact mechanism underlying this ability is still unclear and will be the focus of the dissertation project. One hypothesis is that the second-order modulation may be transformed to first-order modulations in off-frequency filters. To test this hypothesis, new experiments on modulation perception will be performed at high carrier frequencies and low levels where off-frequency information should be negligible. An alternative effective model of second-order modulation perception was proposed in Verhey et al. (2003). They assumed that the auditory system extracts the second order modulations by means of calculating the Hilbert envelope of the ac-coupled first-order envelope (referred to as venelope) explicitly. The model explains the sensitivity to higher-order modulations and can be tested by comparing the model predictions to psychophysical data obtained in the dissertation project. In addition, a more physiologically motivated model will be developed incorporating compression, adaptation and thresholding and tested on the basis of psychophysical results. Some recent physiological studies also indicate selectivity to spectro-temporal modulations of the auditory system (Kowalski et al. 1996). The project will investigate, if such spectro-temporal modulation-frequency selectivity can also be found psychoacoustically. The outcome of the study will provide for a concept of the internal representation of spectro-temporal patterns that can be used to develop applications capable of acoustic source separation based on complex modulation patterns.

The detection of signals from a specific source in an acoustic scene that are embedded in or masked by noise is a vital task for living beings for which their auditory systems have been optimized in evolution. Resulting from the mechanisms of signal production and the propagation characteristics of sounds in the environment, natural stimuli, i.e. the mixture of signals to be detected and the masking noise, often carry a comodulation structure (i.e., temporally correlated amplitude modulations in different frequency bands). This comodulation structure can be exploited to segregate signals and background noise being both represented in the discharge of the neurons in the auditory periphery and throughout the auditory pathway (e.g., Hofer & Klump 2003, Neuert, Verhey & Winter 2004, see also review by Verhey et al. 2003). Several experimental effects (e.g. comodulation detection difference [CDD] or comodulation masking release [CMR]) suggest that the auditory system is able to exploit the comodulation. In our previous work, we developed a model explaining the characteristics of the CDD-effect based on an analysis of the statistics of the stimulus envelope (Buschermöhle et al. submitted). This model can predict the patterns observed in neurophysiological measurements of auditory forebrain neurons (Langemann et al. 2005) and will be extended to other effects and stimulus conditions such as CMR. Further investigations can clarify in more detail how the spectral and temporal properties of the stimulus envelope influence the detection of simple or complex signals within masking noise (collaboration between Feudel, Verhey & Klump). In this context, an extension of the concept of standard stochastic resonance (SR) towards signal detection in comodulated noise and noise with uncorrelated frequency components is planned that may predict improved signal detection in comodulated background noise.

A cochlear implant is a device that is used in deaf and several hearing-impaired patients. For these patients, a regular acoustic hearing aid does not provide sufficient hearing improvement. A cochlear implant functions by direct electrical stimulation of the auditory nerve. With current implant technologies, patient achieve phonem perception scores of typically 60-70%. Although these scores are truly sensational, current implant technologies have some important limitations. One limitation is that implant users are hardly able to identify the pitch of a sound. It is hard for them to distinguish between male and female voices. Although very limited data is available, it must be assumed that speakers of tonal languages (e.g. Manderin), where pitch is a central queue in word meaning, must experience severe limitations. The proposed research aims at introducing the percept of pitch to cochlear implant users. This is done by supplying pitch cues to the speech coding strategy of the implant. Specifically, this is achieved by linking the electrical pulses provided by the implant to the phase of the acoustic signal. The results of the application of the new coding strategy will be evaluated by psychoacoustic methods.

The precedence effect (PE) or "law of the first wavefront" (c.f. overview by Blauert, 1997; Litovsky et al. 1999) allows the listener to localize a sound source primarily by the direct sound arriving at the listener's ear and to suppress the conflicting localization information from early reflections/echoes from the acoustical surrounding. Own previous work using acoustically evoked potentials and related psychoacoustics (e.g., dissertation Damaschke 2004, Damaschke et al. 2005) has shown that at the brainstem level the echoes are still represented, while an echo suppression similar to the psychoacoustical findings can be found at the cortical level using the mismatch negativity paradigm. This contrasts with neurophysiological findings that indicate the generation of the PE already at the brainstem level (e.g., see review of the literature in Tollin et al. 2004). The planned dissertation project will quantify the amount of top-down versus bottom-up processing involved in the precedence effect in humans by using EEG recordings, psychoacoustics and auditory models. Specifically, the suppression effect of echoes from a different direction should be considered with more ecologically relevant stimuli and scenarios (i.e. by using HRTF-filtered clicks and chirp signals instead of simple delayed clicks and by considering multiple echoes from different directions). From the comparison between psychoacoustics and AEP the amount of echo suppression at successive auditory processing stages will quantify the bottom-up contribution, while the influence of different psychophysical tasks (related to the buildup-effect and breakdown of the PE) will help to clarify the influence of top-down processing.

The capability of our auditory system to reliably localize and track acoustical objects in complex, reverberant environment is still not well understood and is by far not matched by any technical device. A number of basic binaural processing mechanisms have been characterized and modelled in the literature (e.g., Jeffress, 1948, Colburn, 1996, McAlpine & Grothe, 2003, Harper & McAlpine, 2004). A number of computational models of spatial hearing have been developed both for better understanding of human localization and for transferring knowledge from biology into technical systems (cf. Blauert, 1997, Braasch, 2002). In own previous work, the weighting of localization cues (dissertation Otten, 2001) and the statistical properties of these cues for localization (dissertation Nix, 2005) in realistic, noisy situations has been characterized and employed for localization models. In addition, modulation processing in the brain was characterized and modelled (e.g., Dau et al. 1997, dissertation, Dicke, 2003). Since both modulation frequency separation and binaural interaction are basic processes performed by the brain stem to separate auditory objects, it is feasible that a combined "binaural modulation map" exists for each center frequency. Such a two-dimensional matrix of modulation frequency vs. binaural displacement might be very efficient in separating acoustical objects occupying the same frequency band, but different modulation frequencies and binaural displacements. Indeed, there is evidence from animal experiments suggesting the relevance of such a representation (Keller & Takahashi, 2001). The existence of such a binaural modulation map in humans will be searched for both with psychophysical methods (i.e., discrimination and detection of objects differing in binaural displacement and modulation frequency in the same spectral region) and with EEG experiments. A computational model will be developed that uses the well-known processing characteristics of the first stages of binaural interaction and modulation processing in combination with cognition-motivated object classification algorithms.

Fish possess the ability to detect moving and vibrating underwater objects using their mechanosensory lateral line organ. The lateral line canal organ is an ensemble of mechano-sensory units, called neuromasts, distributed along the body in canals. We have previously investigated the hydrodynamic excitation and mechanics of canal neuromasts in detail, using mechano- and electrophysiological methods (e.g. van Netten, 1991; Wiersinga-Post & van Netten, 2000, Curčic-Blake & van Netten, 2005). Now we are studying how position and angle of vibration of a hydrodynamic dipole source are encoded in a linear array of neuromasts in the fish lateral line canal by electrophysiologically determining the activity of neuromasts. Results so far, demonstrate that the information on a vibrating source's distance to the fish is linearly coded in the spatial characteristics of the excitation pattern of neuromasts along the lateral line canal. In the planned project several algorithms will be evaluated that could potentially be used by a fish to decode these excitation patterns allowing to localize a source and its axis of vibration.

Evolution should optimize the processing of visual scenes serving the detection and identification of flowers that provide food sources to butterflies. To understand the adaptations in the processing of optical scenes, the mechanisms of visual neurosensory encoding in species of various butterfly families living in different habitats as well as closely related species living in a similar habitat will be analyzed. The spatial and spectral properties of the butterflies' eyes will be mapped with optical imaging, microspectrophotometry, electrophysiology and molecular biological methods. The physiological investigations of eyes and retinae will yield the data for quantitative modeling of the light capture by photoreceptors, specifically of their angular and spectral sensitivity. These characteristics will be used to interpret sensitivity spectra to be recorded from central, colour-sensitive neurons. Modeling of butterfly colour vision will contribute to our understanding how the visual characteristics of butterflies are tuned to the environment.

Studies in humans have demonstrated that the visual perception of a brief flash is modified by auditory input (Shams et al. 2000). If a series of short tone pulses is presented simultaneously with the single flash, multiple flash events are perceived matching the number of tone pulses. Thus, for temporal pattern perception the auditory percept dominates the visual percept. EEG recordings by Shams and colleagues (Shams et al. 2001) suggest that activity in visual cortical areas is modulated by the sound. The occurrence of neurons at lower levels of the CNS, such as neurons in the optic tectum of birds (e.g., Cotter, 1976, Knudsen, 1982) that can both be stimulated by auditory and by visual input, suggests that the mechanism underlying this illusion may already operate at the level of the midbrain. In the planned dissertation project, birds (European starlings) will be trained to discriminate optical temporal patterns consisting of different numbers of flashes. By presenting a single flash with different numbers of tone pulses as a probe stimulus and observing the subject's response, it will be tested whether the starling has the same audio-visual illusion as humans. By recording bimodal neurons from the optic tectum of the starling when stimulated with auditory, visual and audio-visual stimuli it can be directly tested whether this effect already occurs at the level of neurons in the midbrain suggesting that multi-modal integration in the analysis of sensory scenes may already be established at this level.

Multisensory integration is usually considered a fast, mandatory, and automatic process that is not affected by attentional limits. However, recent investigations with the McGurk effect (Tiippana et al. 2004, van Wassenhove et al. 2005) have suggested that the integration of visual and auditory information is modulated by simultaneous demands on attention. It is not yet entirely clear which stage of multisensory integration is most vulnerable to attentional factors. Extending our previous work with saccadic reaction times to simple visual and auditory stimuli under divided- and focused-attention paradigms (Diederich & Colonius, 2004, Arndt & Colonius, 2003), this project will try to elucidate the temporal role of attention in visual-auditory object formation. In a visual-auditory speeded identification (or n-alternative-forced choice) task, speech sounds dubbed onto a facial display articulating a sound will be presented. V-A pairs will either be congruent or incongruent and the task will be performed under different attention instructions (divided or focused attention with respect to modality). The EEG results of van Wassenhove et al. (2005) suggest that the predictive value of the visual input modulates early stages of auditory processing. If early multisensory integration holds, varying this predictive value under specific attention instructions should result in a specific pattern of identification performance.

Psychoacoustic studies in humans (e.g., Balota & Duchek, 1986, Clement et al. 1999) and in animals (e.g., for an example in a songbird, the European starling, see Zokoll et al. 2005) indicate that auditory stimulus characteristics such as pitch can be compared over intervals of up to 20s. Such a time span for auditory memory is important for evaluating long sequences of acoustic communication sounds as are found in human speech or birdsong. In an ongoing dissertation project of the InterGK (Zokoll) the duration of auditory memory is determined in the European starling using psychophysical methods. In the planned project the neural correlate of auditory short term memory will be studied. Changes in neural response patterns that are related to the previous sensory stimulation in a behavioural task have been observed both in sensory areas of the bird forebrain (e.g. in starlings trained to discriminate natural sounds, see Gentner & Margoliash, 2003) and in associative areas of the bird brain that are comparable to the mammalian prefrontal cortex (pigeons trained in a Go/NoGo task involving both auditory and visual stimulation, see Kalt et al. 1999). We will record from auditory and associative areas in the forebrain of the European starling while performing an auditory delayed-non-matching-to sample or delayed-matching-to-sample task. The response patterns will be analysed with reference to models of mechanisms of sensory stimulus discrimination over extended time periods (e.g., Machens et al. 2005). The project aims at characterizing the mechanisms underlying auditory short term memory and at identifying brain areas that are potentially involved.