Brain Machine Interfacing:
In a collaboration with Fraunhofer in Magdeburg, the Knight Lab at UC Berkeley and UC San Francisco we recently started a project on Brain Machine Interfacing (BMI). Our goals are to use non-invasive and invasive brain activation to control robotic devices, and for cognitive BMIs. We have recently organized the interdisciplinary 1st Magdeburg Workshop an Brain Machine Interfacing.
Publications on BMI
Collaborators: Robert T. Knight, Ulrich Schmucker, Edward Chang
CRITICAL SYSTEMS ENGINEERING FOR SOCIO-TECHNICAL SYSTEMS: THE CAR THAT CARES
The socio-technical relationship in a future caring car is characterized by the relationship between an individual driver with the car’s technical system and the entire surrounding traffic and roadside infrastructure. The scenarios of relevance for this project are complex driving situations where several drivers, cars and smart infrastructure cooperatively plan and execute driving maneuvers. The vision for a future socio-technical car system–a car that cares–goes far beyond: cars will be equipped with sophisticated cooperative assistance systems that allow the individual traffic participants to solve specific driving maneuvers cooperatively and negotiate the course of action as part of the socio-technical system of the driver, the car and the surrounding traffic participants.
The overall objective of The Car that Cares (CtC) is to develop processes, techniques and software tools to enable the development of socio-technical car systems that continuously adapt to their internal context (e.g. cognitive state of the driver, current tasks) and external context (e.g. weather, traffic) context and offer situation-aware cooperative interaction with the driver of the car.
The principal investigators in the project are from related research areas of cognitive psychology, human modeling, ambient assisted living, multimodal user interfaces, and critical systems engineering. In different work packages their work will focus on the following tasks: The development of a model the cooperation of many drivers and many systems involving innovative forms of driver assistance system interaction. Multisensory observation of the driver within the social-technical system will drive the assessment of the drivers cognitive, emotional and health state. Based on the situation in the socio-technical, cognitive, emotional and health state of the driver, the car that cares offers assistance and recommendations to the driver to increase safety, comfort and health through adaptive multimodal user interfaces involving different senses. The methods, techniques and tools developed will be integrated in a design methodology that keeps human factors and functional design in balance during the phases of system design, implementation and evaluation.
A detailed description of the entire project can be found in the link below:
Acquisition of information from natural scenes:
The human visual system acquires information from cluttered natural scenes much faster and more efficient than one might expected from experiments using relatively simple stimuli (dots, gratings etc.). Only 40 ms cortical processing of a photography of a natural scene are sufficient to discriminate scenes according to their semantic content, for semantic object contetxt interactions to develop and to obtain enough information to rember the seen as previously seen. We investigate the dynamics of the information acquisition from natural scenes and the interactions between object and context at several cognitive and perceptual levels. In our investigations we combine fMRI, MEG, EEG, psychophysics, and single trial classification approaches to analyze the sequence of the brain processes involved in information extraction and recognition, the role of prior knowledge about the structure of the natural world, and to test the predictivity of brain processes for the subjective percept on a trial-by-trial basis.
Constructive perception and eye movements:
We perceive objects in our environment as integrated wholes, even when they are covered by other objects, and thus only some fragments of the object are simulatneously visible. These subjective object percepts are of high ecological importance as they allows us to recognize and react to obects even when they are only partly visible (e.g. a predator sneaking behind trees). How the visual system constructs the subjective object percepts is still a mystery. Suggestions range from the highly cognitive "knowing what it is hypothesis" to the sensory, so-called "retinal painting hypothesis" which was put forward for example by Helmholtz some 150 years ago. The latter assumes that the eyes follow the occluded "object" and thereby paint successively visible object parts onto adjacent parts of the retina. Our investigations show that under natural free viewing conditions the retinal painting hypothsis can be rejected because retinal painting by smooth pursuit eye movements is neither necessary nor sufficient to explain the figure percept. We currently investigate the brain networks that construct the object percepts and how the brain switches between the percept of the physical stimulus and the subjective object percept.
Publications on constructive object perception and eye movements.
Collaborator: Robert Fendrich
Voluntary eye movements:
Humans scan the visual environment with a rapid sequence of voluntary saccadic eye movements that move the center of regard between differnt point of interest in a scene. Despite the shifts of the retinal image we do not perceive the world as moving between saccades. This is, however, the case when the retinal images are presented as a movies. Our aim in this project was to investigate the effects of voluntary eye movements in the visual system and how the brain constructs the stable percept of the world.
Color and motion processing:
According to the classic view the brain processes visual information in a fast color insensitive and in a slow color sensitive channel.
We investigate via parametric designs using fMRI the temporal and chromatic sensitivity of visual ares in the brain with simple and complex stimuli. We implemented retinotopic mapping and other functional localizers to perform detailed measurements in independently localized visual areas.
Publications on color coding
Collaborators: Karl Gegenfurtner, Brian Wandell
Consciousness and the freedom of will
I'm interested in interdisciplinary, epistemological, and ethical aspects of the discussion about consciousness and freedom of will. Together with colleagues from Philosophy and Psychology we organized in 2002 an interdisciplinary workshop on the topic. The results are published as a book (sorry in German only).