When machines decide

Computing scientists Werner Damm and Martin Fränzle are researching an ethical concept for autonomous systems such as self-driving cars - so that people are not neglected. In an interview for EINBLICKE magazine, they talk about goals and challenges. An extract. QUESTION: Automation is advancing inexorably. You are working intensively on the technological foundations. Where is the journey heading - are people in the world of work currently doing away with themselves? FRÄNZLE: That is controversial among experts. Some experts are convinced of this and are therefore calling for an unconditional basic income. Others believe that the historical evidence speaks against it. They point to past industrial revolutions, which have instead brought about major changes in occupational fields. DAMM: Either way, we are facing major changes. For example, in the area of mobility - entire professions such as taxi drivers, bus and lorry drivers will disappear in the medium to long term. QUESTION: In which areas do you expect particular progress? DAMM: At the university, for example, we are researching how medical care and aftercare can be technically optimised. (...) FRÄNZLE: Another research focus is the design of the energy supply. Even today, the fluctuating demand and supply of electricity are constantly being controlled and balanced. This task is becoming ever more complex with the increasing proportion of energy from renewable sources. (...) DAMM: The third major field is autonomous driving with the aim of reducing accidents, using resources more efficiently and minimising emissions. New technology must serve people QUESTION: Automation will ultimately turn driving into a mobility service. Are people ready for this yet? FRÄNZLE: It depends: Do I have a situational autonomous driving system that only manoeuvres on the motorway between on-ramps and off-ramps? This allows me to retain my own driving ability, try things out and build up confidence. Or does the car drive fully automatically? Then I could put my children in the car and say: "Drive them to music school." The reservations would certainly be much greater here. DAMM: Overall, acceptance depends heavily on the extent to which people can build trust in autonomous driving by remaining selectively in the loop. One thing must always be clear: ultimately, the new technology only makes sense if it serves people and is accepted by them. QUESTION: Mr Damm, as a member of a working group of the Federal Transport Minister's Ethics Commission, you helped to adopt guidelines on automated driving. What were they about? DAMM: The challenge that we have to face with highly automated driving is that everything that a human would normally decide and take responsibility for is also taken over by the vehicle in critical driving situations. The behaviour of the system must be programmed accordingly. Two of the central questions that we discussed in the working group were therefore: How can we organise the decision in such a way that it is easy for people to understand and transparent? And how can we ensure that it corresponds to socially recognised values? QUESTION: What were the commission's findings? DAMM: Firstly, that human life is in principle more worthy of protection than things. Point two relates to the much-cited dilemma situation in which an autonomous vehicle has to decide between two evils: Does it drive into a group of people with two people or one with five? Should human lives be weighed against each other? The Commission has clearly rejected this, by analogy with a ruling by the Federal Constitutional Court on aircraft hijackings and the authorisation to shoot down aircraft discussed in this context. In such a case, the lives of the passengers on board must not be sacrificed in favour of the lives of a larger number of people on the ground. QUESTION: These are highly complex questions that certainly cannot be answered by Computing Science experts alone... FRÄNZLE: That's right. We are indeed currently undergoing a paradigm shift. Until now, as engineers, we were used to building and analysing system models that were perfect blueprints for our software and hardware products. With semi-autonomous driving, our models can no longer be perfect because they have to incorporate people and their characteristics into the overall system. This is the only way we can analyse the dynamic interaction between man and machine. Rules that machines must also abide by QUESTION: So you also need the findings of other disciplines, especially social sciences? FRÄNZLE: That is the case, yes. For example, the findings of psychology, philosophy, sociology, political science and Law. The systems have to be programmed in such a way that they are orientated towards ethical standards, for example. QUESTION: Philosophy, for example, would be in demand here... FRÄNZLE: Exactly, because it's about adopting rules for human actions that a machine must also adhere to. This is linked to the question of the conditions under which certain actions are permitted or prohibited. DAMM: Psychology, on the other hand, provides us with important approaches to explaining human behaviour: how can the technical system find out what a person is up to? How can their attention be controlled? And how can systems explain their decisions to people? QUESTION: What about the legal controllability of autonomous systems? Humans hand over responsibility to the machine - that certainly has consequences in terms of liability. DAMM: That is an important aspect - and the situation is complex. After all, safety-relevant driving decisions not only include information that comes from the vehicle itself, i.e. for which the manufacturer is responsible. Information from other vehicles is also transmitted by radio. But what if the car in front of you perceives the world differently and provides incorrect data? Or if communication lines have been "hacked"? Secondly, the autonomous system needs highly up-to-date maps - and this data comes from the cloud. Who ensures that the maps have not been falsified? (...) You can find the full interview in the current issue of our research magazine EINBLICKE (from page 16).