In order to successfully complete the Master's degree programme in Computing Science/Business Informatics at the University of Oldenburg, students must take a project group. A project group is a course in which students carry out a complete (software) project over the course of a year. The main objective is to go through the entire development process from analysing the problem to implementing the software solution based on a given problem. Students not only learn the methods and content of their degree programme, but also typical professional working methods such as teamwork, task allocation and taking responsibility. At the same time, personal skills such as the preparation of content, goal-oriented argumentation as well as presentation and judgement skills are promoted.

Project groups are often carried out in co-operation with companies or research institutes. The OFFIS Institute for Computing Science has often offered project groups in the field of transport together with the University of Oldenburg, which has now been accommodated at DLR-SE. The topic of the current PG is orientated towards the research area of autonomous driving and in particular the challenge of environment recognition and classification in road traffic. The name ACDC also reflects this goal.

The solution for developing an automated driving system can be realised using a pattern with SENSE-PLAN-ACT components (also known as the Perception-Planning-Control (PPC) pattern). A system that realises SENSE-PLAN-ACT interacts with an environment. The perception of this is realised by SENSE , while the next step is planned by PLAN and implemented in ACT . This again involves interacting with the environment, whereby changes take place and a new perception is created.

ADORe has already travelled through urban Braunschweig in a presentation by DLR-SE. In the pattern shown, ADORe can be categorised in the PLAN and ACT components. The SENSE component used in Braunschweig cannot be used in a simulation and is not available as open source software. As already motivated, the use of simulation for the development of autonomous driving systems could be implemented more cost-effectively and efficiently. The development of a SENSE component for the CARLA simulation would solve precisely this problem.

A module already exists that can run ADORe in the CARLA simulation. This interface is called the adore_if_carla interface. There is a problem with this: The data that adore_if_carla uses is collected by so-called ground truth providers. Ground truth providers are not sensors, but have knowledge of metadata and the actual data of the CARLA simulation. They can therefore make assumptions that cannot be measured in reality using sensors. This means that a realistic implementation with realistic test data cannot be realised with adore_if_carla . What is missing in the adore_if_carla module is that only realistic sensor data from the simulation is transferred from CARLA to ADORe. The project group would like to solve this problem and therefore develop a solution (SENSE component) that uses realistic sensors and passes this data on to ADORe. With such a module, the software solution could be tested and validated with realistic boundary conditions in the simulation, which is summarised by the following research question:

How can a software-based solution be realised and implemented that prepares the perception for ADORe so that only realistic sensors from CARLA are used for the perception of the environment?

The developers of ADORe have used and tested the system in reality. This means that perception has already been developed in reality. Perception cannot be used by the project group as they do not have access to it. Realistic data has already been transmitted to ADORe. The difference to the ACDC project is that the project group uses a simulation environment. As already mentioned, there are several advantages to testing systems such as ADORe in simulation environments. The development of a solution that is compatible with the CARLA simulation opens up new possibilities for the further development of various solutions within this simulation environment.
What is important is not only the pure implementation of the problem, but also the documentation of the implementation. This means recording which methods and technologies were used. The sensor data for ADORe must be processed because ADORe requires certain information, such as where other road users are located. Therefore, the perception of the environment
must be created in such a way that this information is recognised from the sensor data of CARLA. This means that the challenges for the perceptions are the recognition of objects as well as the environment and the localisation of one's own position.

As defined in the research question, a solution is to be developed that processes sensor data from CARLA and passes it on to ADORe as information. CARLA simulates an environment and then processes the control system of ADORe. In concrete terms, this means that CARLA represents the platform for a vehicle with the simulated sensor data. The data is provided via a Python interface or can be provided by the ROS-Bridge. The ROS Bridge publishes the sensor data under various ROS topics in a defined message type. The processing software solution has no influence on the environment, as the control is determined by ADORe and passed directly to the simulation. Accordingly, only sensor data and environments as defined by the CARLA simulation can be used.

The software solution to be developed is limited by the CARLA simulation used. This means that only maps and vehicles from the CARLA library are used, through which the sensor data is recorded and thus the software solution can be evaluated as a whole. Marginal scenarios, such as a roadworks site or potholes in the road, are not taken into account as they do not exist in CARLA. ADORe provides a perception API to forward the data from the SENSE component to ADORe. The expected data is defined as ROS1 messages, which must be sent to the perception API by the software solution of this project group. The API fully defines what exactly the expected description of the environment should look like and how the corresponding messages should be sent so that control can be determined by the ADORe.

As already described above, there is already an interface solution adore_if_carla, which is based on ground_truth-data to forward the expected inputs to ADORe. The solution to the research question is differentiated by the fact that a selection of sensors is used, which must correspond to reality.

The research question is answered by developing a software solution that forwards the perception of the sensor data from CARLA to ADORe. For this purpose, the environment must be fully recognised by the software solution. Only realistic sensors from CARLA should be used. Realistic means that the data is collected by virtual sensors, which could also be collected by sensors on a vehicle in reality. In contrast to these sensors are ground truth providers. Although they can be used in CARLA as a sensor on a vehicle in the simulation, they deliver ground truth data from the simulation and do not represent real sensors. Realistic sensor data is processed by the ACDC software solution so that the data can be passed to the perception application programming interface (API). The API expects the information about the global and local vehicle position. When using CARLA as a vehicle platform and simulation, these two interfaces can be combined as localisation. Furthermore, the current status of the traffic control system is required, which includes traffic lights and road signs. Information about other road users and unclassifiable objects that could block the road, for example, is also expected.

In summary, the following objective was defined, derived from the research question:

The goal is to design, implement and test the software-based solution ACDC, which prepares the perception for ADORe and uses only realistic sensors from the simulation CARLA for the perception of the environment.

In order to realise the solution, the following use cases are implemented. Firstly, the sensor data must be retrieved from CARLA and processed into information accordingly. As the perception API of ADORe requires four types of information, one use case is created for each of these.

The objectives result in various use cases that the ACDC software solution has to handle.

On the one hand, there is CARLA as a simulation. To answer the research question, sensor data must be received and further processed by the solution. ADORe selebr cannot use sensor data, but provides its own interface through ROS1, on which the data must be published. The components of ADORe's perception API must be supplied with the correct data. This means that ACDC must process information about road users, localisation, unclassifiable objects and traffic control from the simulation from the sensor data in order to be able to transfer this to the perception API of ADORe.