In recent years, the application of machine learning methods led to massive advancements in various fields. This was due to the rapid growth of computation power as well as due to progress in related algorithms. Regarding the field of intelligent control of vehicles, especially the combination of reinforcement learning and deep learning appears to be a very promising way to handle both the agent-environment character of the tasks as well as its representation in continuous spaces.
Since many machine learning tasks can be regarded as having such an agent-environment character, this combination additionally led to a whole new area of machine learning research with a rapidly increasing number of contributions and applications and is usually known as Deep Reinforcement Learning (DRL). We at VSI Frankfurt concentrate on both the field in general as well as on its applications with a focus on intelligent control of vehicles.
This project encompasses both the development of a new software to simulate vehicles on realistic road networks as well as the development of a new DRL algorithm for autonomous driving. Compared to already existent software solutions, our software is intended to provide a more realistic driving situation with, e.g., intersections and other vehicles as well as provide a tailored backend and a Graphical User Interface (GUI) simplifying the development and validation of DRL algorithms for autonomous driving.
Main features of the software:
- a GUI showing the currently loaded road network and the vehicles (see first image on the left)
- create artificial road networks
- load and save road networks
- import realistic road networks from Open Street Maps
- choose different algorithms for path calculation
- determine vehicle properties (e.g. mass of a vehicle)
- show statistics about vehicles (e.g. current velocity of a vehicle) (see second image on the left)
- load actor and critic networks (special to many DRL algorithms)
- disable/enable training and exploration
- save trained models
In this project, we address fully autonomous driving in a complex simulated city-like environment.
The simulation environment consists of the following components:
- a ‘hallucinated’ city, based on real map data
- a versatile and realistic vehicle model
The approach allows to ‘drive’ multiple vehicles at the same time, each of them controlled by an AI agent (or a real driver, should this be desired).
As we have created the complete environment, we have complete understanding of each scene element and can easily produce images of the scene which are perfectly semantically labeled pixelwise. This includes the possibility to produce semantically labeled ‘birds eye views’ of the scene, which are proxies for environment representations obtained from real sensors.
In these environments, each virtual car is controlled by an AI agent which is based on a fusion of control theory, and deep reinforcement learning.
The current challenge is to chose the optimum fusion strategy and the optimum architecture for the combination of deep networks and reinforcement learning methods to achieve maximum safety and maximum comfort while driving.