Learning When To Drive in Uncertain Scenarios: A deep Q-learning approach
Doctoral thesis, 2024

The main focus of this thesis is tactical decision-making for autonomous driving (AD) through intersections with other road users. Human drivers can navigate diverse environments and situations, even those they have never encountered before. Autonomous vehicles are expected to have similar capabilities. This thesis specifically addresses the challenge of navigating intersections where the intentions of other drivers are unknown, as these intentions can be influenced by factors such as driver mood, attention, right-of-way, and traffic signals.

To tackle the complexity of manually specifying reactions for every possible situation, this thesis adopts a learning-based strategy using reinforcement learning (RL). The problem is formulated as a partially observable Markov decision process (POMDP) to account for the uncertainty of unknown driver intentions. A general decision-making agent, based on the deep Q-learning algorithm, is proposed. The contributions of this thesis include the development and application of this method to various simulated intersection scenarios, demonstrating its adaptability and effectiveness in different environments with minimal modifications. By accounting for the inherent uncertainty in driver behavior, this approach enhances the robustness and reliability of the autonomous driving system.

transfer learning

Autonomous driving

Partially observable Markov decision process

uncertain environments

neural networks

model predictive control

deep Q-learning

reinforcement learning

decision making

HC1, Hörsalsvägen 14
Opponent: Professor Miguel Ángel Sotelo, University of Alcala, Spanien

Author

Tommy Tram

Chalmers, Electrical Engineering, Systems and control

Learning Negotiating Behavior Between Cars in Intersections using Deep Q-Learning

IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC,;Vol. 2018-November(2018)p. 3169-3174

Paper in proceeding

Learning When to Drive in Intersections by Combining Reinforcement Learning and Model Predictive Control

2019 IEEE Intelligent Transportation Systems Conference, ITSC 2019,;(2019)p. 3263-3268

Paper in proceeding

Reinforcement Learning with Uncertainty Estimation for Tactical Decision-Making in Intersections

IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC,;(2020)

Paper in proceeding

Reinforcement Learning in the Wild with Maximum Likelihood-based Model Transfer

Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS,;Vol. 2024(2024)p. 516-524

Paper in proceeding

T. Tram, M. Bouton, J. Fredriksson, J. Sjöberg, M. J. Kochenderfer Belief State Representations in Reinforcement Learning for Autonomous Driving in Intersections

Our world of transportation is rapidly evolving, with autonomous driving (AD) technology set to revolutionize urban mobility by redefining how we navigate cities, enhancing safety, and paving the way for a more efficient, inclusive, and sustainable future. However, navigating intersections safely remains a critical challenge on the path to this future. This thesis explores using Deep Q-learning to train decision-making agents for intersection navigation, focusing on understanding and managing uncertainty in other drivers' intentions.
Deep Q-learning is a Reinforcement Learning (RL) algorithm which learns through trial and error. In this thesis, agents were developed and tested to master intersection strategies within simulated environments. The goal is to equip these agents with the ability to make split-second decisions based on real-time data about vehicle positions and velocities to estimate what other drivers might do next.

Key highlights of the research include the successful integration of RL algorithms with advanced control methods like Model Predictive Control (MPC), addressing uncertainties through sophisticated AI techniques, and leveraging previous models to train new ones. This combined approach significantly enhanced the agents' ability to anticipate and respond to unpredictable driver behaviors, contributing to the advancement of safety and reliability in autonomous driving systems within real-world environments. The findings of this thesis hold promise for future innovations in AI-driven transportation systems, aiming towards safer roads and more efficient traffic management solutions in our increasingly interconnected world.

WASP SAS: Structuring data for continuous processing and ML systems

Wallenberg AI, Autonomous Systems and Software Program, 2018-01-01 -- 2023-01-01.

Areas of Advance

Transport

Subject Categories

Control Engineering

ISBN

978-91-8103-089-1

Doktorsavhandlingar vid Chalmers tekniska högskola. Ny serie: 5547

Publisher

Chalmers

HC1, Hörsalsvägen 14

Online

Opponent: Professor Miguel Ángel Sotelo, University of Alcala, Spanien

More information

Latest update

9/2/2024 9