Reinforcement Learning with Uncertainty Estimation for Tactical Decision-Making in Intersections
Paper i proceeding, 2020

This paper investigates how a Bayesian reinforcement learning method can be used to create a tactical decision-making agent for autonomous driving in an intersection scenario, where the agent can estimate the confidence of its decisions. An ensemble of neural networks, with additional randomized prior functions (RPF), are trained by using a bootstrapped experience replay memory. The coefficient of variation in the estimated Q-values of the ensemble members is used to approximate the uncertainty, and a criterion that determines if the agent is sufficiently confident to make a particular decision is introduced. The performance of the ensemble RPF method is evaluated in an intersection scenario and compared to a standard Deep Q-Network method, which does not estimate the uncertainty. It is shown that the trained ensemble RPF agent can detect cases with high uncertainty, both in situations that are far from the training distribution, and in situations that seldom occur within the training distribution. This work demonstrates one possible application of such a confidence estimate, by using this information to choose safe actions in unknown situations, which removes all collisions from within the training distribution, and most collisions outside of the distribution.

Författare

Carl-Johan E Hoel

Chalmers, Mekanik och maritima vetenskaper, Fordonsteknik och autonoma system

AI Sweden

Volvo Group

Tommy Tram

Zenuity AB

Chalmers, Elektroteknik, System- och reglerteknik, Mekatronik

AI Sweden

Jonas Sjöberg

Chalmers, Elektroteknik, System- och reglerteknik, Mekatronik

IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC

9294407

2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC)
Rhodes, Greece,

WASP SAS

Wallenberg AI, Autonomous Systems and Software Program, 2018-01-01 -- 2023-01-01.

Styrkeområden

Transport

Ämneskategorier

Elektroteknik och elektronik

Sannolikhetsteori och statistik

Datorsystem

DOI

10.1109/ITSC45102.2020.9294407

Mer information

Senast uppdaterat

2021-09-27