Improved Tactical Decision Making and Control Architecture for Autonomous Truck in SUMO Using Reinforcement Learning

Paper in proceeding, 2023

We employ Reinforcement Learning (RL) techniques with improved state and action spaces for tactical decision making in an autonomous truck. Specifically, we implement Adaptive Cruise Control (ACC) and lane change maneuvers for the autonomous truck in a highway scenario. We show the results obtained using three reinforcement learning algorithms: Deep Q-Network (DQN), Advantage Actor-Critic (A2C) and Proximal Policy Optimization (PPO). Our results demonstrate that it is beneficial to separate high-level decision-making processes and low-level control actions between the RL agent and the low-level controllers based on physical models. Furthermore, we design a realistic reward function based on the Total Cost of Operation (TCOP) of the truck to guide the RL agent towards optimal driving strategy.