Communication Scheduling by Deep Reinforcement Learning for Remote Traffic State Estimation with Bayesian Inference
Journal article, 2022

Traffic awareness is the prerequisite of autonomous driving. Given the limitation of on-board sensors (e.g., precision and price), remote measurement from either infrastructure or other vehicles can improve traffic safety. However, the wireless communication carrying the measurement result undergoes fading, noise and interference and has a certain probability of outage. When the communication fails, the vehicle state can only be predicted by Bayesian filtering with a low precision. Higher communication resource utilization (e.g., transmission power) reduces the outage probability and hence results in an improved estimation precision. The power control subject to an estimate variance constraint is a difficult problem due to the complicated mapping from transmit power to vehicle-state estimate variance. In this paper, we develop an estimator consisting of several Kalman filters (KFs) or extended Kalman filters (EKFs) and an interacting multiple model (IMM) to estimate and predict the vehicle state. We propose to apply deep reinforcement learning (DRL) for the transmit power optimization. In particular, we consider an intersection and a lane-changing scenario and apply proximal policy optimization (PPO) and soft actor-critic (SAC) to train the DRL model. Testing results show satisfactory power control strategies confining estimate variances below given threshold. SAC achieves higher performance compared to PPO.

Optimization

Estimation

Autonomous driving

Bayesian filtering

Noise measurement

resource allocation

deep reinforcement learning

interacting multiple model

Vehicle dynamics

Bayes methods

Time measurement

Sensors

proximal policy optimization

soft actorcritic

power control

Author

Bile Peng

Technische Universität Braunschweig

Yuhang Xie

Technische Universität Braunschweig

G. Seco-Granados

Universitat Autonoma de Barcelona (UAB)

Henk Wymeersch

Chalmers, Electrical Engineering, Communication, Antennas and Optical Networks

Eduard A. Jorswieck

Technische Universität Braunschweig

IEEE Transactions on Vehicular Technology

0018-9545 (ISSN) 1939-9359 (eISSN)

Vol. 71 4 4287-4300

Subject Categories

Telecommunications

Control Engineering

Signal Processing

DOI

10.1109/TVT.2022.3145105

More information

Latest update

1/15/2023