Communication Scheduling by Deep Reinforcement Learning for Remote Traffic State Estimation with Bayesian Inference
Artikel i vetenskaplig tidskrift, 2022

Traffic awareness is the prerequisite of autonomous driving. Given the limitation of on-board sensors (e.g., precision and price), remote measurement from either infrastructure or other vehicles can improve traffic safety. However, the wireless communication carrying the measurement result undergoes fading, noise and interference and has a certain probability of outage. When the communication fails, the vehicle state can only be predicted by Bayesian filtering with a low precision. Higher communication resource utilization (e.g., transmission power) reduces the outage probability and hence results in an improved estimation precision. The power control subject to an estimate variance constraint is a difficult problem due to the complicated mapping from transmit power to vehicle-state estimate variance. In this paper, we develop an estimator consisting of several Kalman filters (KFs) or extended Kalman filters (EKFs) and an interacting multiple model (IMM) to estimate and predict the vehicle state. We propose to apply deep reinforcement learning (DRL) for the transmit power optimization. In particular, we consider an intersection and a lane-changing scenario and apply proximal policy optimization (PPO) and soft actor-critic (SAC) to train the DRL model. Testing results show satisfactory power control strategies confining estimate variances below given threshold. SAC achieves higher performance compared to PPO.

resource allocation



soft actorcritic

Noise measurement

Bayes methods

power control

Autonomous driving

deep reinforcement learning

Bayesian filtering

interacting multiple model


Vehicle dynamics

proximal policy optimization

Time measurement


Bile Peng

Technische Universität Braunschweig

Yuhang Xie

Technische Universität Braunschweig

G. Seco-Granados

Universitat Autonoma de Barcelona (UAB)

Henk Wymeersch

Chalmers, Elektroteknik, Kommunikation, Antenner och Optiska Nätverk, Kommunikationssystem

Eduard A. Jorswieck

Technische Universität Braunschweig

IEEE Transactions on Vehicular Technology

0018-9545 (ISSN)

Vol. In Press







Mer information

Senast uppdaterat