IRIS: Inverse Reinforcement-Learning and Intelligent Swarm Algorithms for Resilient Transportation Networks

Research Project, 2020 – 2022

Optimization of mixed-traffic flow with multiple objectives (efficiency and safety) is an open problem with societal and economic repercussions when human operated vehicles (HOV) and connected autonomous vehicles (CAV) with different autonomy levels must co-exist. The impact of partially or fully autonomous vehicles with V2V/I2V connection capabilities on the efficiency and safety of the traffic flow must be further explored for their effective deployment. To benefit fully from CAV, traffic flow should be observed and regulated using innovative methods accounting for all types of vehicles. The mixed-traffic flow optimization problem has several challenges: (1) Road network capacity changes dynamically depending on the time-slot in the day, weather, road-closures due to constructions or accidents, (2) The observation and prediction of HOV trajectories are necessary to plan and coordinate the navigation and dispatch routes of CAV, since HOV introduce extra stochastic parameters in microscopic interaction. (3) Quality, reliability, availability of communication network influences vehicular CAV-HOV network, (4) The actual autonomy-level of some of the CAV may degrade into partial or no-autonomy if the sensors and eventually perception and decision-making modules fail.
Formal and analytical methods are not fully able to capture this dynamic and complex nature of mixed traffic-flow. The complexity also stems from the relation between the single vehicle behavior, examined by microscopic traffic models and traffic network dynamic behavior, represented by macroscopic models.
Therefore, we will consider both HOV and CAV as agents and use multi-agent simulations featuring swarm intelligence to better understand the emergent behavior at macroscopic level by realistically representing the CAV and HOV at microscopic level. The CAV could be considered as programmable and regulating agents whereas HOV behavior could be observed and predicted in an offline manner using inverse reinforcement learning and updated when necessary.
We aim to uncover couplings between microscopic parameters (related to kinematics and decision-making sequence) of individual HOV and CAV agent behavior and the emergent macroscopic traffic network performance and congestion parameters. In return, by discovering the stable modes of traffic flow, we will propose control mechanisms affecting the individual agent behavior so that the stability can be preserved under different weather, sensor malfunction, CAV/HOV ratio and connectivity conditions. The ultimate goal of this project is to obtain self-regulating and resilient traffic networks where a heterogenous mixture of vehicle agents must interact.