Predicting Pedestrian Behavior in Urban Traffic Scenarios Using Deep Learning Methods

Licentiate thesis, 2022

Background: The statistics on global road safety show a great demand for reducing the fatalities caused by pedestrian-vehicle collisions. By utilizing artificial intelligence such as deep learning, human drivers can be supported by better driver assistance systems, and thereby the fatalities caused by human errors can be reduced. Therefore, accurately predicting pedestrian behavior is crucial for drivers and automated vehicles to better understand pedestrians in complex scenarios to avoid pedestrian-vehicle collisions. Objectives: This thesis aims to use deep learning to predict pedestrian behavior in urban traffic more accurately. The research goals are: 1) reviewing, categorizing, and analyzing existing research to identify research gaps in pedestrian behavior prediction, 2) developing a model that can more accurately predict pedestrian trajectories in urban traffic by using deep learning to model social interactions, and 3) considering pedestrian-vehicle interactions using deep learning methods when predicting pedestrian trajectories. Methods: In Paper A, the methodology to find and collect existing papers is based on direct search and snowballing. The IEEE Xplore digital library and Google Scholar is used for direct search. Paper B and C have considered social interactions and pedestrian-vehicle interactions using deep learning methods when predicting pedestrian trajectories. A real-world, large-scale open dataset released by Waymo is used for training and evaluation. The average displacement error (ADE) and final displacement error (FDE) are used to quantitatively evaluate the prediction accuracy. Results: Paper A has reviewed 92 papers, 50 from direct searching and 42 from snowballing, and analyzed the models that considered different factors influencing the pedestrian behavior. The advantages and drawbacks of using different prediction methods have been outlined. Research gaps and possible research directions have been pointed out. In Paper B, while the performance on ADE and FDE has been slightly improved by 1.50% and 1.82% compared to the state-of-the-art model, the inference speed has been significantly improved by 4.7 times faster on total inference speed and 54.8 times faster on data pre-processing speed. In Paper C, our proposed pedestrian-vehicle interaction extractor is applied to both sequential and non-sequential models. For sequential models, our model improved the ADE and FDE by 7.46% and 5.24% compared to the state-of-the-art models, and for non-sequential models, our model improved the ADE and FDE by 2.10% and 1.27%. Conclusions: Paper A has shown that including more influencing factors in trajectory prediction has the potential to improve accuracy. Paper B and C have shown that including social interactions and pedestrian-vehicle interactions can improve the accuracy of pedestrian trajectory prediction. By reducing the predicting error and reducing the inference time, our research findings contribute to making approaches for the perception in automated vehicles and driver assistant systems safer than the current state-of-the-art.

Marie Skłodowska-Curie Actions; Innovative Training Network (ITN); Project name: SHAPE-IT; Grant number: 860410; Publication date: 3 June 2023