Predicting Pedestrian Behavior in Urban Traffic Scenarios Using Deep Learning Methods
Licentiate thesis, 2022

Background: The statistics on global road safety show a great demand for reducing the fatalities caused by pedestrian-vehicle collisions. By utilizing artificial intelligence such as deep learning, human drivers can be supported by better driver assistance systems, and thereby the fatalities caused by human errors can be reduced. Therefore, accurately predicting pedestrian behavior is crucial for drivers and automated vehicles to better understand pedestrians in complex scenarios to avoid pedestrian-vehicle collisions. Objectives: This thesis aims to use deep learning to predict pedestrian behavior in urban traffic more accurately. The research goals are: 1) reviewing, categorizing, and analyzing existing research to identify research gaps in pedestrian behavior prediction, 2) developing a model that can more accurately predict pedestrian trajectories in urban traffic by using deep learning to model social interactions, and 3) considering pedestrian-vehicle interactions using deep learning methods when predicting pedestrian trajectories. Methods: In Paper A, the methodology to find and collect existing papers is based on direct search and snowballing. The IEEE Xplore digital library and Google Scholar is used for direct search. Paper B and C have considered social interactions and pedestrian-vehicle interactions using deep learning methods when predicting pedestrian trajectories. A real-world, large-scale open dataset released by Waymo is used for training and evaluation. The average displacement error (ADE) and final displacement error (FDE) are used to quantitatively evaluate the prediction accuracy. Results: Paper A has reviewed 92 papers, 50 from direct searching and 42 from snowballing, and analyzed the models that considered different factors influencing the pedestrian behavior. The advantages and drawbacks of using different prediction methods have been outlined. Research gaps and possible research directions have been pointed out. In Paper B, while the performance on ADE and FDE has been slightly improved by 1.50% and 1.82% compared to the state-of-the-art model, the inference speed has been significantly improved by 4.7 times faster on total inference speed and 54.8 times faster on data pre-processing speed. In Paper C, our proposed pedestrian-vehicle interaction extractor is applied to both sequential and non-sequential models. For sequential models, our model improved the ADE and FDE by 7.46% and 5.24% compared to the state-of-the-art models, and for non-sequential models, our model improved the ADE and FDE by 2.10% and 1.27%. Conclusions: Paper A has shown that including more influencing factors in trajectory prediction has the potential to improve accuracy. Paper B and C have shown that including social interactions and pedestrian-vehicle interactions can improve the accuracy of pedestrian trajectory prediction. By reducing the predicting error and reducing the inference time, our research findings contribute to making approaches for the perception in automated vehicles and driver assistant systems safer than the current state-of-the-art.

urban traffic

Automated driving

deep learning

pedestrian trajectory prediction

pedestrian behavior

social interaction

pedestrian-vehicle interaction

Jupiter322
Opponent: Prof. Maria Riveiro at the Department of Computing, School of Engineering, at Jönköping University

Author

Chi Zhang

University of Gothenburg

Chi Zhang and Christian Berger. Pedestrian Behavior Prediction Using Deep Learning Methods for Urban Scenarios: A Review

Social-IWSTCNN: A social interaction-weighted spatio- temporal convolutional neural network for pedestrian trajectory prediction in urban traffic scenarios

IEEE Intelligent Vehicles Symposium, Proceedings,; Vol. 2021-July(2021)p. 1515-1522

Paper in proceeding

Learning the Pedestrian-Vehicle Interaction for Pedestrian Trajectory Prediction

2022 8th International Conference on Control, Automation and Robotics (ICCAR),; Vol. 2022-April(2022)p. 230-236

Paper in proceeding

Supporting the interaction of Humans and Automated vehicles: Preparing for the Environment of Tomorrow (Shape-IT)

European Commission (EC) (EC/H2020/860410), 2019-10-01 -- 2023-09-30.

Areas of Advance

Information and Communication Technology

Transport

Subject Categories

Vehicle Engineering

Computer Vision and Robotics (Autonomous Systems)

Publisher

University of Gothenburg

Jupiter322

Online

Opponent: Prof. Maria Riveiro at the Department of Computing, School of Engineering, at Jönköping University

Related datasets

Waymo Open Dataset [dataset]

More information

Latest update

7/19/2023