The use of AI in AV human-factors research and human-factors requirements in AI-based AV design : Deliverable 2.4 in the EC ITN project SHAPE-IT

Report, 2023

The design of automated vehicles (AVs) today is being enabled by the rise of new technologies, actually in particular recent advances in Artificial Intelligence (AI). Navigating the challenges and potential of this technology is crucial for the organizations that develop AVs, as well as for societies that rely on smart transportation.

In this report, we consider two perspectives on these technologies in AV research and design, with a particular focus on human factors (HF): (A) Human-Factors Requirements in AV Development, and (B) The Use of AI in Research about Vehicle-Human-Interaction. We describe each part separately; they are different enough to stand on their own, while both descriptions together make up this report.

We start with the first perspective – investigating how AI can facilitate HF research and practical use of AI to predict human behaviour for use by HF designers.

To support HF researchers and automation designers with tools for classifying and predicting interaction behaviours between AVs/vehicles and pedestrians in urban environments, we developed AI-based models (eg., Zhang et al., 2023) to predict the outcomes of pedestrian-vehicle interactions at unsignalised crossings. The models include random forest models, support vector machine models, and neural network models. The input consists of multiple features such as time to arrival (TTA), pedestrian waiting time, presence of a zebra crossing, and properties and personality traits of both pedestrians and drivers. The output consists of interaction outcomes such as crossing behaviour, crossing duration, and crossing initiation time. The predicted outcomes can contribute to a better understanding of the interactions. In addition, we analysed the interaction factors in order to support HF researchers and automation designers in their efforts to design safer interaction interface.

We reviewed a large selection of papers that used AI to predict pedestrian behaviour and interactions (Zhang and Berger, 2023). We proposed a framework of AI-based tools for predicting pedestrian behaviours and summarized some guidelines for using AI—especially deep learning methods for pedestrian behaviour and interaction prediction. Furthermore, our own body of work (Zhang et al., 2021, Zhang and Berger, 2022a, Zhang and Berger, 2022b, Zhang et al., 2023) provides detailed steps for developing an example of an AI model.

A key contribution of our research is metrics that allow the evaluation and assessment of AI’s success at classifying and predicting pedestrian-vehicle interactions. In our study, we compared AI models with traditional linear models (Zhang et al., 2023). Further, we compared the performance of AI models and traditional methods with fewer input factors; traditional methods perform well when there are fewer, while AI-based methods perform better when dealing with more input factors. This finding provides information for optimal model selection in different scenarios. To summarize, our findings suggest that AI can help us understand the intentions of human actors and predict their next steps when they interact with AVs.

The second perspective investigates how HF research can facilitate AV development activities. We had anticipated that the reliance of AV on AI technology might play a major role in how developers need to think about HF (hence, this aspect is also reflected in the title of this report). Our reasoning was that AI-based AV provide a larger surface of interaction between humans and AVs, not only through the traditional human machine interface. However, early in the project we identified that there was a need to address not only the AI-based aspects of HF requirements in AV development, but also to address HF requirements overall in AV development – not the least within agile ways of working. We therefore decided to include AI-based AV development considerations as part of the larger scope of studying HF requirements in the context of AV development, with focus on agile processes. The agile angle was chosen as AV development increasingly incorporates agile and continuous development approaches. We find that it is conceptually unclear how to systematically incorporate HF in such a fast-paced environment. Further, the automotive industry used as our subject of study is lacking guidelines (as well as best practices) for incorporating HF into these ways of working. We propose the development and application of a HF requirements strategy to manage key implications, for which our research suggests useful templates and guidelines.