No Ground Truth at Sea - Developing High-Accuracy AI Decision-Support for Complex Environments

Paper i proceeding, 2023

As AI decision-support systems are increasingly developed for applications outside of traditional organizational confinements, developers are confronted with new sources of complexity they need to address. However, we know little about how AI applications are developed for natural use domains with high environmental complexity, stemming from physical influences outside of the developers' control. This study investigates what challenges emerge from such complexity and how developers mitigate them. Drawing upon a rich longitudinal single-case study on the development of AI decision-support for maritime navigation, findings show that achieving high output accuracy is complicated by the physical environment hindering training data creation. Further, developers chose to reduce the output accuracy and adapt the HMI design to successfully situate the AI application in an existing sociotechnical context. This study contributes to IS literature following recent calls for phenomenon-based examination of emerging challenges when extending the scope frontier of AI and provides practical recommendations for developing AI decision-support for complex environments.