Temporal Evaluation of Uncertainty Quantification Under Distribution Shift
Paper i proceeding, 2025

Uncertainty quantification is emerging as a critical tool in high-stakes decision-making processes, where trust in automated predictions that lack accuracy and precision can be time-consuming and costly. In drug discovery, such high-stakes decisions are based on modeling the properties of potential drug compounds on biological assays. So far, existing uncertainty quantification methods have primarily been evaluated using public datasets that lack the temporal context necessary to understand their performance over time. In this work, we address the pressing need for a comprehensive, large-scale temporal evaluation of uncertainty quantification methodologies in the context of assay-based molecular property prediction. Our novel framework benchmarks three ensemble-based approaches to uncertainty quantification and explores the effect of adding lower-quality data during training in the form of censored labels. We investigate the robustness of the predictive performance and the calibration and reliability of predictive uncertainty by the models as time evolves. Moreover, we explore how the predictive uncertainty behaves in response to varying degrees of distribution shift. By doing so, our analysis not only advances the field but also provides practical implications for real-world pharmaceutical applications.

drug discovery

deep learning

molecular property prediction

uncertainty quantification

temporal evaluation

distribution shift

Författare

Emma Svensson

Johannes Kepler Universität Linz (JKU)

AstraZeneca AB

Hannah Rosa Friesacher

KU Leuven

AstraZeneca AB

Ádám Arany

KU Leuven

Lewis H. Mervin

AstraZeneca AB

Ola Engkvist

AstraZeneca AB

Chalmers, Data- och informationsteknik, Data Science och AI

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

03029743 (ISSN) 16113349 (eISSN)

Vol. 14894 LNCS 132-148
9783031723803 (ISBN)

1st International Workshop on AI in Drug Discovery, AIDD 2024, held as a part of the 33rd International Conference on Artificial Neural Networks, ICANN 2024
Lugano, Switzerland,

Ämneskategorier

Data- och informationsvetenskap

Biologiska vetenskaper

DOI

10.1007/978-3-031-72381-0_11

Mer information

Senast uppdaterat

2024-10-07