Data-driven modelling of hydraulic-head time series: results and lessons learned from the 2022 Groundwater Time Series Modelling Challenge
Journal article, 2024
This paper presents the results of the 2022 Groundwater Time Series Modelling Challenge, where 15 teams from different institutes applied various data-driven models to simulate hydraulic-head time series at four monitoring wells. Three of the wells were located in Europe and one was located in the USA in different hydrogeological settings in temperate, continental, or subarctic climates. Participants were provided with approximately 15 years of measured heads at (almost) regular time intervals and daily measurements of weather data starting some 10 years prior to the first head measurements and extending around 5 years after the last head measurement. The participants were asked to simulate the measured heads (the calibration period), to provide a prediction for around 5 years after the last measurement (the validation period for which weather data were provided but not head measurements), and to include an uncertainty estimate. Three different groups of models were identified among the submissions: lumped-parameter models (three teams), machine learning models (four teams), and deep learning models (eight teams). Lumped-parameter models apply relatively simple response functions with few parameters, while the artificial intelligence models used models of varying complexity, generally with more parameters and more input, including input engineered from the provided data (e.g. multi-day averages). The models were evaluated on their performance in simulating the heads in the calibration period and in predicting the heads in the validation period. Different metrics were used to assess performance, including metrics for average relative fit, average absolute fit, fit of extreme (high or low) heads, and the coverage of the uncertainty interval. For all wells, reasonable performance was obtained by at least one team from each of the three groups. However, the performance was not consistent across submissions within each group, which implies that the application of each method to individual sites requires significant effort and experience. In particular, estimates of the uncertainty interval varied widely between teams, although some teams submitted confidence intervals rather than prediction intervals. There was not one team, let alone one method, that performed best for all wells and all performance metrics. Four of the main takeaways from the model comparison are as follows: (1) lumped-parameter models generally performed as well as artificial intelligence models, which means they capture the fundamental behaviour of the system with only a few parameters. (2) Artificial intelligence models were able to simulate extremes beyond the observed conditions, which is contrary to some persistent beliefs about these methods. (3) No overfitting was observed in any of the models, including in the models with many parameters, as performance in the validation period was generally only a bit lower than in the calibration period, which is evidence of appropriate application of the different models. (4) The presented simulations are the combined results of the applied method and the choices made by the modeller(s), which was especially visible in the performance range of the deep learning methods; underperformance does not necessarily reflect deficiencies of any of the models. In conclusion, the challenge was a successful initiative to compare different models and learn from each other. Future challenges are needed to investigate, for example, the performance of models in more variable climatic settings to simulate head series with significant gaps or to estimate the effect of drought periods.
Author
Raoul Collenteur
Eawag - Swiss Federal Institute of Aquatic Science and Technology
Ezra Haaf
Geology and Geotechnics
Mark Bakker
Delft University of Technology
Tanja Liesch
Karlsruhe Institute of Technology (KIT)
Andreas Wunsch
Fraunhofer Institute for Optronics, System Technologies and Image Exploitation IOSB
Jenny Soonthornrangsan
Delft University of Technology
Jeremy White
INTERA Incorporated
Nick Martin
Southwest Research Institute
Rui Hugman
INTERA Incorporated
Ed De Sousa
INTERA Incorporated
Didier Vanden Berghe
GINGER
Xinyang Fan
University of Bern
University of Erlangen-Nuremberg (FAU)
Tim J. Peterson
Monash University
Jānis Bikše
University of Latvia
Antoine Di Ciacca
Lincoln Agritech Ltd
Xinyue Wang
Brown University
Yang Zheng
Brown University
Maximilian Nölscher
Federal Institute for Geosciences and Natural Resources
Julian Koch
Geological Survey of Denmark and Greenland (GEUS)
Raphael Schneider
Geological Survey of Denmark and Greenland (GEUS)
Nikolas Benavides Höglund
Lund University
Sivarama Krishna Reddy Chidepudi
M2C - Coastal and Continental Morphodynamics Laboratory
BRGM
Abel Henriot
BRGM
Nicolas Massei
M2C - Coastal and Continental Morphodynamics Laboratory
Abderrahim Jardani
M2C - Coastal and Continental Morphodynamics Laboratory
Max Gustav Rudolph
Technische Universität Dresden
Amir Rouhani
Helmholtz
J. Jaime Gómez-Hernández
Polytechnic University of Valencia (UPV)
Seifeddine Jomaa
Helmholtz
Anna Pölz
Vienna University of Technology
Interuniversity Cooperation Centre for Water and Health
Tim Franken
Sumaqua
Morteza Behbooei
David R. Cheriton School of Computer Science
Jimmy Lin
David R. Cheriton School of Computer Science
Rojin Meysami
David R. Cheriton School of Computer Science
Hydrology and Earth System Sciences
1027-5606 (ISSN) 16077938 (eISSN)
Vol. 28 23 5193-5208Subject Categories
Earth and Related Environmental Sciences
Electrical Engineering, Electronic Engineering, Information Engineering
DOI
10.5194/hess-28-5193-2024