Leave-One-Out Cross-Validation for Bayesian Model Comparison in Large Data
Paper in proceeding, 2020

Recently, new methods for model assessment, based on subsampling and posterior approximations, have been proposed for scaling leave-one-out cross-validation (LOO) to large datasets. Although these methods work well for estimating predictive performance for individual models, they are less powerful in model comparison. We propose an efficient method for estimating differences in predictive performance by combining fast approximate LOO surrogates with exact LOO subsampling using the difference estimator and supply proofs with regards to scaling characteristics. The resulting approach can be orders of magnitude more efficient than previous approaches, as well as being better suited to model comparison.

Author

Mans Magnusson

Aalto University

Michael Riis Andersen

Technical University of Denmark (DTU)

Johan Jonasson

University of Gothenburg

Chalmers, Mathematical Sciences, Analysis and Probability Theory

Aki Vehtari

Aalto University

Proceedings of Machine Learning Research

26403498 (eISSN)

Vol. 108 341-350

23rd International Conference on Artificial Intelligence and Statistics (AISTATS)
Online, ,

Subject Categories

Bioinformatics (Computational Biology)

Probability Theory and Statistics

Control Engineering

More information

Latest update

1/15/2024