Bayesian Linear Regression on Deep Representations
Paper in proceedings, 2019

A simple approach to obtaining uncertainty-aware neural networks for regression is to do Bayesian linear regression (BLR) on the representation from the last hidden layer. Recent work [Riquelme et al., 2018, Azizzadenesheli et al., 2018] indicates that the method is promising, though it has been limited to homoscedastic noise. In this paper, we propose a novel variation that enables the method to flexibly model heteroscedastic noise. The method is benchmarked against two prominent alternative methods on a set of standard datasets, and finally evaluated as an uncertainty-aware model in model-based reinforcement learning. Our experiments indicate that the method is competitive with standard ensembling, and ensembles of BLR outperforms the methods we compared to.

bayesian linear regression

deep learning

uncertainty estimation

Author

John Moberg

Chalmers, Electrical Engineering, Signal Processing and Biomedical Engineering, Signal Processing

Lennart Svensson

Chalmers, Electrical Engineering, Signal Processing and Biomedical Engineering, Signal Processing

Juliano Pinto

Chalmers, Electrical Engineering, Signal Processing and Biomedical Engineering, Signal Processing

Henk Wymeersch

Chalmers, Electrical Engineering, Communication and Antenna Systems, Communication Systems

Advances in Neural Information Processing Systems 32

4th workshop on Bayesian Deep Learning (NeurIPS 2019)
Vancouver, Canada,

Areas of Advance

Information and Communication Technology

Subject Categories

Computer Science

More information

Latest update

12/28/2020