Joint Energy-based Model for Deep Probabilistic Regression

Paper in proceeding, 2022

It is desirable that a deep neural network trained on
a regression task does not only achieve high prediction accuracy,
but its prediction posteriors are also well-calibrated, especially in
safety-critical settings. Recently, energy-based models specifically
to enrich regression posteriors have been proposed and achieve
state-of-art results in object detection tasks. However, applying
these models at prediction time is not straightforward as the
resulting inference methods require to minimize an underlying
energy function. Furthermore, these methods empirically do not
provide accurate prediction uncertainties. Inspired by recent joint
energy-based models for classification, in this work we propose
to utilize a joint energy model for regression tasks and describe
architectural differences needed in this setting. Within this frame-
work, we apply our methods to three computer vision regression
tasks. We demonstrate that joint energy-based models for deep
probabilistic regression improve the calibration property, do not
require expensive inference, and yield competitive accuracy in
terms of the mean absolute error (MAE).