AdaSTE: An Adaptive Straight-Through Estimator to Train Binary Neural Networks
Paper i proceeding, 2022

We propose a new algorithm for training deep neural networks (DNNs) with binary weights. In particular, we first cast the problem of training binary neural networks (BiNNs) as a bilevel optimization instance and subsequently construct flexible relaxations of this bilevel program. The resulting training method shares its algorithmic simplicity with several existing approaches to train BiNNs, in particular with the straight-through gradient estimator successfully employed in BinaryConnect and subsequent methods. In fact, our proposed method can be interpreted as an adaptive variant of the original straight-through estimator that conditionally (but not always) acts like a linear mapping in the backward pass of error propagation. Experimental results demonstrate that our new algorithm offers favorable performance compared to existing approaches.

bilevel optimization

backpropagation

straight-through estimator

Binary Neural Networks

Författare

Huu Le

Chalmers, Elektroteknik, Signalbehandling och medicinsk teknik

Rasmus Kjær Høier

Chalmers, Elektroteknik, Signalbehandling och medicinsk teknik

Che-Tsung Lin

Chalmers, Elektroteknik, Signalbehandling och medicinsk teknik

Christopher Zach

Chalmers, Elektroteknik, Signalbehandling och medicinsk teknik

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

10636919 (ISSN)

Vol. 2022-June 460-469
978-166546946-3 (ISBN)

CVPR 2022
New Orleans, USA,

Ämneskategorier

Datorseende och robotik (autonoma system)

DOI

10.1109/CVPR52688.2022.00055

Mer information

Senast uppdaterat

2022-11-24