On Chow-Liu Forest Based Regularization of Deep Belief Networks
Paper i proceeding, 2019

In this paper we introduce a methodology for the simple integration of almost-independence information on the visible (input) variables of the restricted Boltzmann machines (RBM) into the weight decay regularization of the contrastive divergence and stochastic gradient descent algorithm. After identifying almost independent clusters of the input coordinates by Chow-Liu tree and forest estimation, the RBM regularization strategy is constructed. We show an example of a sparse two hidden layer Deep Belief Net (DBN) applied on the MNIST data classification problem. The performance is quantified by estimating mis-classification rate and measure of manifold disentanglement. Approach is benchmarked to the full model.

Restricted Boltzman machines

Chow-Liu trees

Probabilistic graphs

Författare

Alex Sarishvili

Fraunhofer-Institut für Werkstoffmechanik IWM (Fraunhofer IWM)

Andreas Wirsen

Stiftelsen Fraunhofer-Chalmers Centrum för Industrimatematik

Mats Jirstrand

Chalmers, Elektroteknik, System- och reglerteknik

Lecture Notes in Computer Science

0302-9743 (ISSN)

Vol. 11731 353-364

28th International Conference on Artificial Neural Networks (ICANN)
Munich, Germany,

Ämneskategorier

Matematik

Datavetenskap (datalogi)

DOI

10.1007/978-3-030-30493-5_35

Mer information

Senast uppdaterat

2020-07-30