On Chow-Liu Forest Based Regularization of Deep Belief Networks
Paper in proceeding, 2019

In this paper we introduce a methodology for the simple integration of almost-independence information on the visible (input) variables of the restricted Boltzmann machines (RBM) into the weight decay regularization of the contrastive divergence and stochastic gradient descent algorithm. After identifying almost independent clusters of the input coordinates by Chow-Liu tree and forest estimation, the RBM regularization strategy is constructed. We show an example of a sparse two hidden layer Deep Belief Net (DBN) applied on the MNIST data classification problem. The performance is quantified by estimating mis-classification rate and measure of manifold disentanglement. Approach is benchmarked to the full model.

Restricted Boltzman machines

Chow-Liu trees

Probabilistic graphs

Author

Alex Sarishvili

Fraunhofer Institute for Mechanics of Materials (Fraunhofer IWM)

Andreas Wirsen

Fraunhofer-Chalmers Centre

Mats Jirstrand

Chalmers, Electrical Engineering, Systems and control

Lecture Notes in Computer Science

0302-9743 (ISSN) 16113349 (eISSN)

Vol. 11731 353-364
978-3-030-30493-5 (ISBN)

28th International Conference on Artificial Neural Networks (ICANN)
Munich, Germany,

Subject Categories

Mathematics

Computer Science

DOI

10.1007/978-3-030-30493-5_35

More information

Latest update

7/30/2020