Reliability Analysis of Compressed CNNs
Rapport, 2021

The use of artificial intelligence, Machine Learning and in particular Deep Learning (DL), have recently become a effective and standard de-facto solution for complex problems like image classification, sentiment analysis or natural language processing. In order to address the growing demand of performance of ML applications, research has focused on techniques for compressing the large amount of the parameters required by the Deep Neural Networks (DNN) used in DL. Some of these techniques include parameter pruning, weight-sharing, i.e. clustering of the weights, and parameter quantization. However, reducing the amount of parameters can lower the fault tolerance of DNNs, already sensitive to software and hardware faults caused by, among others, high particles strikes, row hammer or gradient descent attacks, et cetera. In this work we analyze the sensitivity to faults of widely used DNNs, in particular Convolutional Neural Networks (CNN), that have been compressed with the use of pruning, weight clustering and quantization. Our analysis shows that in DNNs that employ all such compression mechanisms, i.e. with their memory footprint reduced up to 86.3x, random single bit faults can result in accuracy drops up to 13.56%.

Fault Tolerance

Caffe

CNNs

Compression

Machine Learning

Författare

Stefano Ribes

Chalmers, Data- och informationsteknik, Datorteknik

Alirad Malek

Chalmers, Data- och informationsteknik, Datorteknik

Pedro Petersen Moura Trancoso

Chalmers, Data- och informationsteknik, Datorteknik

Ioannis Sourdis

Chalmers, Data- och informationsteknik, Datorteknik

Secure Hardware-Software Architectures for Robust Computing Systems (SHARCS)

Europeiska kommissionen (EU) (EC/H2020/644571), 2015-01-01 -- 2018-12-31.

Energy-efficient Heterogeneous COmputing at exaSCALE (ECOSCALE)

Europeiska kommissionen (EU) (EC/H2020/671632), 2015-10-01 -- 2018-12-31.

Meeting Challenges in Computer Architecture (MECCA)

Europeiska kommissionen (EU) (EC/FP7/340328), 2014-02-01 -- 2019-01-31.

Styrkeområden

Informations- och kommunikationsteknik

Ämneskategorier

Inbäddad systemteknik

Datavetenskap (datalogi)

Datorsystem

Mer information

Senast uppdaterat

2021-12-10