Reliability Analysis of Compressed CNNs
Report, 2021

The use of artificial intelligence, Machine Learning and in particular Deep Learning (DL), have recently become a effective and standard de-facto solution for complex problems like image classification, sentiment analysis or natural language processing. In order to address the growing demand of performance of ML applications, research has focused on techniques for compressing the large amount of the parameters required by the Deep Neural Networks (DNN) used in DL. Some of these techniques include parameter pruning, weight-sharing, i.e. clustering of the weights, and parameter quantization. However, reducing the amount of parameters can lower the fault tolerance of DNNs, already sensitive to software and hardware faults caused by, among others, high particles strikes, row hammer or gradient descent attacks, et cetera. In this work we analyze the sensitivity to faults of widely used DNNs, in particular Convolutional Neural Networks (CNN), that have been compressed with the use of pruning, weight clustering and quantization. Our analysis shows that in DNNs that employ all such compression mechanisms, i.e. with their memory footprint reduced up to 86.3x, random single bit faults can result in accuracy drops up to 13.56%.

Machine Learning

CNNs

Caffe

Fault Tolerance

Compression

Author

Stefano Ribes

Chalmers, Computer Science and Engineering (Chalmers), Computer Engineering (Chalmers), Computer Systems

Alirad Malek

Chalmers, Computer Science and Engineering (Chalmers), Computer Engineering (Chalmers), Computer Systems

Pedro Petersen Moura Trancoso

Chalmers, Computer Science and Engineering (Chalmers), Computer Engineering (Chalmers), Computer Systems

Ioannis Sourdis

Chalmers, Computer Science and Engineering (Chalmers), Computer Engineering (Chalmers), Computer Systems

Energy-efficient Heterogeneous COmputing at exaSCALE (ECOSCALE)

European Commission (EC), 2015-10-01 -- 2018-12-31.

Meeting Challenges in Computer Architecture (MECCA)

European Commission (EC), 2014-02-01 -- 2019-01-31.

Secure Hardware-Software Architectures for Robust Computing Systems (SHARCS)

European Commission (EC), 2015-01-01 -- 2018-12-31.

Areas of Advance

Information and Communication Technology

Subject Categories

Embedded Systems

Computer Science

Computer Systems

More information

Latest update

3/29/2021