Performance of deep neural networks on low-power IoT devices
Paper in proceeding, 2021

Advances in deep learning have revolutionized machine learning by solving complex tasks such as image, speech, and text recognition. However, training and inference of deep neural networks are resource-intensive. Recently, researchers made efforts to bring inference to IoT edge and sensor devices which have become the prime data sources nowadays. However, running deep neural networks on low-power IoT devices is challenging due to their resource-constraints in memory, compute power, and energy. This paper presents a benchmark to grasp these trade-offs by evaluating three representative deep learning frameworks: uTensor, TF-Lite-Micro, and CMSIS-NN. Our benchmark reveals significant differences and trade-offs for each framework and its tool-chain: (1) We find that uTensor is the most straightforward framework to use, followed by TF-Micro, and then CMSIS-NN. (2) Our evaluation shows large differences in energy, RAM, and Flash footprints. For example, in terms of energy, CMSIS-NN is the most efficient, followed by TF-Micro and then uTensor, each with a significant gap.

Deep Neural Networks




Christos Profentzas

Chalmers, Computer Science and Engineering (Chalmers), Networks and Systems (Chalmers)

Magnus Almgren

Chalmers, Computer Science and Engineering (Chalmers), Networks and Systems (Chalmers)

Olaf Landsiedel

University of Kiel

CPS-IoTBench '21: Proceedings of the Workshop on Benchmarking Cyber-Physical Systems and Internet of Things
Nashville Tennessee, USA,

KIDSAM: Knowledge and information-sharing in digital collaborative projects

VINNOVA, 2018-11-01 -- 2021-11-30.

AgreeOnIT: Lightweight Consensus and Distributed Computing in the Resource-Constrained Internet of Things

Swedish Research Council (VR), 2019-01-01 -- 2022-12-31.

RIOT: Resilient Internet of Things

Swedish Civil Contingencies Agency, 2019-01-01 -- 2023-12-31.

Subject Categories

Computer Engineering

Computer Science

Computer Systems

Areas of Advance

Information and Communication Technology

Driving Forces

Sustainable development



More information

Latest update