An online guided tuning approach to run CNN pipelines on edge devices
Paper in proceeding, 2021

Modern edge and mobile devices are equipped with powerful computing resources. These are often organized as heterogeneous multi-cores, featuring performance-asymmetric core clusters. This raises the question on how to effectively execute the inference pass of convolutional neural networks (CNN) on such devices. Existing CNN implementations on edge devices leverage offline profiling data to determine a better schedule for CNN applications. This approach requires a time consuming phase of generating a performance profile for each type of representative kernel on various core configurations available on the device, coupled with a search space exploration. We propose an online tuning technique which utilizes compile time hints and online profiling data to generate high throughput CNN pipelines. We explore core heterogeneity and compatible core-layer configurations through an online guided search. Unlike exhaustive search, we adopt an evolutionary approach with a guided starting point in order to find the solution. We show that by pruning and navigating through the complex search space using compile time hints, 79% of the tested configurations turn out to be near-optimal candidates for a throughput maximizing pipeline on NVIDIA Jetson TX2 platform.

task parallel runtimes

edge devices

evolutionary algorithm

design space exploration

CNN pipelines

heterogeneous core clusters

task moldability

online tuning

Author

Pirah Noor Soomro

Chalmers, Computer Science and Engineering (Chalmers), Computer Engineering (Chalmers)

Mustafa Abduljabbar

Chalmers, Computer Science and Engineering (Chalmers), Computer Engineering (Chalmers)

Jeronimo Castrillon

Technische Universität Dresden

Miquel Pericas

Chalmers, Computer Science and Engineering (Chalmers), Computer Engineering (Chalmers)

Proceedings of the 18th ACM International Conference on Computing Frontiers 2021, CF 2021

45-53
9781450384049 (ISBN)

18th ACM International Conference on Computing Frontiers 2021, CF 2021
Virtual, Online, Italy,

Low-energy toolset for heterogeneous computing (LEGaTO)

European Commission (EC) (EC/H2020/780681), 2018-02-01 -- 2021-01-31.

Subject Categories

Computer Engineering

Embedded Systems

Computer Systems

DOI

10.1145/3457388.3458662

More information

Latest update

2/25/2022