An online guided tuning approach to run CNN pipelines on edge devices
Paper i proceeding, 2021

Modern edge and mobile devices are equipped with powerful computing resources. These are often organized as heterogeneous multi-cores, featuring performance-asymmetric core clusters. This raises the question on how to effectively execute the inference pass of convolutional neural networks (CNN) on such devices. Existing CNN implementations on edge devices leverage offline profiling data to determine a better schedule for CNN applications. This approach requires a time consuming phase of generating a performance profile for each type of representative kernel on various core configurations available on the device, coupled with a search space exploration. We propose an online tuning technique which utilizes compile time hints and online profiling data to generate high throughput CNN pipelines. We explore core heterogeneity and compatible core-layer configurations through an online guided search. Unlike exhaustive search, we adopt an evolutionary approach with a guided starting point in order to find the solution. We show that by pruning and navigating through the complex search space using compile time hints, 79% of the tested configurations turn out to be near-optimal candidates for a throughput maximizing pipeline on NVIDIA Jetson TX2 platform.

task parallel runtimes

edge devices

evolutionary algorithm

design space exploration

CNN pipelines

heterogeneous core clusters

task moldability

online tuning


Pirah Noor Soomro

Chalmers, Data- och informationsteknik, Datorteknik

Mustafa Abduljabbar

Chalmers, Data- och informationsteknik, Datorteknik

Jeronimo Castrillon

Technische Universität Dresden

Miquel Pericas

Chalmers, Data- och informationsteknik, Datorteknik

Proceedings of the 18th ACM International Conference on Computing Frontiers 2021, CF 2021

9781450384049 (ISBN)

18th ACM International Conference on Computing Frontiers 2021, CF 2021
Virtual, Online, Italy,

Low-energy toolset for heterogeneous computing (LEGaTO)

Europeiska kommissionen (EU) (EC/H2020/780681), 2018-02-01 -- 2021-01-31.



Inbäddad systemteknik




Mer information

Senast uppdaterat