RAINBOW: Multi-Dimensional Hardware-Software Co-Design for DL Accelerator On-Chip Memory

Paper in proceeding, 2023

Deep Learning (DL) is developing at an extremely fast pace. The increased number of applications, optimizations and hardware devices available, results in a multi-dimensional design space where the best performance is achieved with a detailed analysis of the hardware-software co-design process. Furthermore, the high demands for memory and the off-chip latency cost result in the on-chip memory becoming critical for achieving high performance and efficiency. In this work, we propose RAINBOW, a tool to assist in the hardware-software co-design for DL accelerators' on-chip memory. The purpose is to help the design and/or deployment of a DL model to a dedicated accelerator. RAINBOW generates different analyses results and feeds them to the optimizers. The result is a heterogeneous execution plan combining different approaches and techniques depending on the dynamic requirements and constraints. In our analysis, we concluded that given the opportunity, RAINBOW'S heterogeneous plans are able to reduce the DRAM accesses to approximately half when compared to homogeneous plans.