SAPER-AI accelerator: a systolic array-based power-efficient reconfigurable AI accelerator

Abstract

Deep learning (DL) accelerators are critical for handling the growing computational demands of modern neural networks. Systolic array (SA)-based accelerators consist of a 2D mesh of processing elements (PEs) working cooperatively to accelerate matrix multiplication. The power efficiency of such accelerators is of primary importance, especially considering the edge AI regime. This work presents the SAPER-AI accelerator, an SA accelerator with power intent specified via a unified power format representation in a simplified manner with negligible microarchitectural optimization effort. Our proposed accelerator switches off rows and columns of PEs in a coarse-grained manner, thus leading to SA microarchitecture complying with the varying computational requirements of modern DL workloads. Our analysis demonstrates enhanced power efficiency ranging between 10% and 25% for the best case 32×32 and 64×64 SA designs, respectively. Additionally, the power delay product (PDP) exhibits a progressive improvement of around 6% for larger SA sizes. Moreover, a performance comparison between the MobileNet and ResNet50 models indicates generally better SA performance for the ResNet50 workload. This is due to the more regular convolutions portrayed by ResNet50 that are more favored by SAs, with the performance gap widening as the SA size increases.

1. Introduction

The meteoric rise in deep learning (DL)-based solutions has revolutionized a variety of domains such as image processing, pattern recognition, and transportation. To make full use of the benefits that such DL models can accord, huge operational costs need to be incurred due to the computational complexities accompanying such models (Yüzügüler et al., 2023). Therefore, specialized DL accelerators are necessary to offer enhanced computational prowess while still being energy efficient. Several such accelerators have been proposed on both sides of the DL continuum, i.e., the cloud (Bobda et al., 2022; Li et al., 2023) and the edge (Seshadri et al., 2022; Loh et al., 2025).

Deep neural networks (DNNs) incorporate matrix multiplication as the primary primitive, which in turn, fortunately offers a lot of parallelism, and its acceleration is imperative to achieve tremendous processing demands of the DL workloads (Muslim et al., 2024). Several general matrix multiplication (GEMM) accelerators found in the literature are based on systolic arrays (SAs) (Jouppi et al., 2017; Song et al., 2019; Lai and Zhang, 2024). SA is a two-dimensional mesh of processing elements (PEs) with each PE being fed inputs from the left and top sides. Each PE performs a multiplication and accumulation (MAC) operation every clock cycle, and the partial result and the input are fed to the neighboring PE via pipeline registers. In this way, the PEs collaborate to offer enhanced parallelism in processing data efficiently and accelerate the DNN computation. A typical SA architecture is depicted in Fig. 1.

One of the main factors necessitating the usage of customized architectures for DL acceleration is the resulting improvement in the energy efficiency offered by the accelerators. Such customized hardware must comply with the tight power budget, which is important both at the cloud and the edge. However, the power efficiency of such designs at the edge due to the constrained power availability is of even greater importance (Kim et al., 2020). Thus, a lot of research has been done in recent years to improve the power efficiency of such hardware accelerators. The research found in the literature mainly targets complex microarchitectural optimizations, e.g., by reducing expensive memory access optimizations or by quantization of the networks leading to approximate computing, thus leading to enhanced energy savings (Chen YJ et al., 2016; Moons et al., 2016). Moreover, the modern DNNs are accompanied by increased sparsity, much like their biological counterparts and can generalize well when compared to their denser versions (Hoefler et al., 2021; Guo et al., 2024).