An end-to-end automatic methodology to accelerate the accuracy evaluation of deep neural networks under hardware transient faults

Abstract

Hardware transient faults are proven to have a significant impact on deep neural networks (DNNs), whose safety-critical misclassification (SCM) in autonomous vehicles, healthcare, and space applications is increased up to four times. However, the inaccuracy evaluation using accurate fault injection is time-consuming and requires several hours and even a couple of days on a complete simulation platform. To accelerate the evaluation of hardware transient faults on DNNs, we design a unified and end-to-end automatic methodology, A-Mean, using the silent data corruption (SDC) rate of basic operations (such as convolution, addition, multiply, ReLU, and max-pooling) and a static two-level mean calculation mechanism to rapidly compute the overall SDC rate, for estimating the general classification metric accuracy and application-specific metric SCM. More importantly, a max-policy is used to determine the SDC boundary of non-sequential structures in DNNs. Then, the worst-case scheme is used to further calculate the enlarged SCM and halved accuracy under transient faults, via merging the static results of SDC with the original data from one-time dynamic fault-free execution. Furthermore, all of the steps mentioned above have been implemented automatically, so that this easy-to-use automatic tool can be employed for prompt evaluation of transient faults on diverse DNNs. Meanwhile, a novel metric "fault sensitivity" is defined to characterize the variation of transient fault-induced higher SCM and lower accuracy. The comparative results with a state-of-the-art fault injection method TensorFI+ on five DNN models and four datasets show that our proposed estimation method A-Mean achieves up to 922.80 times speedup, with just 4.20% SCM loss and 0.77% accuracy loss on average. The artifact of A-Mean is publicly available at https://github.com/breatrice321/A-Mean.

1. Introduction

Transient faults primarily result from inherent circuit malfunctions, such as external radiation and internal electrical interference. These transient faults often occur randomly and cause performance loss or area/power overhead for detection and mitigation, so that some ignored faults have the potential to corrupt the program or lead to incorrect data. However, in safety-sensitive applications, such as autonomous vehicles, healthcare, and space applications, transient faults could lead to immeasurable loss of life and property. In recent years, deep neural networks (DNNs) have been adopted in safety-critical applications due to their remarkable problem-solving capabilities. However, these safety-critical applications necessitate dependable, robust, and efficient support from popular DNNs. Consequently, it is important to assess the accuracy of various DNN models under transient faults to guarantee their acceptable prediction quality.

Two categories of mainstream approaches are usually employed to evaluate the impact of faults on DNNs: (1) Fault injection involves injecting faults into DNNs intentionally many times so that the impacts can be quantified by the ratio of the number of injections with observed wrong results to the total number of injections, such as CAFI, saca-FI, TensorFI, and TensorFI+. (2) Fault-free analysis uses a few fault-free simulations and simple analytical models for fast evaluation of fault impacts on DNNs, such as SERN, APPRAISER, DeepVigor, and saca-AVF. The former is accurate but time-consuming, while the latter is fast but inaccurate. The DNN-driven safety-critical applications require not only high reliability for guaranteed safety but also high performance for real-time decisions. Therefore, fast fault-free analysis is preferred for safety-critical applications. However, how to improve the evaluation accuracy of the impacts of transient faults on DNNs with the advantage of high speed is a challenge.