S3Det: a fast object detector for remote sensing images based on artificial to spiking neural network conversion

Abstract

Artificial neural networks (ANNs) have made great strides in the field of remote sensing image object detection. However, low detection efficiency and high power consumption have always been significant bottlenecks in remote sensing. Spiking neural networks (SNNs) process information in the form of sparse spikes, creating the advantage of high energy efficiency for computer vision tasks. However, most studies have focused on simple classification tasks, and only a few researchers have applied SNNs to object detection in natural images. In this study, we consider the parsimonious nature of biological brains and propose a fast ANN-to-SNN conversion method for remote sensing image detection. We establish a fast sparse model for pulse sequence perception based on group sparse features and conduct transform-domain sparse resampling of the original images to enable fast perception of image features and encoded pulse sequences. In addition, to meet accuracy requirements in relevant remote sensing scenarios, we theoretically analyze the transformation error and propose channel self-decaying weighted normalization (CSWN) to eliminate neuron overactivation. We propose S3Det, a remote sensing image object detection model. Our experiments, based on a large publicly available remote sensing dataset, show that S3Det achieves an accuracy performance similar to that of the ANN. Meanwhile, our transformed network is only 24.32% as sparse as the benchmark and consumes only 1.46 W, which is 1/122 of the original algorithm’s power consumption.

1. Introduction

Due to the rapid development of remote sensing technology in recent years, the automatic analysis of massive remote sensing images with all-round intelligent technology has become urgently needed by academia and industry. Object detection, a fundamental task in this domain, finds extensive application in essential areas such as meteorological analysis, surface surveying, and transportation planning (Chen L et al., 2023).

Artificial neural networks (ANNs), represented by convolutional neural networks (CNNs) (LeCun et al., 1998) and Transformers (Vaswani et al., 2017), have developed rapidly over the past decade and continue to energize object detection tasks. However, ANNs often demand substantial computing resources, posing challenges for their deployment on resource-limited devices. In contrast, spiking neural networks (SNNs) (Maass, 1997) emulate the biological structure of the brain, encoding and transmitting information through 0 – 1 spikes, mimicking human biological systems. Because spiking neurons trigger calculations in response to only external stimuli, they have advantages of low power consumption and fast reasoning. At present, SNNs have achieved excellent performance on a variety of hardware platforms, which also validates the feasibility of exploring high-performance and highly efficient brain-inspired intelligence through SNNs.

Despite binary spiking enabling the extreme energy efficiency of SNNs, the complex dynamics involved and the non-differentiable mathematical characteristics of spiking neurons result in a scarcity of training algorithms. Therefore, researchers have tried to find the balance of the scales in the conversion method. ANN-to-SNN conversion (Kim et al., 2020; Li et al., 2022; Hu et al., 2023; Yao et al., 2023) has already yielded promising results on common object detection datasets, such as PASCAL VOC (Everingham et al., 2010) and COCO (Lin et al., 2014). However, as the number of network layers increases, the accuracy error caused by the conversion method further increases. While increasing the firing rate of discrete spikes can significantly mitigate these accuracy errors, it leads to a substantial increase in the time step, undermining the efficiency advantage of SNNs.