End-to-end object detection using a query-selection encoder with hierarchical feature-aware attention

Abstract

End-to-end object detection methods have attracted extensive interest recently since they alleviate the need for complicated human-designed components and simplify the detection pipeline. However, these methods suffer from slower training convergence and inferior detection performance compared to conventional detectors, as their feature fusion and selection processes are constrained by insufficient positive supervision. To address this issue, we introduce a novel query-selection encoder (QSE) designed for end-to-end object detectors to improve the training convergence speed and detection accuracy. QSE is composed of multiple encoder layers stacked on top of the backbone. A lightweight head network is added after each encoder layer to continuously optimize features in a cascading manner, providing more positive supervision for efficient training. Additionally, a hierarchical feature-aware attention (HFA) mechanism is incorporated in each encoder layer, including in- and cross-level feature attention, to enhance the interaction between features from different levels. HFA can effectively suppress similar feature representations and highlight discriminative ones, thereby accelerating the feature selection process. Our method is highly versatile in accommodating both CNN- and Transformer-based detectors. Extensive experiments were conducted on the popular benchmark datasets MS COCO, CrowdHuman, and PASCAL VOC to demonstrate the effectiveness of our method. The results showed that CNN- and Transformer-based detectors using QSE can achieve better end-to-end performance within fewer training epochs.

1. Introduction

Object detection is a crucial task in computer vision, aiming to find targets of interest in images by circling bounding boxes and predicting categories (Pu et al., 2021; Qin et al., 2023; Wang CY et al., 2023). Traditional object detectors built by convolutional neural networks (CNNs) adopt a dense prediction paradigm, which perform classification and localization tasks based on pre-defined densely tiled bounding boxes (Girshick, 2015; Ren et al., 2015; Lin TY et al., 2017) or grid points in the two-dimensional (2D) image plane (Tian et al., 2019; Zhou et al., 2019). One-to-many label assignments are the core scheme of these methods, in which each ground-truth box is assigned to multiple predictions of detectors as the supervised target. Despite their excellent performance, these detectors rely heavily on hand-designed components, i.e., non-maximum suppression (NMS), to remove duplicated predictions during inference, which introduces additional hyper-parameters to tune and thus causes sub-optimal performance in dense scenes (Li S et al., 2023; Zhang SL et al., 2023).

To achieve a more flexible end-to-end detection, DEtection TRansformer (DETR) (Carion et al., 2020) was proposed, viewing object detection as a set prediction problem and introducing a Transformer encoder–decoder architecture. Adopting a sparse prediction paradigm, DETR reasons about the global image context and outputs the final predictions by using a small set of learnable object queries. The one-to-one label assignment plays a crucial role in DETR for conducting end-to-end detection, where each ground-truth box is assigned only one prediction. Hence, DETR outputs only a single prediction for each object during inference and NMS is no longer necessary. This approach has encouraged many subsequent improvements (Yao et al., 2021; Wang YN et al., 2022; Li F et al., 2023). In addition, POTO (Wang JF et al., 2021) and OneNet (Sun PZ et al., 2021b) attempt to adopt one-to-one label assignment in CNN-based detectors to realize end-to-end detection. However, these methods suffer from extremely slow training convergence and relatively low performance on small objects. The core reason for this problem is a conflict between one-to-one label assignment and sufficient positive supervision (Jia et al., 2023; Hou et al., 2024). During the training process, the one-to-one matching scheme assigns a single positive prediction to each ground-truth box, which leads to negative predictions dominating most of the loss function, causing insufficient positive supervision. Therefore, more training iterations are required for convergence. To alleviate this issue, previous studies introduced additional training-only architectures, such as query denoising (Li F et al., 2022), mul...