A ground-based dataset and diffusion model for on-orbit low-light image enhancement

Abstract

On-orbit service is important for maintaining the sustainability of the space environment. A space-based visible camera is an economical and lightweight sensor for situational awareness during on-orbit service. However, it can be easily affected by the low illumination environment. Recently, deep learning has achieved remarkable success in image enhancement of natural images, but it is seldom applied in space due to the data bottleneck. In this study, we first propose a dataset of BeiDou navigation satellites for on-orbit low-light image enhancement (LLIE). In the automatic data collection scheme, we focus on reducing the domain gap and improving the diversity of the dataset. We collect hardware-in-the-loop images based on a robotic simulation testbed imitating space lighting conditions. To evenly sample poses of different orientations and distances without collision, we propose a collision-free workspace and pose-stratified sampling. Subsequently, we develop a novel diffusion model. To enhance the image contrast without over-exposure and blurred details, we design fused attention guidance to highlight the structure and the dark region. Finally, a comparison of our method with previous methods indicates that our method has better on-orbit LLIE performance.

1. Introduction

Due to the high frequency (HF) of space activities, the space environment has been seriously degraded as a by-product of these activities. As of February 2022, more than 25,000 space objects have been identified, including retired satellites, spacecraft, rocket bodies, and debris (Cowardin and Miller, 2022), which gravely threaten both the existing and newly launched spacecraft (Ledkov and Aslanov, 2022). Hence, to ensure the long-term sustainability of the space environment, clean up existing space debris, and extend the lifespan of operational spacecraft, active debris removal (ADR) and on-orbit servicing (OOS) have emerged as popular research areas (Poozhiyil et al., 2023). The space robotic arm plays a vital role in performing various tasks associated with ADR and OOS, including capturing, docking, repairing, and refurbishing, as demonstrated in Fig. 1a. To realize safe and efficient robotic control, reliable sensors and data analysis are required to provide better space situational awareness (SSA), especially when manipulating unknown targets (Harris et al., 2021; Civardi et al., 2024). A space-based visible camera is one of the most essential sensors because of its advantages: it is lightweight in mass, compact in size, economical in power, and informative in data (Diao et al., 2011). However, the variable illumination in space can seriously affect the quality of the captured images, especially when the satellite is in the Earth's shadow, as shown in Figs. 1b and 1c. The captured images can be extremely invisible, making it difficult to extract key features. To recover the buried details and improve data usability for downstream tasks and the surveillance efficiency of SSA, we propose a novel framework to solve the problem of on-orbit low-light image enhancement (LLIE).

Image enhancement methods based on classical theory (Rahman et al., 1996; Guo et al., 2017; Ying et al., 2017; Li MD et al., 2018; Ren et al., 2018) have been widely used to improve image contrast. Unfortunately, these methods are suitable only for situations in which images already contain a good representation of the scene content. For extremely dark and noisy images, they are likely to cause severe noise amplification after enhancement. Deep learning (DL) methods have been applied to solve problems in various industries, demonstrating significant advancements (Li CY et al., 2022; Xu et al., 2023, 2024). DL-based methods for LLIE can be classified as one-way neural networks (Wei et al., 2018; Li N and Zhang, 2019; Xu et al., 2021; Dang et al., 2023; Huang et al., 2023; Shi et al., 2024; Yan et al., 2024) and generative adversarial networks.