A digital simulation platform with human-interactive immersive design for navigation, motion, and teleoperated manipulation of work-class remotely operated vehicle

Abstract

Digital simulation of the full operation of a remotely operated vehicle (ROV) is an economically feasible way for algorithm pretesting and operator training prior to the actual underwater tasks, due to the huge difficulties encountered during the underwater test, high equipment cost, and the time-consuming nature of the process. In this paper, a human-interactive digital simulation platform is established for the navigation, motion, and teleoperated manipulation of work-class ROVs, and provides the human operator with the visualized full operation process. Specially, two mechanisms are presented in this platform: one provides the virtual simulation platform for operator training; the other provides real-time visual and force feedback when implementing the actual tasks. Moreover, an open data interface is designed for researchers for pretesting various algorithms before implementing the actual underwater tasks. Additionally, typical underwater scenarios of the ROV, including underwater sediment sampling and pipeline docking tasks, are selected as the case studies for hydrodynamics-based simulation. Human operator can operate the manipulator installed on the ROV via the master manipulator with the visual and force feedback after the ROV is navigated to the desired position. During the full operation, the dynamic windows approach (DWA)-based local navigation algorithm, sliding mode control (SMC) controller, and the teleoperation control framework are implemented to show the effectiveness of the designed platform. Finally, a user study on the ROV operation mode is carried out, and several metrics are designed to evaluate the superiority and accuracy of the digital simulation platform for immersive underwater teleoperation.

1. Introduction

With the development of robotics (Hokayem and Spong, 2006; Liu et al., 2017; Chen et al., 2023), remotely operated vehicles (ROVs) have attracted increasing attention, whereby work-class ROVs can effectively replace human operators during the execution of remote and complex underwater tasks from a safe place onboard (Forbrigger and Pan, 2018; Wang et al., 2020; Lu et al., 2022). Therefore, work-class ROVs are widely used in offshore and deep-sea operations, such as sampling and pipeline docking. The full operation process of a work-class ROV starts with navigation to the designated position (Long et al., 2022). Once the ROV reaches the designated operation area, it transitions from the navigation mode to the operation mode. At this stage, the human operator takes over, and teleoperates the manipulator installed on the ROV to perform precise tasks (Zhang JJ et al., 2018). To adapt to various operational environments and to improve the safety and precision of underwater tasks, tailored ROV navigation, motion control algorithms, and teleoperation control algorithms need to be developed for work-class ROVs. However, there still remain challenges in algorithm development for work-class ROVs, mainly due to the difficulties encountered during the underwater test, high equipment cost, and the time-consuming nature of the process. To address these issues, the establishment of a digital simulation platform becomes a feasible solution for algorithm pretesting and operator training for the full operation process before undertaking the actual tasks (Li et al., 2022).

Due to the complex terrain and the limited light in underwater environments, autonomous navigation and motion control of ROVs can effectively assist human operators in controlling the ROV to reach the desired positions (Kinsey et al., 2014; Zhao et al., 2014; Manzanilla et al., 2019; Tani et al., 2023). The typical method to test the navigation and motion control algorithm is to apply offline simulators, which have high calculation efficiency (Huang et al., 2023). However, it is difficult to support inter