Title: An improved mask R-CNN example segmentation algorithm based on RGB-D

Authors: Gongfa Li; Boao Li; Du Jiang; Bo Tao; Juntong Yun

Addresses: Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of Education, Wuhan University of Science and Technology, Wuhan, Hubei, China; Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, Hubei, China ' Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of Education, Wuhan University of Science and Technology, Wuhan, Hubei, China ' Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, Hubei, China ' National Demonstration Centre for Experimental Mechanical Education, Wuhan University of Science and Technology, Wuhan, Hubei, China ' Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, Hubei, China

Abstract: Combining the characteristics of RGB images and depth images, we propose a reverse fusion instance segmentation algorithm that effectively combines the advantages of RGB and depth information by fusing high-level semantic features with low-level edge detail features. The algorithm uses RGB and depth information in RGB-D images, extracts features from RGB and depth images separately using a Feature Pyramid Network (FPN) and upsamples the high-level features to the same size as the bottommost features. Subsequently, we apply the inverse fusion method to fuse the high-level features with the low-level features. At the same time, a mask optimisation structure is introduced in the mask branch to achieve RGB-D reverse fusion instance segmentation. Experimental results show that this reverse fusion feature model gives satisfactory performance in RGB-D instance segmentation. On the basis of using ResNet-101 as the backbone network, the average accuracy is improved by 10.6% compared with Mask R-CNN without fusing depth information.

Keywords: depth image; instance segmentation; feature fusion; inverse fusion.

DOI: 10.1504/IJWMC.2024.137861

International Journal of Wireless and Mobile Computing, 2024 Vol.26 No.3, pp.302 - 309

Received: 18 Sep 2023
Accepted: 07 Dec 2023
Published online: 05 Apr 2024 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article