A novel target detection and localization method in indoor environment for mobile robot based on improved YOLOv5

• Published in: Multimedia tools and applications Vol. 82; no. 18; pp. 28643 - 28668
• Main Authors: Qian, Weijie, Hu, Chunhua, Wang, Hanzhao, Lu, Li, Shi, Zefeng
• Format: Journal Article
• Language: English
• Published: New York Springer US 01-07-2023; Springer Nature B.V
• Subjects: Blurring; Cameras; Color imagery; Computer Communication Networks; Computer Science; Coordinates; Data Structures and Information Theory; Datasets; Effectiveness; Grasping (robotics); Image enhancement; Indoor environments; Localization; Localization method; Multimedia Information Systems; Occlusion; Robots; Special Purpose and Application-Based Systems; Target detection; Track 6: Computer Vision for Multimedia Applications; Two dimensional models; YOLOv5; Indoor; Localization; Target detection
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-023-14569-w

Indoor mobile robots, especially those for the elderly and the disabled, are becoming more and more important to improve their quality of life. The strong interest related to this field can be explained by that the robots can help people grasp or carry things. Accurate detection and localization of target in indoor environment is the premise of this task. Aiming to complete this work, a novel indoor target detection and localization method based on improved YOLOv5 is proposed in this paper for indoor mobile robot equipped with KinectV2 camera. First, we made an indoor scene dataset containing 2000 RGB images and 2000 depth images to enhance the robustness of the 2D detection model in the case of image blur, strong and weak illumination and target occlusion. Second, we proposed an improved YOLOv5-S network for indoor 2D target detection and verified its effectiveness from both theoretical and experimental aspects. When tested on our dataset, our improved YOLOv5-S target detection method achieves the mAP@0.5 indicator of 95.9% and the FPS indicator of 65.36. Third, we proposed an improved mean filtering method to process the depth value of the target center point, so as to solve the noise problem of depth image. Finally, we deduced and sorted out the transformation formula of the target center point from the 2D pixel coordinate system to the 3D camera coordinate system, and used the chessboard calibration method to calibrate our KinectV2 camera, so as to realize the 3D localization of the target center point. When conducting localization experiments in the range of 0.5 m–3 m, the MAE indicator of the localization results of our proposed method is only 11.59 mm, which proves the effectiveness of our proposed method.