A learning-based crack defect detection and 3D localization framework for automated fluorescent magnetic particle inspection

• Published in: Expert systems with applications Vol. 214; p. 118966
• Main Authors: Wu, Qiang, Qin, Xunpen, Dong, Kang, Shi, Aixian, Hu, Zeqi
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 15-03-2023
• Subjects: 3D reconstruction; CNN; Crack segmentation; Magnetic Particle Inspection; Crack segmentation; Magnetic Particle Inspection; CNN; 3D reconstruction
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2022.118966

With the rapid development of deep learning, target detection and segmentation in dirty backgrounds have been readily available. Fluorescent magnetic particle inspection (MPI) based on such technology will be a promising alternative for automated crack defect inspection. Most previous studies in MPI have focused only on crack detection. Instead, we frame it as a crack 3D localization problem, since cracks on non-machined surfaces of metal parts need to be polished and re-inspected, which relies on their 3D positions. Although good results were obtained in defect detection, it is still challenging to perform pixel-level segmentation of micro-cracks from the large background to obtain crack 2D pixels for 3D reconstruction. This paper proposes a two-stage convolutional neural network (CNN) method for metal parts crack defect detection and segmentation at the image-pixel level. The first stage detects and crops the potential cracks to a small area, and the second stage can learn the context of cracks in the detected patches. A window-based stereo matching method is then used to find matching pixels of cracks and to map crack image plane points to the 3D world points. We also illustrate the entire system’s model deployment and signaling work to apply these methods. Both computational and experimental results based on the system are presented for validation. The training precision of target detection reaches 96.3%, its average precision reaches 85.4%, and the average precision reaches 98.3% when the Intersection-over-Union (IoU) threshold is 0.5. The Dice score reaches 94% in pixel-level segmentation, and the average precision is 99.3% when the probability threshold is set to 0.5. The corresponding efficiency reaches 19 FPS and 18 FPS, and the mean absolute errors of 3D coordinates of reconstructed crack defects are all within 1 mm in X-, Y- and Z- directions.
•Crack defect detection and 3D location for fluorescent magnetic particle inspection.•A two-stage convolutional neural network with object detection.•Visual system calibration and efficiency 3D reconstruction.•Experiments and results for detection, segmentation, and 3D reconstruction.•Deep learning model deployment, signal communication, computational efficiency.