Defect classification for specular surfaces based on deflectometry and multi-modal fusion network

• Published in: Optics and lasers in engineering Vol. 163; p. 107488
• Main Authors: Guan, Jingtian, Fei, Jingjing, Li, Wei, Jiang, Xiaoke, Wu, Liwei, Liu, Yakun, Xi, Juntong
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01-04-2023
• Subjects: Dataset; Deep learning; Defect classification; Deflectometry; Specular surface; Deep learning; Defect classification; Deflectometry; Dataset; Specular surface
• ISSN: 0143-8166; 1873-0302
• DOI: 10.1016/j.optlaseng.2023.107488

•Deep-learning-based method is introduced for the automated defect classification of specular surfaces.•Multi-modal feature fusion network is proposed to further extract the image features from different modalities and increase the performance of defect classification.•Both geometrical and textural defects can be correctly classified.•The benchmark defect dataset for specular surfaces named SpecularDefect9 is generated and released.
Automated defect inspection for specular surfaces is still a challenge in the manufacturing industry because of their specular reflection property. Deflectometry provides surface information based on the captured fringe patterns through the reflection of the specular surfaces and has been widely applied in defect detection for specular surfaces. Conventional methods combined deflectometry with machine learning approaches, but the hand-crafted features need to be defined for each specific task. Combined with the deep neural network, the input images are obtained from deflectometry, and the network completes the identification of the defects. Nevertheless, conventional deep-learning-based defect inspection methods approached the problem as a binary classification, or only certain obvious defects can be correctly classified. In this study, we generated and released, for the first time, to the best of our knowledge, the benchmark dataset named SpecularDefect9 with various defects for specular surfaces, and the classification accuracy of some kinds of defects may be low with only one kind of input image. To classify all kinds of defects accurately, the proposed method applied the light intensity contrast map combined with the original captured fringe pattern as the input of the network, and a fusion network was introduced to extract features from multi-modal inputs. Experimental results based on the released benchmark dataset verified the effectiveness and robustness of the proposed multi-modal defect classification method.