Facial Landmark Identification and Data Preparation Can Significantly Improve the Extraction of Newborns' Facial Features

• Published in: 2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG) pp. 1 - 7
• Main Authors: Del Corso, Giulio, Germanese, Danila, Pascali, Maria Antonietta, Bardelli, Serena, Cuttano, Armando, Festante, Fabrizia, Guzzetta, Andrea, Rocchitelli, Lucia, Colantonio, Sara
• Format: Conference Proceeding
• Language: English
• Published: IEEE 27-05-2024
• Subjects: Data mining; Feature extraction; Location awareness; Morphology; Mouth; Pediatrics; Tongue
• ISSN: 2770-8330
• DOI: 10.1109/FG59268.2024.10581971

Automatic extraction of facial feature can provide valuable information on the health of newborns. However, determining an optimal facial features extraction strategy, especially for preterm infants, is a challenging task due to significant differences in facial morphology and frequent pose changes. In this work, we collected video data from 10 newborns (8 preterm, 2 at term, ≤ 4 weeks post term equivalent age), obtaining a novel dataset of over 41, 000 labeled frames (Open Mouth, Closed Mouth, Tongue Protrusion). On the collected images, we applied a strong data preparation procedure (including mouth localization, cropping, and reorientation with models trained on adults), an adaptive image normalization strategy, and a proper data augmentation scheme. Thus, we trained a convolutional classifier with a large number of trainable parameters (i.e., ~1.2 million), coupled with multiple criteria to avoid overspecialization and consequent loss of generalization capability. This approach allows for highly reliable results (accuracy, precision, and recall over 92% on unseen data) and generalizes well to newborns with significantly different characteristics, even without including time-dependent information in the analysis. Therefore, these results prove that proper data preparation can narrow the gap between the classification of neonatal and adult facial features, allowing the integration of methods originally developed for adults into the complex setting of preterm infant analysis.