Contour and region harmonic features for sub-local facial expression recognition

• Published in: Journal of visual communication and image representation Vol. 73; p. 102949
• Main Authors: Raza Shahid, Ali, Khan, Sheheryar, Yan, Hong
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01-11-2020
• Subjects: Contour description; Facial expression recognition; Local facial shape harmonics; Region description; Local facial shape harmonics; Facial expression recognition; Contour description; Region description
• ISSN: 1047-3203; 1095-9076
• DOI: 10.1016/j.jvcir.2020.102949

•An automatic sub-local facial feature with contour and regions for FER.•Facial harmonics in terms of contour and region descriptors and their combination.•Descriptors with new definitions and strategies for geometrical FER.•Comprehensive evaluation and comparison to state of the art three public datasets.•Simulation and results demonstrate the effectiveness of LFSH method.
Expression recognition relies on intensity, edges, and geometry that overlooks the actual shape curvatures of facial regions. This paper presents a novel two-stage approach to distinguish seven expressions on the basis of eleven different facial areas. The combination of contour and region harmonics is used to develop the interrelationship of sub-local areas in the human face for expression recognition. We applied a multi-class support vector machine (SVM) with subject dependent k-fold cross-validation to classify the human emotions into expressions. We tested our proposed method on three public facial expression datasets for sub-local regions in human face and achieved 94.90%, 93.43%, and 92.57% recognition rate for the CK+, CFEE, and MUG datasets respectively. Experiments show that the contour and region harmonics have high classification power and can be computed efficiently. Our method provides higher accuracy, less computing time, and less memory space than existing techniques, including deep learning.