AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild

• Published in: IEEE transactions on affective computing Vol. 10; no. 1; pp. 18 - 31
• Main Authors: Mollahosseini, Ali, Hasani, Behzad, Mahoor, Mohammad H.
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01-01-2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Affective computing; Affective computing in the wild; Annotations; Arousal; Artificial neural networks; Automation; Computational modeling; Computer vision; continuous dimensional space; Emotions; Face; Face recognition; facial expressions; Image classification; Machine learning; Magnetic heads; Neural networks; Search engines; valence
• ISSN: 1949-3045; 1949-3045
• DOI: 10.1109/TAFFC.2017.2740923

Automated affective computing in the wild setting is a challenging problem in computer vision. Existing annotated databases of facial expressions in the wild are small and mostly cover discrete emotions (aka the categorical model). There are very limited annotated facial databases for affective computing in the continuous dimensional model (e.g., valence and arousal). To meet this need, we collected, annotated, and prepared for public distribution a new database of facial emotions in the wild (called AffectNet). AffectNet contains more than 1,000,000 facial images from the Internet by querying three major search engines using 1,250 emotion related keywords in six different languages. About half of the retrieved images were manually annotated for the presence of seven discrete facial expressions and the intensity of valence and arousal. AffectNet is by far the largest database of facial expression, valence, and arousal in the wild enabling research in automated facial expression recognition in two different emotion models. Two baseline deep neural networks are used to classify images in the categorical model and predict the intensity of valence and arousal. Various evaluation metrics show that our deep neural network baselines can perform better than conventional machine learning methods and off-the-shelf facial expression recognition systems.