Measuring Representation of Race, Gender, and Age in Children's Books: Face Detection and Feature Classification in Illustrated Images

• Published in: 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) pp. 3371 - 3380
• Main Authors: Szasz, Teodora, Harrison, Emileigh, Liu, Ping-Jung, Lin, Ping-Chang, Birali Runesha, Hakizumwami, Adukia, Anjali
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01-01-2022
• Subjects: Data models; Deep learning; Face detection; Feature extraction; Object Detection/Recognition/Categorization Biometrics -> Face Processing; Deep Learning; Explainable AI; Fairness; Accountability; Privacy and Ethics in Vision; Multimedia Applications; Visual Reasoning; Analysis and Understanding; Predictive models; Training data; Visualization
• ISSN: 2642-9381
• DOI: 10.1109/WACV51458.2022.00343

Images in children's books convey messages about society and the roles that people play in it. Understanding these messages requires systematic measurement of who is represented. Computer vision face detection tools can provide such measurements; however, state-of-the-art face detection models were trained with photographs, and 80% of images in children's books are illustrated; thus existing methods both misclassify and miss classifying many faces. In this paper, we introduce a new approach to analyze images using AI tools, resulting in data that can assess representation of race, gender, and age in both illustrations and photographs in children's books. We make four primary contributions to the fields of deep learning and social sciences: (1) We curate an original face detection data set (IllusFace 1.0) by manually labeling 5,403 illustrated faces with bounding boxes. (2) We train two AutoML-based face detection models for illustrations: (i) using IllusFace 1.0 (FDAI); (ii) using iCartoon, a publicly available data set (FDAI iC), each optimized for illustrated images, detecting 2.5 times more faces in our testing data than the established face detector using Google Vision (FDGV). (3) We curate a data set of the race, gender, and age of 980 faces manually labeled by three different raters (CBFeatures 1.0). (4) We train an AutoML feature classification model (FCA) using CBFeatures 1.0. We compare FCA with the performance of another AutoML model that we trained on UTKFace, a public data set (FCA UTK) and of an established model using FairFace (FCF). Finally, we examine distributions of character identities over the last century across the models. We find that FCA is 34% more accurate than FCF in its race predictions. These contributions provide tools to educators, caregivers, and curriculum developers to assess the representation contained in children's content.