Automatized analysis of children's exposure to child-directed speech in reschool settings: Validation and application

• Published in: PloS one Vol. 15; no. 11; p. e0242511
• Main Authors: Gonzalez Villasanti, Hugo, Justice, Laura M, Chaparro-Moreno, Leidy Johana, Lin, Tzu-Jung, Purtell, Kelly
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 25-11-2020; Public Library of Science (PLoS)
• Subjects: Adult; Adults; Automated Facial Recognition - methods; Automatic speech recognition; Biology and Life Sciences; Child, Preschool - education; Child-directed speech; Children; Children & youth; Classifiers; Classroom communication; Classroom techniques; Classrooms; Cloud Computing; Computer vision; Decoding; Educational aspects; Educational psychology; Engineering and Technology; Error analysis; Exposure; Facial Expression; Feature extraction; Female; Humans; Interpersonal Relations; Language; Language acquisition; Language Development; Linguistics; Machine Learning; Medicine and Health Sciences; Methods; Peer Group; People and Places; Phonetics; Preschool children; Psychological aspects; Researchers; Scaling (Speech recognition); Sensory integration; Social Sciences; Source code; Speech; Speech Perception; Speech recognition; Speech Recognition Software; Speeches; Teaching; Transcription; Voice recognition; United States; United States > US; Ohio
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0242511

The present study explored whether a tool for automatic detection and recognition of interactions and child-directed speech (CDS) in preschool classrooms could be developed, validated, and applied to non-coded video recordings representing children's classroom experiences. Using first-person video recordings collected by 13 preschool children during a morning in their classrooms, we extracted high-level audiovisual features from recordings using automatic speech recognition and computer vision services from a cloud computing provider. Using manual coding for interactions and transcriptions of CDS as reference, we trained and tested supervised classifiers and linear mappings to measure five variables of interest. We show that the supervised classifiers trained with speech activity, proximity, and high-level facial features achieve adequate accuracy in detecting interactions. Furthermore, in combination with an automatic speech recognition service, the supervised classifier achieved error rates for CDS measures that are in line with other open-source automatic decoding tools in early childhood settings. Finally, we demonstrate our tool's applicability by using it to automatically code and transcribe children's interactions and CDS exposure vertically within a classroom day (morning to afternoon) and horizontally over time (fall to winter). Developing and scaling tools for automatized capture of children's interactions with others in the preschool classroom, as well as exposure to CDS, may revolutionize scientific efforts to identify precise mechanisms that foster young children's language development.