Brain responses to audiovisual speech mismatch in infants are associated with individual differences in looking behaviour

• Published in: The European journal of neuroscience Vol. 38; no. 9; pp. 3363 - 3369
• Main Authors: Kushnerenko, Elena, Tomalski, Przemyslaw, Ballieux, Haiko, Ribeiro, Helena, Potton, Anita, Axelsson, Emma L., Murphy, Elizabeth, Moore, Derek G.
• Format: Journal Article
• Language: English
• Published: Oxford Blackwell Publishing Ltd 01-11-2013; Blackwell
• Subjects: Age Factors; audiovisual integration; audiovisual mismatch response; Biological and medical sciences; Brain - growth & development; Brain - physiology; event-related potentials (ERPs); Evoked Potentials; Eye and associated structures. Visual pathways and centers. Vision; Eye Movements; eye-tracking; Female; Fundamental and applied biological sciences. Psychology; Humans; Infant; Male; McGurk illusion; Psychomotor Performance; Speech Perception; Vertebrates: nervous system and sense organs; Visual Perception; Central nervous system; Electrophysiology; Audiovisual integration; Encephalon; Eye; Visual system; McGurk illusion; audiovisual mismatch response; Speech; eye-tracking; Behavior; Event evoked potential; event-related potentials (ERPs); audiovisual integration
• ISSN: 0953-816X; 1460-9568; 1460-9568
• DOI: 10.1111/ejn.12317

Research on audiovisual speech integration has reported high levels of individual variability, especially among young infants. In the present study we tested the hypothesis that this variability results from individual differences in the maturation of audiovisual speech processing during infancy. A developmental shift in selective attention to audiovisual speech has been demonstrated between 6 and 9 months with an increase in the time spent looking to articulating mouths as compared to eyes (Lewkowicz & Hansen‐Tift. (2012) Proc. Natl Acad. Sci. USA, 109, 1431–1436; Tomalski et al. (2012) Eur. J. Dev. Psychol., 1–14). In the present study we tested whether these changes in behavioural maturational level are associated with differences in brain responses to audiovisual speech across this age range. We measured high‐density event‐related potentials (ERPs) in response to videos of audiovisually matching and mismatched syllables /ba/ and /ga/, and subsequently examined visual scanning of the same stimuli with eye‐tracking. There were no clear age‐specific changes in ERPs, but the amplitude of audiovisual mismatch response (AVMMR) to the combination of visual /ba/ and auditory /ga/ was strongly negatively associated with looking time to the mouth in the same condition. These results have significant implications for our understanding of individual differences in neural signatures of audiovisual speech processing in infants, suggesting that they are not strictly related to chronological age but instead associated with the maturation of looking behaviour, and develop at individual rates in the second half of the first year of life.
The present study demonstrates the importance of combining electrophysiological (high‐density ERPs) and behavioural (eye‐tracking) measures in infancy research. The results have significant implications for understanding differences in brain responses rates of maturation in infants, demonstrating that individual differences in neural signatures of audiovisual integration are related to the behavioural level of maturity of audiovisual processing rather than to chronological age.