Reversing the Hands of Time Changing the Mapping From Seeing to Saying

• Published in: Journal of experimental psychology. Learning, memory, and cognition Vol. 37; no. 3; pp. 748 - 756
• Main Authors: Kuchinsky, Stefanie E, Bock, Kathryn, Irwin, David E
• Format: Journal Article
• Language: English
• Published: Washington, DC American Psychological Association 01-05-2011
• Subjects: Attention - physiology; Basic Skills; Biological and medical sciences; Cognitive Mapping; Cognitive Maps; Cognitive Processes; College Students; Evidence; Experimental psychology; Experiments; Eye Movements; Eye Movements - physiology; Fundamental and applied biological sciences. Psychology; Hand; Human; Humans; Language; Language Usage; Linguistics; Motion Perception - physiology; Perception; Production and perception of spoken language; Psychology. Psychoanalysis. Psychiatry; Psychology. Psychophysiology; Reaction Time - physiology; Sensory perception; Speech; Speech - physiology; Students; Time; Time Factors; Universities; Visual Environment; Visual Perception; Visual Tracking; Voice; Word Order; time telling; eyetracking; language production; Human; Language; Vision; Perception; Speech; Cognition; Verbal production; Experimental study; Eye movement
• ISSN: 0278-7393; 1939-1285
• DOI: 10.1037/a0022637

To describe a scene, speakers must map visual information to a linguistic plan. Eye movements capture features of this linkage in a tendency for speakers to fixate referents just before they are mentioned. The current experiment examined whether and how this pattern changes when speakers create atypical mappings. Eye movements were monitored as participants told the time from analog clocks. Half of the participants did this in the usual manner. For the other participants, the denotations of the clock hands were reversed, making the big hand the hour and the little hand the minute. Eye movements revealed that it was not the visual features or configuration of the hands that determined gaze patterns, but rather top-down control from upcoming referring expressions. Differences in eye-voice spans further suggested a process in which scene elements are relationally structured before a linguistic plan is executed. This provides evidence for structural rather than lexical incrementality in planning and supports a "seeing-for-saying" hypothesis in which the visual system is harnessed to the linguistic demands of an upcoming utterance.