The role of auditory and visual cues in the interpretation of Mandarin ironic speech

• Published in: Journal of pragmatics Vol. 201; pp. 3 - 14
• Main Authors: Li, Shanpeng, Chen, Aoju, Chen, Ying, Tang, Ping
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 01-11-2022; Elsevier Science Ltd
• Subjects: Audio-visual cues; Compliments; Cues; Facial expressions; Interpretation of Irony; Ironic speech; Irony; Linguistics; Mandarin; Mandarin Chinese; Pragmatics; Prosody; Speech perception; Interpretation of Irony; Mandarin Chinese; Ironic speech; Audio-visual cues
• ISSN: 0378-2166; 1879-1387
• DOI: 10.1016/j.pragma.2022.09.007

Irony is a form of non-literal language. Its decoding may require receivers to use information from both the auditory (e.g., prosody) and visual (e.g., facial expressions) modalities. However, little is known about the role of cues in different modalities in understanding irony. Therefore, the present study examined the relative contributions of auditory and visual cues in the interpretation of Mandarin ironic speech.
Sixty-three native Mandarin speakers participated in a perception experiment in which two types of irony (ironic criticisms and ironic compliments) and their literal counterparts (sincere criticisms and sincere compliments) were presented in audio-only, visual-only, and audio-visual conditions. A two-alternative forced-choice task was adopted to measure the listeners’ interpretation of ironic expressions.
Ironic criticisms and ironic compliments were interpreted best in the visual-only condition as compared to the other conditions. Furthermore, the participants could employ auditory cues alone to accurately interpret ironic criticisms (accuracy significantly above chance), but not ironic compliments.
This study demonstrated a dominant role of visual information in interpreting Mandarin ironic speech, while the role of auditory information is contingent on the type of irony.
•Ironic speech is usually encoded through multiple auditory and visual/facial cues.•The contributions of these cues in interpreting Mandarin ironic speech are unclear.•A multimodal perception experiment was conducted with 63 native Mandarin speakers.•Both types of irony rely on visual cues.•Ironic compliments do not rely on auditory cues.