The effect of seeing the interlocutor on auditory and visual speech production in noise

Talkers modify their speech production in noisy environments partly as a reflex but also as an intentional communicative strategy to facilitate the transmission of the speech signal to the interlocutor. Previous studies have shown that talkers can adapt both auditory and visual elements of speech produced in noise. The current study examined whether talkers adapt their auditory and visual speech production strategy as a function of their communicative setting. Participants completed an interactive communication game in various quiet and in noise conditions with/without being able to see their interlocutor. We found that the energy of talkers’ speech modifications was significantly lower in conditions when interlocutors could see each other relative to conditions where they could not. Further, talkers increased the saliency of their visual speech production (measured as lip-area and lip-width) in noisy conditions for face-to-face communication. In a set of perception studies, using the speech materials from the production study as stimuli, we also demonstrated that the shift in speech production strategy across the face-to-face and non-visual communicative conditions corresponded with changes in the auditory and auditory–visual intelligibility of the speech signal produced. The results suggest that talkers actively monitor their environment and are able to adopt appropriate speech production strategies for efficient and effective communication in adverse conditions.