Visual prosody and speech intelligibility : head movement improves auditory speech perception

Kevin G. Munhall, Jeffrey A. Jones, Daniel E. Callan, Takaaki Kuratate, Eric Vatikiotis-Bateson

    Research output: Contribution to journalArticle

    406 Citations (Scopus)

    Abstract

    People naturally move their heads when they speak, and our study shows that this rhythmic head motion conveys linguistic information. Three-dimensional head and face motion and the acoustics of a talker producing Japanese sentences were recorded and analyzed. The head movement correlated strongly with the pitch (fundamental frequency) and amplitude of the talker's voice. In a perception study, Japanese subjects viewed realistic talking-head animations based on these movement recordings in a speech-in-noise task. The animations allowed the head motion to be manipulated without changing other characteristics of the visual or acoustic speech. Subjects correctly identified more syllables when natural head motion was present in the animation than when it was eliminated ordistorted. These results suggest that nonverbal gestures suchas head movements play a more direct role in the perception of speech than previously known.
    Original languageEnglish
    Number of pages5
    JournalPsychological Science
    DOIs
    Publication statusPublished - 2004

    Keywords

    • Japanese language
    • head
    • movements
    • nonverbal communication
    • speech perceptions

    Fingerprint

    Dive into the research topics of 'Visual prosody and speech intelligibility : head movement improves auditory speech perception'. Together they form a unique fingerprint.

    Cite this