Visual cues in Mandarin tone perception

Hansjorg Mixdorff, Yu Hu, Denis K. Burnham

    Research output: Chapter in Book / Conference PaperConference Paper

    Abstract

    ![CDATA[This paper presents results concerning the exploitation of visual cues in the perception of Mandarin tones. The lower part of a female speaker's face was recorded on digital video as she uttered 25 sets of syllabic tokens covering the four different tones of Mandarin. Then in a perception study the audio sound track alone, as well an audio plus video condition were presented to native Mandarin speakers who were required to decide which tone they perceived. Audio was presented in various conditions: clear, babble-noise masked at different SNR levels, as well as devoiced and amplitude-modulated noise conditions using LPC resynthesis. In the devoiced and the clear audio conditions, there is little augmentation of audio alone due to the addition of video. However, the addition of visual information did significantly improve perception in the babble-noise masked condition, and this effect increased with decreasing SNR. This outcome suggests that the improvement in noise-masked conditions is not due to additional information in the video per se, but rather to an effect of early integration of acoustic and visual cues facilitating auditory-visual speech perception.]]
    Original languageEnglish
    Title of host publicationInterspeech Lisboa 2005: 9th European Conference on Speech Communication and Technology, September 4-8, 2005
    PublisherCausal Productions
    Number of pages4
    Publication statusPublished - 2005
    EventEuropean Conference on Speech Communication and Technology -
    Duration: 1 Jan 2005 → …

    Publication series

    Name
    ISSN (Print)1018-4074

    Conference

    ConferenceEuropean Conference on Speech Communication and Technology
    Period1/01/05 → …

    Keywords

    • Chinese language
    • speech perception
    • visual perception
    • Mandarin dialects
    • tone (phonetics)

    Fingerprint

    Dive into the research topics of 'Visual cues in Mandarin tone perception'. Together they form a unique fingerprint.

    Cite this