Advanced

Acoustic features of multimodal prominences : Do visual beat gestures affect verbal pitch accent realization?

Ambrazaitis, Gilbert LU and House, David LU (2017) International Conference on Auditory-Visual Speech Processing In Proceedings of The 14th International Conference on Auditory-Visual Speech Processing (AVSP2017)
Abstract (Swedish)
The interplay of verbal and visual prominence cues has attracted recent attention, but previous findings are inconclusive as to whether and how the two modalities are integrated in the production and perception of prominence. In particular, we do not know whether the phonetic realization of pitch accents is influenced by co-speech beat gestures, and previous findings seem to generate different predictions. In this study, we investigate acoustic properties of
prominent words as a function of visual beat gestures in a corpus of read news from Swedish television. The corpus was annotated for head and eyebrow beats as well as sentence-level pitch accents. Four types of prominence cues occurred
particularly frequently in the corpus: (1)... (More)
The interplay of verbal and visual prominence cues has attracted recent attention, but previous findings are inconclusive as to whether and how the two modalities are integrated in the production and perception of prominence. In particular, we do not know whether the phonetic realization of pitch accents is influenced by co-speech beat gestures, and previous findings seem to generate different predictions. In this study, we investigate acoustic properties of
prominent words as a function of visual beat gestures in a corpus of read news from Swedish television. The corpus was annotated for head and eyebrow beats as well as sentence-level pitch accents. Four types of prominence cues occurred
particularly frequently in the corpus: (1) pitch accent only, (2) pitch accent plus head, (3) pitch accent plus head plus eyebrows, and (4) head only. The results show that (4) differs from (1-3) in terms of a smaller pitch excursion and shorter syllable duration. They also reveal significantly larger pitch excursions in (2) than in (1), suggesting that the realization of a pitch accent is to some extent influenced by the presence of visual prominence cues. Results are discussed in terms of the interaction between beat gestures and prosody with a potential functional difference between head and eyebrow beats. (Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
keywords
Audio-visual prosody , Co-speech gestures, News speech, Swedish, Multimodality
in
Proceedings of The 14th International Conference on Auditory-Visual Speech Processing (AVSP2017)
editor
Ouni, Slim; Davis, Chris; Jesse, Alexandra; Beskow, Jonas; ; ; and
pages
6 pages
publisher
KTH
conference name
International Conference on Auditory-Visual Speech Processing
ISSN
2308-975X
language
English
LU publication?
yes
id
f67e2abb-c856-4a26-85af-dcc514180138
date added to LUP
2017-09-02 22:26:51
date last changed
2017-09-21 08:51:26
@inproceedings{f67e2abb-c856-4a26-85af-dcc514180138,
  abstract     = {The interplay of verbal and visual prominence cues has attracted recent attention, but previous findings are inconclusive as to whether and how the two modalities are integrated in the production and perception of prominence. In particular, we do not know whether the phonetic realization of pitch accents is influenced by co-speech beat gestures, and previous findings seem to generate different predictions. In this study, we investigate acoustic properties of<br/>prominent words as a function of visual beat gestures in a corpus of read news from Swedish television. The corpus was annotated for head and eyebrow beats as well as sentence-level pitch accents. Four types of prominence cues occurred<br/>particularly frequently in the corpus: (1) pitch accent only, (2) pitch accent plus head, (3) pitch accent plus head plus eyebrows, and (4) head only. The results show that (4) differs from (1-3) in terms of a smaller pitch excursion and shorter syllable duration. They also reveal significantly larger pitch excursions in (2) than in (1), suggesting that the realization of a pitch accent is to some extent influenced by the presence of visual prominence cues. Results are discussed in terms of the interaction between beat gestures and prosody with a potential functional difference between head and eyebrow beats.},
  author       = {Ambrazaitis, Gilbert and House, David},
  booktitle    = {Proceedings of The 14th International Conference on Auditory-Visual Speech Processing (AVSP2017)},
  editor       = {Ouni, Slim and Davis, Chris and Jesse, Alexandra and Beskow, Jonas},
  issn         = {2308-975X},
  keyword      = {Audio-visual prosody ,Co-speech gestures,News speech,Swedish,Multimodality},
  language     = {eng},
  pages        = {6},
  publisher    = {KTH},
  title        = {Acoustic features of multimodal prominences : Do visual beat gestures affect verbal pitch accent realization?},
  year         = {2017},
}