Advanced

Audio-visual speech comprehension in noise with real and virtual speakers

Nirme, Jens LU ; Sahlén, Birgitta LU ; Lyberg Åhlander, Viveka LU ; Brännström, Jonas LU and Haake, Magnus LU (2020) In Speech Communication 116. p.44-55
Abstract

This paper presents a study where a 3D motion-capture animated ‘virtual speaker’ is compared to a video of a real speaker with regards to how it facilitates children's speech comprehension of narratives in background multitalker babble noise. As secondary measures, children self-assess the listening- and attentional effort demanded by the task, and associates words describing positive or negative social traits to the speaker. The results show that the virtual speaker, despite being associated with more negative social traits, facilitates speech comprehension in babble noise compared to a voice-only presentation but that the effect requires some adaptation. We also found the virtual speaker to be at least as facilitating as the video. We... (More)

This paper presents a study where a 3D motion-capture animated ‘virtual speaker’ is compared to a video of a real speaker with regards to how it facilitates children's speech comprehension of narratives in background multitalker babble noise. As secondary measures, children self-assess the listening- and attentional effort demanded by the task, and associates words describing positive or negative social traits to the speaker. The results show that the virtual speaker, despite being associated with more negative social traits, facilitates speech comprehension in babble noise compared to a voice-only presentation but that the effect requires some adaptation. We also found the virtual speaker to be at least as facilitating as the video. We interpret these results to suggest that audiovisual integration supports speech comprehension independently of children's social perception of the speaker, and discuss virtual speakers’ potential in research and pedagogical applications.

(Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Contribution to journal
publication status
published
subject
in
Speech Communication
volume
116
pages
12 pages
publisher
Elsevier
external identifiers
  • scopus:85075736895
ISSN
0167-6393
DOI
10.1016/j.specom.2019.11.005
language
English
LU publication?
yes
id
60db71d2-9717-4f27-b0b9-e7f833e34ef3
date added to LUP
2019-12-16 09:23:10
date last changed
2020-01-13 02:36:25
@article{60db71d2-9717-4f27-b0b9-e7f833e34ef3,
  abstract     = {<p>This paper presents a study where a 3D motion-capture animated ‘virtual speaker’ is compared to a video of a real speaker with regards to how it facilitates children's speech comprehension of narratives in background multitalker babble noise. As secondary measures, children self-assess the listening- and attentional effort demanded by the task, and associates words describing positive or negative social traits to the speaker. The results show that the virtual speaker, despite being associated with more negative social traits, facilitates speech comprehension in babble noise compared to a voice-only presentation but that the effect requires some adaptation. We also found the virtual speaker to be at least as facilitating as the video. We interpret these results to suggest that audiovisual integration supports speech comprehension independently of children's social perception of the speaker, and discuss virtual speakers’ potential in research and pedagogical applications.</p>},
  author       = {Nirme, Jens and Sahlén, Birgitta and Lyberg Åhlander, Viveka and Brännström, Jonas and Haake, Magnus},
  issn         = {0167-6393},
  language     = {eng},
  pages        = {44--55},
  publisher    = {Elsevier},
  series       = {Speech Communication},
  title        = {Audio-visual speech comprehension in noise with real and virtual speakers},
  url          = {http://dx.doi.org/10.1016/j.specom.2019.11.005},
  doi          = {10.1016/j.specom.2019.11.005},
  volume       = {116},
  year         = {2020},
}