Advanced

What speakers do and what addressees look at: Visual attention to gestures in human interaction live and on video

Gullberg, Marianne LU and Holmqvist, Kenneth LU (2006) In Pragmatics & Cognition 14(1). p.53-82
Abstract
This study investigates whether addressees visually attend to speakers’ gestures in interaction and whether attention is modulated by changes in social setting and display size. We compare a live face-to-face setting to two video conditions. In all conditions, the face dominates as a fixation target and only a minority of gestures draw fixations. The social and size parameters affect gaze mainly when combined and in the opposite direction from the predicted with fewer gestures fixated on video than live. Gestural holds and speakers’ gaze at their own gestures reliably attract addressees’ fixations in all conditions. The attraction force of holds is unaffected by changes in social and size parameters, suggesting a bottom-up response,... (More)
This study investigates whether addressees visually attend to speakers’ gestures in interaction and whether attention is modulated by changes in social setting and display size. We compare a live face-to-face setting to two video conditions. In all conditions, the face dominates as a fixation target and only a minority of gestures draw fixations. The social and size parameters affect gaze mainly when combined and in the opposite direction from the predicted with fewer gestures fixated on video than live. Gestural holds and speakers’ gaze at their own gestures reliably attract addressees’ fixations in all conditions. The attraction force of holds is unaffected by changes in social and size parameters, suggesting a bottom-up response, whereas speaker-fixated gestures draw significantly less attention in both video conditions, suggesting a social effect for overt gaze-following and visual joint attention. The study provides and validates a video-based paradigm enabling further experimental but ecologically valid explorations of cross-modal information processing. (Less)
Please use this url to cite or link to this publication:
author
publishing date
type
Contribution to journal
publication status
published
subject
keywords
cross-modal information processing, eye gaze, Gesture, interaction
in
Pragmatics & Cognition
volume
14
issue
1
pages
53 - 82
publisher
John Benjamins Publishing Company
external identifiers
  • scopus:33748423854
ISSN
0929-0907
DOI
10.1075/pc.14.1.05gul
language
English
LU publication?
no
id
5e3ad670-b4cc-43b3-ac22-fd5c22de651c (old id 1611779)
date added to LUP
2010-06-04 14:34:27
date last changed
2017-01-01 07:44:29
@article{5e3ad670-b4cc-43b3-ac22-fd5c22de651c,
  abstract     = {This study investigates whether addressees visually attend to speakers’ gestures in interaction and whether attention is modulated by changes in social setting and display size. We compare a live face-to-face setting to two video conditions. In all conditions, the face dominates as a fixation target and only a minority of gestures draw fixations. The social and size parameters affect gaze mainly when combined and in the opposite direction from the predicted with fewer gestures fixated on video than live. Gestural holds and speakers’ gaze at their own gestures reliably attract addressees’ fixations in all conditions. The attraction force of holds is unaffected by changes in social and size parameters, suggesting a bottom-up response, whereas speaker-fixated gestures draw significantly less attention in both video conditions, suggesting a social effect for overt gaze-following and visual joint attention. The study provides and validates a video-based paradigm enabling further experimental but ecologically valid explorations of cross-modal information processing.},
  author       = {Gullberg, Marianne and Holmqvist, Kenneth},
  issn         = {0929-0907},
  keyword      = {cross-modal information processing,eye gaze,Gesture,interaction},
  language     = {eng},
  number       = {1},
  pages        = {53--82},
  publisher    = {John Benjamins Publishing Company},
  series       = {Pragmatics & Cognition},
  title        = {What speakers do and what addressees look at: Visual attention to gestures in human interaction live and on video},
  url          = {http://dx.doi.org/10.1075/pc.14.1.05gul},
  volume       = {14},
  year         = {2006},
}