Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Multimodal-first or pantomime-first? : Communicating events through pantomime with and without vocalization

Zlatev, Jordan LU ; Wacewicz, Sławomir ; Zywiczynski, Przemyslaw and van de Weijer, Joost LU orcid (2017) In Interaction Studies 18(3). p.465-488
Abstract
A persistent controversy in language evolution research has been whether language emerged in the gestural-visual or in the vocal-auditory modality. A "dialectic" solution to this age-old debate has now been gaining ground: language was fully multimodal from the start, and remains so to this day. In this paper, we show this solution to be too simplistic and outline a more specific theoretical proposal, which we designate as pantomime-first. To decide between the multimodal-first and pantomime-first alternatives, we review several lines of interdisciplinary evidence and complement it with a cognitive-semiotic experiment. In the study, the participants saw – and then matched to hand-drawn images – recordings of short transitive events enacted... (More)
A persistent controversy in language evolution research has been whether language emerged in the gestural-visual or in the vocal-auditory modality. A "dialectic" solution to this age-old debate has now been gaining ground: language was fully multimodal from the start, and remains so to this day. In this paper, we show this solution to be too simplistic and outline a more specific theoretical proposal, which we designate as pantomime-first. To decide between the multimodal-first and pantomime-first alternatives, we review several lines of interdisciplinary evidence and complement it with a cognitive-semiotic experiment. In the study, the participants saw – and then matched to hand-drawn images – recordings of short transitive events enacted by 4 actors in two conditions: visual (only body movement), and multimodal (body movement accompanied by nonlinguistic vocalization). Significantly, the matching accuracy was greater in the visual than the multimodal condition, though a follow-up experiment revealed that the emotional pro les of the events enacted in the multimodal condition could be reliably detected from the sound alone. We see these results as supporting the proposed pantomime-first scenario. (Less)
Please use this url to cite or link to this publication:
author
; ; and
organization
publishing date
type
Contribution to journal
publication status
published
subject
keywords
bodily mimesis, pantomime, multimodal origins, multimodality, language origins, language evolution, gesture
in
Interaction Studies
volume
18
issue
3
pages
465 - 488
publisher
John Benjamins Publishing Company
external identifiers
  • scopus:85027680169
ISSN
1572-0373
language
English
LU publication?
yes
id
38c86c6d-8f98-4fb0-a1a7-e0ba30cb8b29
date added to LUP
2017-10-27 15:33:54
date last changed
2023-11-17 08:13:56
@article{38c86c6d-8f98-4fb0-a1a7-e0ba30cb8b29,
  abstract     = {{A persistent controversy in language evolution research has been whether language emerged in the gestural-visual or in the vocal-auditory modality. A "dialectic" solution to this age-old debate has now been gaining ground: language was fully multimodal from the start, and remains so to this day. In this paper, we show this solution to be too simplistic and outline a more specific theoretical proposal, which we designate as pantomime-first. To decide between the multimodal-first and pantomime-first alternatives, we review several lines of interdisciplinary evidence and complement it with a cognitive-semiotic experiment. In the study, the participants saw – and then matched to hand-drawn images – recordings of short transitive events enacted by 4 actors in two conditions: visual (only body movement), and multimodal (body movement accompanied by nonlinguistic vocalization). Significantly, the matching accuracy was greater in the visual than the multimodal condition, though a follow-up experiment revealed that the emotional pro les of the events enacted in the multimodal condition could be reliably detected from the sound alone. We see these results as supporting the proposed pantomime-first scenario.}},
  author       = {{Zlatev, Jordan and Wacewicz, Sławomir and Zywiczynski, Przemyslaw and van de Weijer, Joost}},
  issn         = {{1572-0373}},
  keywords     = {{bodily mimesis; pantomime; multimodal origins; multimodality; language origins; language evolution; gesture}},
  language     = {{eng}},
  number       = {{3}},
  pages        = {{465--488}},
  publisher    = {{John Benjamins Publishing Company}},
  series       = {{Interaction Studies}},
  title        = {{Multimodal-first or pantomime-first? : Communicating events through pantomime with and without vocalization}},
  volume       = {{18}},
  year         = {{2017}},
}