The role of manual gestures in second language comprehension : A simultaneous interpreting experiment
(2023) In Frontiers in Psychology 14. p.01-15- Abstract
- Manual gestures and speech form a single integrated system during nativelanguage comprehension. However, it remains unclear whether this hold forsecond language (L2) comprehension, more specifically for simultaneousinterpreting (SI), which involves comprehension in one language and simultaneousproduction in another. In a combined mismatch and priming paradigm,we presented Swedish speakers fluent in L2 English with multimodal stimuli inwhich speech was congruent or incongruent with a gesture. A picture prime wasdisplayed before the stimuli. Participants had to decide whether the video wasrelated to the prime, focusing either on the auditory or the visual information.Participants performed the task either during passive viewing or during SI... (More)
- Manual gestures and speech form a single integrated system during nativelanguage comprehension. However, it remains unclear whether this hold forsecond language (L2) comprehension, more specifically for simultaneousinterpreting (SI), which involves comprehension in one language and simultaneousproduction in another. In a combined mismatch and priming paradigm,we presented Swedish speakers fluent in L2 English with multimodal stimuli inwhich speech was congruent or incongruent with a gesture. A picture prime wasdisplayed before the stimuli. Participants had to decide whether the video wasrelated to the prime, focusing either on the auditory or the visual information.Participants performed the task either during passive viewing or during SI intotheir L1 Swedish (order counterbalanced). Incongruent stimuli yielded longerreaction times than congruent stimuli, during both viewing and interpreting.Visual and audio targets were processed equally easily in both activities. However,in both activities incongruent speech was more disruptive for gesture processingthan incongruent gesture was for speech processing. Thus, the data only partlysupports the expected mutual and obligatory interaction of gesture and speechin L2 comprehension. Interestingly, there were no differences between activitiessuggesting that the language comprehension component in SI shares featureswith other (L2) comprehension tasks. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/8e5f2c6a-6a6a-40e8-8cce-c490d8c440f4
- author
- Arbona, Eléonore ; Seeber, Kilian G. and Gullberg, Marianne LU
- organization
- publishing date
- 2023-06-27
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- gesture, multimodality, simultaneous interpreting, bilingualism, second language comprehension, eye-tracking, integrated-systems hypothesis
- in
- Frontiers in Psychology
- volume
- 14
- pages
- 15 pages
- publisher
- Frontiers Media S. A.
- external identifiers
-
- pmid:37441333
- scopus:85164727860
- ISSN
- 1664-1078
- DOI
- 10.3389/fpsyg.2023.1188628
- project
- Embodied bilingualism (a Wallenberg Scholar project)
- language
- English
- LU publication?
- yes
- id
- 8e5f2c6a-6a6a-40e8-8cce-c490d8c440f4
- date added to LUP
- 2023-06-10 21:08:33
- date last changed
- 2023-11-22 23:14:29
@article{8e5f2c6a-6a6a-40e8-8cce-c490d8c440f4, abstract = {{Manual gestures and speech form a single integrated system during nativelanguage comprehension. However, it remains unclear whether this hold forsecond language (L2) comprehension, more specifically for simultaneousinterpreting (SI), which involves comprehension in one language and simultaneousproduction in another. In a combined mismatch and priming paradigm,we presented Swedish speakers fluent in L2 English with multimodal stimuli inwhich speech was congruent or incongruent with a gesture. A picture prime wasdisplayed before the stimuli. Participants had to decide whether the video wasrelated to the prime, focusing either on the auditory or the visual information.Participants performed the task either during passive viewing or during SI intotheir L1 Swedish (order counterbalanced). Incongruent stimuli yielded longerreaction times than congruent stimuli, during both viewing and interpreting.Visual and audio targets were processed equally easily in both activities. However,in both activities incongruent speech was more disruptive for gesture processingthan incongruent gesture was for speech processing. Thus, the data only partlysupports the expected mutual and obligatory interaction of gesture and speechin L2 comprehension. Interestingly, there were no differences between activitiessuggesting that the language comprehension component in SI shares featureswith other (L2) comprehension tasks.}}, author = {{Arbona, Eléonore and Seeber, Kilian G. and Gullberg, Marianne}}, issn = {{1664-1078}}, keywords = {{gesture; multimodality; simultaneous interpreting; bilingualism; second language comprehension; eye-tracking; integrated-systems hypothesis}}, language = {{eng}}, month = {{06}}, pages = {{01--15}}, publisher = {{Frontiers Media S. A.}}, series = {{Frontiers in Psychology}}, title = {{The role of manual gestures in second language comprehension : A simultaneous interpreting experiment}}, url = {{http://dx.doi.org/10.3389/fpsyg.2023.1188628}}, doi = {{10.3389/fpsyg.2023.1188628}}, volume = {{14}}, year = {{2023}}, }