Uncovering Scientific and Multimodal Literacy through Audio Description
Holsanova, Jana LU
(2020)
In Journal of Visual Literacy
39(3-4).
p.132-148
- Abstract
- Today’s scientific texts are complex and multimodal. Due to new technology, the number of images is increasing, as is their diversity and complexity. Interaction with complex texts and visualisations becomes a challenge. How can we help readers and learners achieve multimodal literacy? We use data from the audio description of a popular scientific journal and think-aloud protocols to uncover knowledge and competences necessary for reading and understanding multimodal scientific texts. Four issues of the printed journal were analysed. The aural version of the journal was compared with the printed version to show how the semiotic interplay has been presented for the users. Additional meaning-making activities have been identified from the... (More)
- Today’s scientific texts are complex and multimodal. Due to new technology, the number of images is increasing, as is their diversity and complexity. Interaction with complex texts and visualisations becomes a challenge. How can we help readers and learners achieve multimodal literacy? We use data from the audio description of a popular scientific journal and think-aloud protocols to uncover knowledge and competences necessary for reading and understanding multimodal scientific texts. Four issues of the printed journal were analysed. The aural version of the journal was compared with the printed version to show how the semiotic interplay has been presented for the users. Additional meaning-making activities have been identified from the think-aloud protocol. As a result, we could reveal how the audio describer combined the contents of the available resources, made judgements about relevant information, determined ways of verbalising visual information, used conceptual knowledge, filled in the gaps missing in the interplay of the resources, and reordered information for optimal flow and understanding. We argue that the meaning-making activities identified through audio description and think-aloud protocols can be incorporated into instruction in educational contexts and can thereby improve readers’ competencies for reading and understanding multimodal scientific texts. (Less)
- Abstract (Swedish)
- Today’s scientific texts are complex and multimodal. Due to new technology, the number of images is increasing, as is their diversity and complexity. Interaction with complex texts and visualisations becomes a challenge. How can we help readers and learners achieve multimodal literacy? We use data from the audio description of a popular scientific journal and think- aloud protocols to uncover knowledge and competences necessary for reading and understanding multimodal scientific texts. Four issues of the printed journal were analysed. The aural version of the journal was compared with the printed version to show how the semiotic interplay has been presented for the users. Additional meaning-making activities have been identified from the... (More)
- Today’s scientific texts are complex and multimodal. Due to new technology, the number of images is increasing, as is their diversity and complexity. Interaction with complex texts and visualisations becomes a challenge. How can we help readers and learners achieve multimodal literacy? We use data from the audio description of a popular scientific journal and think- aloud protocols to uncover knowledge and competences necessary for reading and understanding multimodal scientific texts. Four issues of the printed journal were analysed. The aural version of the journal was compared with the printed version to show how the semiotic interplay has been presented for the users. Additional meaning-making activities have been identified from the think-aloud protocol. As a result, we could reveal how the audio describer combined the contents of the available resources, made judgements about relevant information, determined ways of verbalising visual information, used conceptual knowledge, filled in the gaps missing in the interplay of the resources, and reordered information for optimal flow and understanding. We argue that the meaning-making activities identified through audio description and think-aloud protocols can be incorporated into instruction in educational contexts and can thereby improve readers’ competencies for reading and understanding multimodal scientific texts. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/888d7ff5-675b-46af-a867-888758f57556
- author
- Holsanova, Jana
LU
- organization
- publishing date
- 2020-11-07
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- scientific and multimodal literacy, popular scientific explanation, audio description, production and reception of multimodality, think-aloud protocol, meaning-making processes, social semiotic and cognitive theories, educational applications
- in
- Journal of Visual Literacy
- volume
- 39
- issue
- 3-4
- pages
- 17 pages
- publisher
- International Visual Literacy Association
- external identifiers
-
- scopus:85092346444
- ISSN
- 1051-144X
- DOI
- 10.1080/1051144X.2020.1826219
- project
- Multimodal learning
- How the blind audience receive and experience audio descriptions of visual events
- Syntolkning - forskning och praktik
- Audio description for accessible communication/ Syntolkning för tillgänglig kommunikation
- Syntolkning
- Audio description and multimodality
- language
- English
- LU publication?
- yes
- id
- 888d7ff5-675b-46af-a867-888758f57556
- date added to LUP
- 2020-09-15 11:51:04
- date last changed
- 2022-04-19 00:39:24
@article{888d7ff5-675b-46af-a867-888758f57556,
abstract = {{Today’s scientific texts are complex and multimodal. Due to new technology, the number of images is increasing, as is their diversity and complexity. Interaction with complex texts and visualisations becomes a challenge. How can we help readers and learners achieve multimodal literacy? We use data from the audio description of a popular scientific journal and think-aloud protocols to uncover knowledge and competences necessary for reading and understanding multimodal scientific texts. Four issues of the printed journal were analysed. The aural version of the journal was compared with the printed version to show how the semiotic interplay has been presented for the users. Additional meaning-making activities have been identified from the think-aloud protocol. As a result, we could reveal how the audio describer combined the contents of the available resources, made judgements about relevant information, determined ways of verbalising visual information, used conceptual knowledge, filled in the gaps missing in the interplay of the resources, and reordered information for optimal flow and understanding. We argue that the meaning-making activities identified through audio description and think-aloud protocols can be incorporated into instruction in educational contexts and can thereby improve readers’ competencies for reading and understanding multimodal scientific texts.}},
author = {{Holsanova, Jana}},
issn = {{1051-144X}},
keywords = {{scientific and multimodal literacy; popular scientific explanation; audio description; production and reception of multimodality; think-aloud protocol; meaning-making processes; social semiotic and cognitive theories; educational applications}},
language = {{eng}},
month = {{11}},
number = {{3-4}},
pages = {{132--148}},
publisher = {{International Visual Literacy Association}},
series = {{Journal of Visual Literacy}},
title = {{Uncovering Scientific and Multimodal Literacy through Audio Description}},
url = {{https://lup.lub.lu.se/search/files/85638299/Uncovering_scientific_and_multimodal_literacy_through_audio_description.pdf}},
doi = {{10.1080/1051144X.2020.1826219}},
volume = {{39}},
year = {{2020}},
}