Language and Vision. Using visual information in real-world language situations
(2012) In Lund University Cognitive Studies 152.- Abstract
- This dissertation is about the role of visual context information in communication. All accounts of language processing and communication agree that visual context is beneficial for understanding. Despite this, it is not clear through which mechanisms visual context works and whether these mechanisms also work in communication outside the lab setting. This dissertation aims to provide answers to these concerns and this is achieved through three different paths:
1. Testing known mechanisms under more realistic conditions.
2. Exploring new mechanisms for how visual context may inform communication
3. Resolve methodological issues that prevents experiments from being carried out under more realistic... (More) - This dissertation is about the role of visual context information in communication. All accounts of language processing and communication agree that visual context is beneficial for understanding. Despite this, it is not clear through which mechanisms visual context works and whether these mechanisms also work in communication outside the lab setting. This dissertation aims to provide answers to these concerns and this is achieved through three different paths:
1. Testing known mechanisms under more realistic conditions.
2. Exploring new mechanisms for how visual context may inform communication
3. Resolve methodological issues that prevents experiments from being carried out under more realistic conditions.
This dissertation reports the work of five different papers. Paper I provided concise answers to the problem of temporal imprecision in eye-trackers with low sampling frequencies. Paper II investigated the habit of looking at visual referents as they are mentioned and found that this habit works even in very demanding situations with complex speech and complex scenes. Paper III suggested a new measure for evaluating communicative success in unrestricted conversations. This paper also found that visual context information allow speakers to understand each other better. Paper IV found that visual context information, which use is often made mandatory in language experiments, is used even if it is completely optional to the communication task. Paper V found that images of high emotional content is perceived as more helpful in a communicative task. This paper also found that participants do not gravitate toward a similar use of emotional language, which is predicted by a current theory. (Less) - Abstract (Swedish)
- Popular Abstract in Swedish
Denna avhandling undersöker hur vi använder visuell kontextinformation i samtal. Alla modeller över språkprocessning och kommunikation är ense om att visuell kontextinformation har en positiv nettoeffekt på förståelse och kommunikation. Trots detta är det inte klart genom vilka mekanismer denna kontextinformation verkar, eller om dessa mekanismer fungerar i situationer utanför språklabbens experiment. Denna avhandling söker svar på dessa frågor genom primärt tre vägar:
1. Att undersöka om befintliga mekanismer fungerar i mer naturliga miljöer.
2. Utforskar nya mekanismer för hur visuell kontext kan vara användbar.
3. Lösa metodologiska utmaningar som hindrar... (More) - Popular Abstract in Swedish
Denna avhandling undersöker hur vi använder visuell kontextinformation i samtal. Alla modeller över språkprocessning och kommunikation är ense om att visuell kontextinformation har en positiv nettoeffekt på förståelse och kommunikation. Trots detta är det inte klart genom vilka mekanismer denna kontextinformation verkar, eller om dessa mekanismer fungerar i situationer utanför språklabbens experiment. Denna avhandling söker svar på dessa frågor genom primärt tre vägar:
1. Att undersöka om befintliga mekanismer fungerar i mer naturliga miljöer.
2. Utforskar nya mekanismer för hur visuell kontext kan vara användbar.
3. Lösa metodologiska utmaningar som hindrar forskare från att göra språkexperiment under mer naturliga omständigheter.
Denna avhandling rapporterar resultat från fem vetenskapliga studier. Studie 1 sökte lösningar på problemet med temporal osäkerhet i ögonrörelsemätare med låg samplingsfrekvens. Studie 2 undersökte vanan att titta på objekt som nämns, och om denna vana även är aktiv i mer utmanande språksituationer. Studie 3 föreslog ett nytt mått för att mäta kommunikativ framgång i naturliga samtalssituationer. Studie 4 fann att talare tenderar att referera till visuell information, även om den inte är obligatorisk för samtalet. Studie 5 fann att bilder med högt emotionellt innehåll uppfattas som mer hjälpsamt i en samtalsinformation med visuell kontext. Studien fann även att talares språkinnehåll inte påverkas av bildernas emotionella innehåll, vilket prediceras av nuvarande forskning. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/2425595
- author
- Andersson, Richard LU
- supervisor
- opponent
-
- Professor Griffin, Zenzi, University of Texas at Austin, USA
- organization
- publishing date
- 2012
- type
- Thesis
- publication status
- published
- subject
- keywords
- language, commmunication, visual context, visual world, pragmatics, ecological validity
- in
- Lund University Cognitive Studies
- volume
- 152
- pages
- 170 pages
- defense location
- Sal 104, Kungshuset, Lundagård, Lund
- defense date
- 2012-04-20 13:15:00
- ISSN
- 1101-8453
- ISBN
- 978-91-7473-292-4
- project
- Thinking in Time: Cognition, Communication and Learning
- language
- English
- LU publication?
- yes
- id
- 47ff1722-7f29-4b7b-81b9-c19498582af1 (old id 2425595)
- date added to LUP
- 2016-04-04 09:33:27
- date last changed
- 2019-05-21 20:49:32
@phdthesis{47ff1722-7f29-4b7b-81b9-c19498582af1,
abstract = {{This dissertation is about the role of visual context information in communication. All accounts of language processing and communication agree that visual context is beneficial for understanding. Despite this, it is not clear through which mechanisms visual context works and whether these mechanisms also work in communication outside the lab setting. This dissertation aims to provide answers to these concerns and this is achieved through three different paths:<br/><br>
<br/><br>
1. Testing known mechanisms under more realistic conditions.<br/><br>
2. Exploring new mechanisms for how visual context may inform communication<br/><br>
3. Resolve methodological issues that prevents experiments from being carried out under more realistic conditions.<br/><br>
<br/><br>
This dissertation reports the work of five different papers. Paper I provided concise answers to the problem of temporal imprecision in eye-trackers with low sampling frequencies. Paper II investigated the habit of looking at visual referents as they are mentioned and found that this habit works even in very demanding situations with complex speech and complex scenes. Paper III suggested a new measure for evaluating communicative success in unrestricted conversations. This paper also found that visual context information allow speakers to understand each other better. Paper IV found that visual context information, which use is often made mandatory in language experiments, is used even if it is completely optional to the communication task. Paper V found that images of high emotional content is perceived as more helpful in a communicative task. This paper also found that participants do not gravitate toward a similar use of emotional language, which is predicted by a current theory.}},
author = {{Andersson, Richard}},
isbn = {{978-91-7473-292-4}},
issn = {{1101-8453}},
keywords = {{language; commmunication; visual context; visual world; pragmatics; ecological validity}},
language = {{eng}},
school = {{Lund University}},
series = {{Lund University Cognitive Studies}},
title = {{Language and Vision. Using visual information in real-world language situations}},
volume = {{152}},
year = {{2012}},
}