Advanced

Language and Vision. Using visual information in real-world language situations

Andersson, Richard LU (2012) In Lund University Cognitive Studies 152.
Abstract (Swedish)
Popular Abstract in Swedish

Denna avhandling undersöker hur vi använder visuell kontextinformation i samtal. Alla modeller över språkprocessning och kommunikation är ense om att visuell kontextinformation har en positiv nettoeffekt på förståelse och kommunikation. Trots detta är det inte klart genom vilka mekanismer denna kontextinformation verkar, eller om dessa mekanismer fungerar i situationer utanför språklabbens experiment. Denna avhandling söker svar på dessa frågor genom primärt tre vägar:



1. Att undersöka om befintliga mekanismer fungerar i mer naturliga miljöer.

2. Utforskar nya mekanismer för hur visuell kontext kan vara användbar.

3. Lösa metodologiska utmaningar som hindrar... (More)
Popular Abstract in Swedish

Denna avhandling undersöker hur vi använder visuell kontextinformation i samtal. Alla modeller över språkprocessning och kommunikation är ense om att visuell kontextinformation har en positiv nettoeffekt på förståelse och kommunikation. Trots detta är det inte klart genom vilka mekanismer denna kontextinformation verkar, eller om dessa mekanismer fungerar i situationer utanför språklabbens experiment. Denna avhandling söker svar på dessa frågor genom primärt tre vägar:



1. Att undersöka om befintliga mekanismer fungerar i mer naturliga miljöer.

2. Utforskar nya mekanismer för hur visuell kontext kan vara användbar.

3. Lösa metodologiska utmaningar som hindrar forskare från att göra språkexperiment under mer naturliga omständigheter.



Denna avhandling rapporterar resultat från fem vetenskapliga studier. Studie 1 sökte lösningar på problemet med temporal osäkerhet i ögonrörelsemätare med låg samplingsfrekvens. Studie 2 undersökte vanan att titta på objekt som nämns, och om denna vana även är aktiv i mer utmanande språksituationer. Studie 3 föreslog ett nytt mått för att mäta kommunikativ framgång i naturliga samtalssituationer. Studie 4 fann att talare tenderar att referera till visuell information, även om den inte är obligatorisk för samtalet. Studie 5 fann att bilder med högt emotionellt innehåll uppfattas som mer hjälpsamt i en samtalsinformation med visuell kontext. Studien fann även att talares språkinnehåll inte påverkas av bildernas emotionella innehåll, vilket prediceras av nuvarande forskning. (Less)
Abstract
This dissertation is about the role of visual context information in communication. All accounts of language processing and communication agree that visual context is beneficial for understanding. Despite this, it is not clear through which mechanisms visual context works and whether these mechanisms also work in communication outside the lab setting. This dissertation aims to provide answers to these concerns and this is achieved through three different paths:



1. Testing known mechanisms under more realistic conditions.

2. Exploring new mechanisms for how visual context may inform communication

3. Resolve methodological issues that prevents experiments from being carried out under more realistic... (More)
This dissertation is about the role of visual context information in communication. All accounts of language processing and communication agree that visual context is beneficial for understanding. Despite this, it is not clear through which mechanisms visual context works and whether these mechanisms also work in communication outside the lab setting. This dissertation aims to provide answers to these concerns and this is achieved through three different paths:



1. Testing known mechanisms under more realistic conditions.

2. Exploring new mechanisms for how visual context may inform communication

3. Resolve methodological issues that prevents experiments from being carried out under more realistic conditions.



This dissertation reports the work of five different papers. Paper I provided concise answers to the problem of temporal imprecision in eye-trackers with low sampling frequencies. Paper II investigated the habit of looking at visual referents as they are mentioned and found that this habit works even in very demanding situations with complex speech and complex scenes. Paper III suggested a new measure for evaluating communicative success in unrestricted conversations. This paper also found that visual context information allow speakers to understand each other better. Paper IV found that visual context information, which use is often made mandatory in language experiments, is used even if it is completely optional to the communication task. Paper V found that images of high emotional content is perceived as more helpful in a communicative task. This paper also found that participants do not gravitate toward a similar use of emotional language, which is predicted by a current theory. (Less)
Please use this url to cite or link to this publication:
author
supervisor
opponent
  • Professor Griffin, Zenzi, University of Texas at Austin, USA
organization
publishing date
type
Thesis
publication status
published
subject
keywords
language, commmunication, visual context, visual world, pragmatics, ecological validity
in
Lund University Cognitive Studies
volume
152
pages
170 pages
defense location
Sal 104, Kungshuset, Lundagård, Lund
defense date
2012-04-20 13:15
ISSN
1101-8453
ISBN
978-91-7473-292-4
project
Cognition, Communication and Learning
language
English
LU publication?
yes
id
47ff1722-7f29-4b7b-81b9-c19498582af1 (old id 2425595)
date added to LUP
2012-03-23 14:14:20
date last changed
2016-09-19 08:45:01
@phdthesis{47ff1722-7f29-4b7b-81b9-c19498582af1,
  abstract     = {This dissertation is about the role of visual context information in communication. All accounts of language processing and communication agree that visual context is beneficial for understanding. Despite this, it is not clear through which mechanisms visual context works and whether these mechanisms also work in communication outside the lab setting. This dissertation aims to provide answers to these concerns and this is achieved through three different paths:<br/><br>
<br/><br>
1. Testing known mechanisms under more realistic conditions.<br/><br>
2. Exploring new mechanisms for how visual context may inform communication<br/><br>
3. Resolve methodological issues that prevents experiments from being carried out under more realistic conditions.<br/><br>
<br/><br>
This dissertation reports the work of five different papers. Paper I provided concise answers to the problem of temporal imprecision in eye-trackers with low sampling frequencies. Paper II investigated the habit of looking at visual referents as they are mentioned and found that this habit works even in very demanding situations with complex speech and complex scenes. Paper III suggested a new measure for evaluating communicative success in unrestricted conversations. This paper also found that visual context information allow speakers to understand each other better. Paper IV found that visual context information, which use is often made mandatory in language experiments, is used even if it is completely optional to the communication task. Paper V found that images of high emotional content is perceived as more helpful in a communicative task. This paper also found that participants do not gravitate toward a similar use of emotional language, which is predicted by a current theory.},
  author       = {Andersson, Richard},
  isbn         = {978-91-7473-292-4},
  issn         = {1101-8453},
  keyword      = {language,commmunication,visual context,visual world,pragmatics,ecological validity},
  language     = {eng},
  pages        = {170},
  school       = {Lund University},
  series       = {Lund University Cognitive Studies},
  title        = {Language and Vision. Using visual information in real-world language situations},
  volume       = {152},
  year         = {2012},
}