Advanced

How we focus attention in picture viewing, picture description, and during mental imagery

Holsanova, Jana LU (2011) In Bilder - sehen - denken : zum Verhältnis von begrifflich-philosophischen und empirisch-psychologischen Ansätzen in der bildwissenschaftlichen Forschung p.291-313
Abstract
We cannot directly uncover the contents of our mind but we can come closer to cognitive processes via overt manifestations. I am interested in how speakers perceive, conceptualise and spontaneously describe complex pictures on higher levels of discourse and I use eye tracking methodology along with verbal protocols and a multimodal scoring method to study these mental processes.

In the studies on picture viewing and picture description (Holsanova 2001, 2008), the combination of visual and verbal data revealed how picture objects are focused on and conceptualised at different levels of specificity and how objects’ location, activities and attributes are evaluated. We can witness a process of stepwise specification, evaluation,... (More)
We cannot directly uncover the contents of our mind but we can come closer to cognitive processes via overt manifestations. I am interested in how speakers perceive, conceptualise and spontaneously describe complex pictures on higher levels of discourse and I use eye tracking methodology along with verbal protocols and a multimodal scoring method to study these mental processes.

In the studies on picture viewing and picture description (Holsanova 2001, 2008), the combination of visual and verbal data revealed how picture objects are focused on and conceptualised at different levels of specificity and how objects’ location, activities and attributes are evaluated. We can witness a process of stepwise specification, evaluation, interpretation and re-conceptualisation of picture elements and of the picture as a whole. During their successive picture discovery, viewers describe not only scene-inherent objects with spatial proximity but also clustered elements distributed across the scene and create new mental groupings based on abstract concepts. The process of mental zooming in and out is documented, where concrete objects are re-fixated and viewed with another concept in mind. In sum, the comparison of visual and verbal foci in the process of picture viewing and picture description shows us the ways information is acquired and processed in the human mind.

As the studies on mental imagery suggest (Johansson 2006 et al., Holsanova et al., forthc.), inner visualisations and mental images are important for speakers who describe pictures and visual scenes. When analysing attentional shifts in mental image inspection, a significant similarity was found between the eye movement patterns during picture viewing and those produced during picture description (when the picture was removed and the informants were looking at a white screen). The eye movements closely reflected the content and the spatial relations of the original picture, suggesting that the informants created a sort of mental image as an aid for their descriptions from memory. Eye movements were thus not dependent on a present visual scene but on a mental record of the scene. In addition, even verbal scene descriptions evoked mental images and elicited eye movements that reflect spatiality. In sum, picture viewing, picture description and mental imagery can be used as windows to the mind showing how language and vision, in concert, elucidate covert mental processes. (Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
keywords
picture viewing, eye movement protocol, cognitive processes, mental imagery, verbal protocol, picture description
in
Bilder - sehen - denken : zum Verhältnis von begrifflich-philosophischen und empirisch-psychologischen Ansätzen in der bildwissenschaftlichen Forschung
editor
Sachs-Hombach, Klaus and Totzke, Rainer
pages
291 - 313
publisher
von Halem
ISBN
978-3-86962-006-0
project
Cognition, Communication and Learning
language
English
LU publication?
yes
id
245dff93-9b7a-4987-a231-52db1b7df18f (old id 1465439)
date added to LUP
2009-09-18 11:45:41
date last changed
2016-09-08 14:29:49
@inbook{245dff93-9b7a-4987-a231-52db1b7df18f,
  abstract     = {We cannot directly uncover the contents of our mind but we can come closer to cognitive processes via overt manifestations. I am interested in how speakers perceive, conceptualise and spontaneously describe complex pictures on higher levels of discourse and I use eye tracking methodology along with verbal protocols and a multimodal scoring method to study these mental processes.<br/><br>
In the studies on picture viewing and picture description (Holsanova 2001, 2008), the combination of visual and verbal data revealed how picture objects are focused on and conceptualised at different levels of specificity and how objects’ location, activities and attributes are evaluated. We can witness a process of stepwise specification, evaluation, interpretation and re-conceptualisation of picture elements and of the picture as a whole. During their successive picture discovery, viewers describe not only scene-inherent objects with spatial proximity but also clustered elements distributed across the scene and create new mental groupings based on abstract concepts. The process of mental zooming in and out is documented, where concrete objects are re-fixated and viewed with another concept in mind. In sum, the comparison of visual and verbal foci in the process of picture viewing and picture description shows us the ways information is acquired and processed in the human mind.<br/><br>
As the studies on mental imagery suggest (Johansson 2006 et al., Holsanova et al., forthc.), inner visualisations and mental images are important for speakers who describe pictures and visual scenes. When analysing attentional shifts in mental image inspection, a significant similarity was found between the eye movement patterns during picture viewing and those produced during picture description (when the picture was removed and the informants were looking at a white screen). The eye movements closely reflected the content and the spatial relations of the original picture, suggesting that the informants created a sort of mental image as an aid for their descriptions from memory. Eye movements were thus not dependent on a present visual scene but on a mental record of the scene. In addition, even verbal scene descriptions evoked mental images and elicited eye movements that reflect spatiality. In sum, picture viewing, picture description and mental imagery can be used as windows to the mind showing how language and vision, in concert, elucidate covert mental processes.},
  author       = {Holsanova, Jana},
  editor       = {Sachs-Hombach, Klaus and Totzke, Rainer},
  isbn         = {978-3-86962-006-0},
  keyword      = {picture viewing,eye movement protocol,cognitive processes,mental imagery,verbal protocol,picture description},
  language     = {eng},
  pages        = {291--313},
  publisher    = {von Halem},
  series       = {Bilder - sehen - denken : zum Verhältnis von begrifflich-philosophischen und empirisch-psychologischen Ansätzen in der bildwissenschaftlichen Forschung},
  title        = {How we focus attention in picture viewing, picture description, and during mental imagery},
  year         = {2011},
}