Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Watching the brain build memories across eye movements: an EEG - eye-tracking coregistration study

Nikolaev, Andrey LU orcid ; Bramao, Ines LU ; Ekström, Axel ; Johansson, Roger LU orcid and Johansson, Mikael LU orcid (2021) The 27th Annual Meeting of The Organization for Human Brain Mapping (OHBM)
Abstract
Introduction
Episodic memory allows us to revisit our past and recollect inter-related elements that characterize life events (Tulving, 1983). When forming such relational memories, we only apprehend a small part of our visual field in full acuity at a time. This limitation is overcome by shifting visual attention over a sequence of eye movements. We bind these visual “samples” of the world into a coherent episodic memory. However, little is known about how we build coherent episodic memories across eye movements. Previous studies have established crucial roles of theta and alpha oscillations in memory formation (Hanslmayr et al. 2012; Herweg et al., 2020), but lacked the necessary precision to elucidate how coherent representations... (More)
Introduction
Episodic memory allows us to revisit our past and recollect inter-related elements that characterize life events (Tulving, 1983). When forming such relational memories, we only apprehend a small part of our visual field in full acuity at a time. This limitation is overcome by shifting visual attention over a sequence of eye movements. We bind these visual “samples” of the world into a coherent episodic memory. However, little is known about how we build coherent episodic memories across eye movements. Previous studies have established crucial roles of theta and alpha oscillations in memory formation (Hanslmayr et al. 2012; Herweg et al., 2020), but lacked the necessary precision to elucidate how coherent representations are formed over time across eye movements. Here, we combine EEG and eye-tracking to investigate the buildup of episodic memory representations in natural viewing behavior.

Methods
Participants (N = 28) were asked to encode events with elements from three categories (faces, places, objects, Fig. 1). In each block (N = 6), nine events were presented for 10 s each. After a distractor task, memory was tested for all event-specific combinations of elements. EEG and eye movements were simultaneously recorded. Only data from the encoding phase was analyzed.

A major problem in coregistration of EEG and eye movements in free viewing concerns overlapping effects of sequential saccades on EEG (Dimigen et al., 2011; Nikolaev et al., 2016). In order to overcome this, we used the deconvolution approach (Ehinger & Dimigen 2019). This regression-based method involves modelling not only experimental conditions of interest but also confounding eye movement characteristics and ordering factors, such as fixation rank, which can be “regressed out”. Before deconvolution, EEG was filtered in the theta and alpha frequency bands. The theta and alpha power were analyzed in epochs from -200 ms to +500 ms relative to the fixation onset.

Results
During the 10 s encoding, participants made, on average, 5.1 gaze transitions between categories. Subsequent memory performance increased with the cumulative number of gaze transitions during encoding (p < 0.001). Theta power in the interval -100+300 ms relative to the fixation onset over the frontal channels increased with time (p = 0.03) and performance (p = 0.02) (Fig. 2). Conversely, alpha power in the interval -100+300 ms over all electrodes prominently decreased with time (p = 0.002), irrespective of memory performance (Fig. 2). Next, we compared EEG for the first fixation on a category (i.e., after an inter-category saccade) with the last fixation on that category (i.e., after an intra-category saccade). The theta power in the interval -100+300 ms was significantly higher for the first than the last fixation (p < 0.001) and was more prominent for high than low memory performance (p < 0.001) (Fig. 1). The alpha power in the interval 0+500 ms was lower for the last than for the first fixation (p < 0.001), and lower for high than low memory performance at the last fixation (p = 0.02).

Conclusions
The frontal theta increase over the time of the event and for successful retrieval support the proposed pivotal role of theta synchronization in the formation of episodic memories (Clouter et al. 2017) and extends it to the binding of event elements across eye movements. The prominent theta effects in the interval of saccade execution and the larger theta power for fixations after inter- than intra-category saccades suggest that binding occurs via association of pre- and post-saccadic information at gaze transitions between categories. Alpha desynchronization over the time course of encoding may reflect an increasing amount of information (Hanslmayr et al. 2016) that is captured at each sequential fixation. The larger desynchronization associated with higher performance for fixations after intra- than inter-category saccades indicates active engagement of cortical areas in information acquisition during encoding. (Less)
Please use this url to cite or link to this publication:
author
; ; ; and
organization
publishing date
type
Contribution to conference
publication status
published
subject
conference name
The 27th Annual Meeting of The Organization for Human Brain Mapping (OHBM)
conference dates
2021-06-21 - 2021-06-25
project
Learning and remembering: The cognitive neuroscience of memory for real-world events
language
English
LU publication?
yes
id
eeb695e1-0c00-4952-ac2d-10b78d452cbd
alternative location
https://anyscreeninc.com/PF/OHBM/2021/OHBM-Ed-Courses-and-Poster-Presenters/pdf_poster_files/Andrey_Nikolaev60ac199b03aa3/Andrey_Nikolaev.pdf
date added to LUP
2021-07-01 18:25:54
date last changed
2022-06-29 11:00:30
@misc{eeb695e1-0c00-4952-ac2d-10b78d452cbd,
  abstract     = {{Introduction <br>
Episodic memory allows us to revisit our past and recollect inter-related elements that characterize life events (Tulving, 1983). When forming such relational memories, we only apprehend a small part of our visual field in full acuity at a time. This limitation is overcome by shifting visual attention over a sequence of eye movements. We bind these visual “samples” of the world into a coherent episodic memory. However, little is known about how we build coherent episodic memories across eye movements. Previous studies have established crucial roles of theta and alpha oscillations in memory formation (Hanslmayr et al. 2012; Herweg et al., 2020), but lacked the necessary precision to elucidate how coherent representations are formed over time across eye movements. Here, we combine EEG and eye-tracking to investigate the buildup of episodic memory representations in natural viewing behavior.<br>
<br>
Methods<br>
Participants (N = 28) were asked to encode events with elements from three categories (faces, places, objects, Fig. 1). In each block (N = 6), nine events were presented for 10 s each. After a distractor task, memory was tested for all event-specific combinations of elements. EEG and eye movements were simultaneously recorded. Only data from the encoding phase was analyzed.<br>
<br>
A major problem in coregistration of EEG and eye movements in free viewing concerns overlapping effects of sequential saccades on EEG (Dimigen et al., 2011; Nikolaev et al., 2016). In order to overcome this, we used the deconvolution approach (Ehinger &amp; Dimigen 2019). This regression-based method involves modelling not only experimental conditions of interest but also confounding eye movement characteristics and ordering factors, such as fixation rank, which can be “regressed out”. Before deconvolution, EEG was filtered in the theta and alpha frequency bands. The theta and alpha power were analyzed in epochs from -200 ms to +500 ms relative to the fixation onset.<br>
<br>
Results<br>
During the 10 s encoding, participants made, on average, 5.1 gaze transitions between categories. Subsequent memory performance increased with the cumulative number of gaze transitions during encoding (p &lt; 0.001). Theta power in the interval -100+300 ms relative to the fixation onset over the frontal channels increased with time (p = 0.03) and performance (p = 0.02) (Fig. 2). Conversely, alpha power in the interval -100+300 ms over all electrodes prominently decreased with time (p = 0.002), irrespective of memory performance (Fig. 2). Next, we compared EEG for the first fixation on a category (i.e., after an inter-category saccade) with the last fixation on that category (i.e., after an intra-category saccade). The theta power in the interval -100+300 ms was significantly higher for the first than the last fixation (p &lt; 0.001) and was more prominent for high than low memory performance (p &lt; 0.001) (Fig. 1). The alpha power in the interval 0+500 ms was lower for the last than for the first fixation (p &lt; 0.001), and lower for high than low memory performance at the last fixation (p = 0.02).<br>
<br>
Conclusions<br>
The frontal theta increase over the time of the event and for successful retrieval support the proposed pivotal role of theta synchronization in the formation of episodic memories (Clouter et al. 2017) and extends it to the binding of event elements across eye movements. The prominent theta effects in the interval of saccade execution and the larger theta power for fixations after inter- than intra-category saccades suggest that binding occurs via association of pre- and post-saccadic information at gaze transitions between categories. Alpha desynchronization over the time course of encoding may reflect an increasing amount of information (Hanslmayr et al. 2016) that is captured at each sequential fixation. The larger desynchronization associated with higher performance for fixations after intra- than inter-category saccades indicates active engagement of cortical areas in information acquisition during encoding.}},
  author       = {{Nikolaev, Andrey and Bramao, Ines and Ekström, Axel and Johansson, Roger and Johansson, Mikael}},
  language     = {{eng}},
  month        = {{06}},
  title        = {{Watching the brain build memories across eye movements: an EEG - eye-tracking coregistration study}},
  url          = {{https://anyscreeninc.com/PF/OHBM/2021/OHBM-Ed-Courses-and-Poster-Presenters/pdf_poster_files/Andrey_Nikolaev60ac199b03aa3/Andrey_Nikolaev.pdf}},
  year         = {{2021}},
}