Advanced

gazeNet : End-to-end eye-movement event detection with deep neural networks

Zemblys, Raimondas ; Niehorster, Diederick C LU and Holmqvist, Kenneth (2019) In Behavior Research Methods 51(2). p.840-864
Abstract

Existing event detection algorithms for eye-movement data almost exclusively rely on thresholding one or more hand-crafted signal features, each computed from the stream of raw gaze data. Moreover, this thresholding is largely left for the end user. Here we present and develop gazeNet, a new framework for creating event detectors that do not require hand-crafted signal features or signal thresholding. It employs an end-to-end deep learning approach, which takes raw eye-tracking data as input and classifies it into fixations, saccades and post-saccadic oscillations. Our method thereby challenges an established tacit assumption that hand-crafted features are necessary in the design of event detection algorithms. The downside of the deep... (More)

Existing event detection algorithms for eye-movement data almost exclusively rely on thresholding one or more hand-crafted signal features, each computed from the stream of raw gaze data. Moreover, this thresholding is largely left for the end user. Here we present and develop gazeNet, a new framework for creating event detectors that do not require hand-crafted signal features or signal thresholding. It employs an end-to-end deep learning approach, which takes raw eye-tracking data as input and classifies it into fixations, saccades and post-saccadic oscillations. Our method thereby challenges an established tacit assumption that hand-crafted features are necessary in the design of event detection algorithms. The downside of the deep learning approach is that a large amount of training data is required. We therefore first develop a method to augment hand-coded data, so that we can strongly enlarge the data set used for training, minimizing the time spent on manual coding. Using this extended hand-coded data, we train a neural network that produces eye-movement event classification from raw eye-movement data without requiring any predefined feature extraction or post-processing steps. The resulting classification performance is at the level of expert human coders. Moreover, an evaluation of gazeNet on two other datasets showed that gazeNet generalized to data from different eye trackers and consistently outperformed several other event detection algorithms that we tested.

(Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Contribution to journal
publication status
published
subject
in
Behavior Research Methods
volume
51
issue
2
pages
25 pages
publisher
Springer
external identifiers
  • pmid:30334148
  • scopus:85055577469
ISSN
1554-351X
DOI
10.3758/s13428-018-1133-5
language
English
LU publication?
yes
id
7912b750-098d-419c-afff-073443ef1e55
date added to LUP
2018-10-26 10:23:30
date last changed
2020-04-02 10:06:06
@article{7912b750-098d-419c-afff-073443ef1e55,
  abstract     = {<p>Existing event detection algorithms for eye-movement data almost exclusively rely on thresholding one or more hand-crafted signal features, each computed from the stream of raw gaze data. Moreover, this thresholding is largely left for the end user. Here we present and develop gazeNet, a new framework for creating event detectors that do not require hand-crafted signal features or signal thresholding. It employs an end-to-end deep learning approach, which takes raw eye-tracking data as input and classifies it into fixations, saccades and post-saccadic oscillations. Our method thereby challenges an established tacit assumption that hand-crafted features are necessary in the design of event detection algorithms. The downside of the deep learning approach is that a large amount of training data is required. We therefore first develop a method to augment hand-coded data, so that we can strongly enlarge the data set used for training, minimizing the time spent on manual coding. Using this extended hand-coded data, we train a neural network that produces eye-movement event classification from raw eye-movement data without requiring any predefined feature extraction or post-processing steps. The resulting classification performance is at the level of expert human coders. Moreover, an evaluation of gazeNet on two other datasets showed that gazeNet generalized to data from different eye trackers and consistently outperformed several other event detection algorithms that we tested.</p>},
  author       = {Zemblys, Raimondas and Niehorster, Diederick C and Holmqvist, Kenneth},
  issn         = {1554-351X},
  language     = {eng},
  number       = {2},
  pages        = {840--864},
  publisher    = {Springer},
  series       = {Behavior Research Methods},
  title        = {gazeNet : End-to-end eye-movement event detection with deep neural networks},
  url          = {http://dx.doi.org/10.3758/s13428-018-1133-5},
  doi          = {10.3758/s13428-018-1133-5},
  volume       = {51},
  year         = {2019},
}