Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Precise localization of corneal reflections in eye images using deep learning trained on synthetic data

Byrne, Sean Anthony ; Nyström, Marcus LU orcid ; Maquiling, Virmarie ; Kasneci, Enkelejda and Niehorster, Diederick C. LU orcid (2023) In Behavior Research Methods
Abstract

We present a deep learning method for accurately localizing the center of a single corneal reflection (CR) in an eye image. Unlike previous approaches, we use a convolutional neural network (CNN) that was trained solely using synthetic data. Using only synthetic data has the benefit of completely sidestepping the time-consuming process of manual annotation that is required for supervised training on real eye images. To systematically evaluate the accuracy of our method, we first tested it on images with synthetic CRs placed on different backgrounds and embedded in varying levels of noise. Second, we tested the method on two datasets consisting of high-quality videos captured from real eyes. Our method outperformed state-of-the-art... (More)

We present a deep learning method for accurately localizing the center of a single corneal reflection (CR) in an eye image. Unlike previous approaches, we use a convolutional neural network (CNN) that was trained solely using synthetic data. Using only synthetic data has the benefit of completely sidestepping the time-consuming process of manual annotation that is required for supervised training on real eye images. To systematically evaluate the accuracy of our method, we first tested it on images with synthetic CRs placed on different backgrounds and embedded in varying levels of noise. Second, we tested the method on two datasets consisting of high-quality videos captured from real eyes. Our method outperformed state-of-the-art algorithmic methods on real eye images with a 3-41.5% reduction in terms of spatial precision across data sets, and performed on par with state-of-the-art on synthetic images in terms of spatial accuracy. We conclude that our method provides a precise method for CR center localization and provides a solution to the data availability problem, which is one of the important common roadblocks in the development of deep learning models for gaze estimation. Due to the superior CR center localization and ease of application, our method has the potential to improve the accuracy and precision of CR-based eye trackers.

(Less)
Please use this url to cite or link to this publication:
author
; ; ; and
organization
publishing date
type
Contribution to journal
publication status
epub
subject
keywords
Eye tracking, Gaze estimation, Neural networks, Simulations, Corneal reflection, P-CR
in
Behavior Research Methods
pages
16 pages
publisher
Springer
external identifiers
  • scopus:85180246294
  • pmid:38114880
ISSN
1554-3528
DOI
10.3758/s13428-023-02297-w
language
English
LU publication?
yes
id
a6dc0d27-e59a-453b-a88a-5a6360f103e0
date added to LUP
2023-12-23 16:23:08
date last changed
2024-04-24 11:11:49
@article{a6dc0d27-e59a-453b-a88a-5a6360f103e0,
  abstract     = {{<p>We present a deep learning method for accurately localizing the center of a single corneal reflection (CR) in an eye image. Unlike previous approaches, we use a convolutional neural network (CNN) that was trained solely using synthetic data. Using only synthetic data has the benefit of completely sidestepping the time-consuming process of manual annotation that is required for supervised training on real eye images. To systematically evaluate the accuracy of our method, we first tested it on images with synthetic CRs placed on different backgrounds and embedded in varying levels of noise. Second, we tested the method on two datasets consisting of high-quality videos captured from real eyes. Our method outperformed state-of-the-art algorithmic methods on real eye images with a 3-41.5% reduction in terms of spatial precision across data sets, and performed on par with state-of-the-art on synthetic images in terms of spatial accuracy. We conclude that our method provides a precise method for CR center localization and provides a solution to the data availability problem, which is one of the important common roadblocks in the development of deep learning models for gaze estimation. Due to the superior CR center localization and ease of application, our method has the potential to improve the accuracy and precision of CR-based eye trackers.</p>}},
  author       = {{Byrne, Sean Anthony and Nyström, Marcus and Maquiling, Virmarie and Kasneci, Enkelejda and Niehorster, Diederick C.}},
  issn         = {{1554-3528}},
  keywords     = {{Eye tracking; Gaze estimation; Neural networks; Simulations; Corneal reflection; P-CR}},
  language     = {{eng}},
  month        = {{12}},
  publisher    = {{Springer}},
  series       = {{Behavior Research Methods}},
  title        = {{Precise localization of corneal reflections in eye images using deep learning trained on synthetic data}},
  url          = {{http://dx.doi.org/10.3758/s13428-023-02297-w}},
  doi          = {{10.3758/s13428-023-02297-w}},
  year         = {{2023}},
}