Toward Gaze-enabled Programming Tool Assistance
Kuang, Peng LU
(2024)
- Abstract
- Programming is a cognitively demanding exercise. In particular, today’s software development requires a collective effort of programmers and the orchestration of a complex programming infrastructure. As disruptive technologies emerge, e.g., AI and quantum computing, the programming practice is undergoing a change, facing an uncertain future that we may not be able to accurately predict but can envision and work toward.
With the maturity of eye-tracking and its integration into everyday consumer electronics such as Alienware’s laptops and Apple’s Vision Pro, we expect it will eventually make its way into everyday use just as touchpad, camera, and micro- phone. Therefore, we see an opportunity to design eye-tracking based assistance to... (More) - Programming is a cognitively demanding exercise. In particular, today’s software development requires a collective effort of programmers and the orchestration of a complex programming infrastructure. As disruptive technologies emerge, e.g., AI and quantum computing, the programming practice is undergoing a change, facing an uncertain future that we may not be able to accurately predict but can envision and work toward.
With the maturity of eye-tracking and its integration into everyday consumer electronics such as Alienware’s laptops and Apple’s Vision Pro, we expect it will eventually make its way into everyday use just as touchpad, camera, and micro- phone. Therefore, we see an opportunity to design eye-tracking based assistance to support programmers. Given programmers spend a large amount of their time reading and understanding code, which heavily relies on eyes, we deem this to be a promising problem domain where eye-tracking can be of assistance.
To explore this inquiry, we undertook two mapping studies to establish the problem and solution constructs. We then surveyed professional developers to understand this representative cohort of our prospective users and gather concrete, situated problems from them. We conducted these studies under the guiding design science model for empirical software engineering which centers on a problem- solution pair.
From the first study, we found that eye-tracking so far is used mostly for education-oriented studies in the research community focused on software devel- opment. There is a need to bring it closer to practitioners. From the second study, we identify that the gaze data produced by eye trackers has been explored with a collection of machine learning techniques. However, these models were trained with small samples that might carry bias and insufficiency. Contemporary ma- chine learning techniques may be able to compensate for that. From the survey, we learned that developers have already adopted AI assistance, and they are mostly positive about it despite room for greater accuracy and capability. As eye-tracking is relatively novel to them, most developers are unsure about how it can help them.
For future work, we plan to practice designing with programmers to develop and evaluate our proof of concept and explore gaze data with more tailored ma- chine learning techniques, which aims to generate integration into our system. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/9e36570c-7839-49a3-b521-39b9f43ca51b
- author
- Kuang, Peng
LU
- supervisor
- organization
- publishing date
- 2024-04-09
- type
- Thesis
- publication status
- published
- subject
- keywords
- eye tracking, computer programming, software development, machine learning, developer tools, gaze
- publisher
- Lund University
- ISBN
- 978-91-7623-306-1
- 978-91-7623-305-4
- language
- English
- LU publication?
- yes
- id
- 9e36570c-7839-49a3-b521-39b9f43ca51b
- date added to LUP
- 2024-04-04 15:37:55
- date last changed
- 2024-04-25 03:06:35
@misc{9e36570c-7839-49a3-b521-39b9f43ca51b,
abstract = {{Programming is a cognitively demanding exercise. In particular, today’s software development requires a collective effort of programmers and the orchestration of a complex programming infrastructure. As disruptive technologies emerge, e.g., AI and quantum computing, the programming practice is undergoing a change, facing an uncertain future that we may not be able to accurately predict but can envision and work toward.<br/>With the maturity of eye-tracking and its integration into everyday consumer electronics such as Alienware’s laptops and Apple’s Vision Pro, we expect it will eventually make its way into everyday use just as touchpad, camera, and micro- phone. Therefore, we see an opportunity to design eye-tracking based assistance to support programmers. Given programmers spend a large amount of their time reading and understanding code, which heavily relies on eyes, we deem this to be a promising problem domain where eye-tracking can be of assistance.<br/>To explore this inquiry, we undertook two mapping studies to establish the problem and solution constructs. We then surveyed professional developers to understand this representative cohort of our prospective users and gather concrete, situated problems from them. We conducted these studies under the guiding design science model for empirical software engineering which centers on a problem- solution pair.<br/>From the first study, we found that eye-tracking so far is used mostly for education-oriented studies in the research community focused on software devel- opment. There is a need to bring it closer to practitioners. From the second study, we identify that the gaze data produced by eye trackers has been explored with a collection of machine learning techniques. However, these models were trained with small samples that might carry bias and insufficiency. Contemporary ma- chine learning techniques may be able to compensate for that. From the survey, we learned that developers have already adopted AI assistance, and they are mostly positive about it despite room for greater accuracy and capability. As eye-tracking is relatively novel to them, most developers are unsure about how it can help them.<br/>For future work, we plan to practice designing with programmers to develop and evaluate our proof of concept and explore gaze data with more tailored ma- chine learning techniques, which aims to generate integration into our system.}},
author = {{Kuang, Peng}},
isbn = {{978-91-7623-306-1}},
keywords = {{eye tracking; computer programming; software development; machine learning; developer tools; gaze}},
language = {{eng}},
month = {{04}},
note = {{Licentiate Thesis}},
publisher = {{Lund University}},
title = {{Toward Gaze-enabled Programming Tool Assistance}},
url = {{https://lup.lub.lu.se/search/files/181329152/Licentiate_thesis_Toward_Gaze_enabled_Programming_Tool_Assistance.pdf}},
year = {{2024}},
}