Hello GPT! Goodbye home examination? An exploratory study of AI chatbots impact on university teachers’ assessment practices
(2023) In Assessment and Evaluation in Higher Education- Abstract
AI chatbots have recently fuelled debate regarding education practices in higher education institutions worldwide. Focusing on Generative AI and ChatGPT in particular, our study examines how AI chatbots impact university teachers’ assessment practices, exploring teachers’ perceptions about how ChatGPT performs in response to home examination prompts in undergraduate contexts. University teachers (n = 24) from four different departments in humanities and social sciences participated in Turing Test-inspired experiments, where they blindly assessed student and ChatGPT-written responses to home examination questions. Additionally, we conducted semi-structured interviews in focus groups with the same teachers examining their reflections... (More)
AI chatbots have recently fuelled debate regarding education practices in higher education institutions worldwide. Focusing on Generative AI and ChatGPT in particular, our study examines how AI chatbots impact university teachers’ assessment practices, exploring teachers’ perceptions about how ChatGPT performs in response to home examination prompts in undergraduate contexts. University teachers (n = 24) from four different departments in humanities and social sciences participated in Turing Test-inspired experiments, where they blindly assessed student and ChatGPT-written responses to home examination questions. Additionally, we conducted semi-structured interviews in focus groups with the same teachers examining their reflections about the quality of the texts they assessed. Regarding chatbot-generated texts, we found a passing rate range across the cohort (37.5 − 85.7%) and a chatbot-written suspicion range (14–23%). Regarding the student-written texts, we identified patterns of downgrading, suggesting that teachers were more critical when grading student-written texts. Drawing on post-phenomenology and mediation theory, we discuss AI chatbots as a potentially disruptive technology in higher education practices.
(Less)
- author
- Farazouli, Alexandra ; Cerratto-Pargman, Teresa ; Bolander-Laksov, Klara LU and McGrath, Cormac
- publishing date
- 2023-08-01
- type
- Contribution to journal
- publication status
- epub
- subject
- keywords
- AI-chatbots, assessment, higher education, home examination, Turing test
- in
- Assessment and Evaluation in Higher Education
- publisher
- Taylor & Francis
- external identifiers
-
- scopus:85166200345
- ISSN
- 0260-2938
- DOI
- 10.1080/02602938.2023.2241676
- language
- English
- LU publication?
- no
- additional info
- Publisher Copyright: © 2023 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
- id
- 41809216-55c8-4ae5-9742-7e536ba15524
- date added to LUP
- 2024-02-09 13:45:10
- date last changed
- 2024-02-13 10:14:27
@article{41809216-55c8-4ae5-9742-7e536ba15524,
abstract = {{<p>AI chatbots have recently fuelled debate regarding education practices in higher education institutions worldwide. Focusing on Generative AI and ChatGPT in particular, our study examines how AI chatbots impact university teachers’ assessment practices, exploring teachers’ perceptions about how ChatGPT performs in response to home examination prompts in undergraduate contexts. University teachers (n = 24) from four different departments in humanities and social sciences participated in Turing Test-inspired experiments, where they blindly assessed student and ChatGPT-written responses to home examination questions. Additionally, we conducted semi-structured interviews in focus groups with the same teachers examining their reflections about the quality of the texts they assessed. Regarding chatbot-generated texts, we found a passing rate range across the cohort (37.5 − 85.7%) and a chatbot-written suspicion range (14–23%). Regarding the student-written texts, we identified patterns of downgrading, suggesting that teachers were more critical when grading student-written texts. Drawing on post-phenomenology and mediation theory, we discuss AI chatbots as a potentially disruptive technology in higher education practices.</p>}},
author = {{Farazouli, Alexandra and Cerratto-Pargman, Teresa and Bolander-Laksov, Klara and McGrath, Cormac}},
issn = {{0260-2938}},
keywords = {{AI-chatbots; assessment; higher education; home examination; Turing test}},
language = {{eng}},
month = {{08}},
publisher = {{Taylor & Francis}},
series = {{Assessment and Evaluation in Higher Education}},
title = {{Hello GPT! Goodbye home examination? An exploratory study of AI chatbots impact on university teachers’ assessment practices}},
url = {{http://dx.doi.org/10.1080/02602938.2023.2241676}},
doi = {{10.1080/02602938.2023.2241676}},
year = {{2023}},
}