ChatGPT and the frustrated Socrates
Gregorcic, Bor and Pendrill, Ann Marie LU
(2023)
In Physics Education
58(3).
- Abstract
We present a case study of a conversation between ourselves and an artificial intelligence-based chatbot ChatGPT. We asked the chatbot to respond to a basic physics question that will be familiar to most physics teachers: ‘A teddy bear is thrown into the air. What is its acceleration in the highest point?’ The chatbot’s responses, while linguistically quite advanced, were unreliable in their correctness and often full of contradictions. We then attempted to engage in Socratic dialogue with the chatbot to resolve the errors and contradictions, but with little success. We found that ChatGPT is not yet good enough to be used as a cheating tool for physics students or as a physics tutor. However, we found it quite reliable in... (More)
We present a case study of a conversation between ourselves and an artificial intelligence-based chatbot ChatGPT. We asked the chatbot to respond to a basic physics question that will be familiar to most physics teachers: ‘A teddy bear is thrown into the air. What is its acceleration in the highest point?’ The chatbot’s responses, while linguistically quite advanced, were unreliable in their correctness and often full of contradictions. We then attempted to engage in Socratic dialogue with the chatbot to resolve the errors and contradictions, but with little success. We found that ChatGPT is not yet good enough to be used as a cheating tool for physics students or as a physics tutor. However, we found it quite reliable in generating incorrect responses on which physics teachers could train assessment of student responses.
(Less)
- author
- Gregorcic, Bor
and Pendrill, Ann Marie
LU
- organization
- publishing date
- 2023-05-01
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- acceleration, artificial intelligence, chatbot, ChatGPT, discourse imitation, kinematics, Socratic dialogue
- categories
- Higher Education
- in
- Physics Education
- volume
- 58
- issue
- 3
- article number
- 035021
- pages
- 9 pages
- publisher
- IOP Publishing
- external identifiers
-
- scopus:85150889894
- ISSN
- 0031-9120
- DOI
- 10.1088/1361-6552/acc299
- language
- English
- LU publication?
- yes
- additional info
- Publisher Copyright: © 2023 The Author(s). Published by IOP Publishing Ltd.
- id
- c6a34293-ec54-43fd-b634-91e37d6e9067
- date added to LUP
- 2023-04-05 06:30:43
- date last changed
- 2023-04-21 16:14:39
@article{c6a34293-ec54-43fd-b634-91e37d6e9067,
abstract = {{<p>We present a case study of a conversation between ourselves and an artificial intelligence-based chatbot ChatGPT. We asked the chatbot to respond to a basic physics question that will be familiar to most physics teachers: ‘<i>A teddy bear is thrown into the air. What is its acceleration in the highest point?</i>’ The chatbot’s responses, while linguistically quite advanced, were unreliable in their correctness and often full of contradictions. We then attempted to engage in Socratic dialogue with the chatbot to resolve the errors and contradictions, but with little success. We found that ChatGPT is not yet good enough to be used as a cheating tool for physics students or as a physics tutor. However, we found it quite reliable in generating incorrect responses on which physics teachers could train assessment of student responses.</p>}},
author = {{Gregorcic, Bor and Pendrill, Ann Marie}},
issn = {{0031-9120}},
keywords = {{acceleration; artificial intelligence; chatbot; ChatGPT; discourse imitation; kinematics; Socratic dialogue}},
language = {{eng}},
month = {{05}},
number = {{3}},
publisher = {{IOP Publishing}},
series = {{Physics Education}},
title = {{ChatGPT and the frustrated Socrates}},
url = {{http://dx.doi.org/10.1088/1361-6552/acc299}},
doi = {{10.1088/1361-6552/acc299}},
volume = {{58}},
year = {{2023}},
}