Adaptive Gen-AI Guidance in Virtual Reality : A Multimodal Exploration of Engagement in Neapolitan Pizza-Making
(2025) 27th International Conference on Multimodal Interaction, ICMI 2025 In ICMI 2025 - Proceedings of the 27th International Conference on Multimodal Interaction p.305-316- Abstract
Virtual reality (VR) offers promising opportunities for procedural learning, particularly in preserving intangible cultural heritage. Advances in generative artificial intelligence (Gen-AI) further enrich these experiences by enabling adaptive learning pathways. However, evaluating such adaptive systems using traditional temporal metrics remains challenging due to the inherent variability in Gen-AI response times. To address this, our study employs multimodal behavioural metrics, including visual attention, physical exploratory behaviour, and verbal interaction, to assess user engagement in an adaptive VR environment. In a controlled experiment with (n = 54) participants, we compared three levels of adaptivity (high, moderate, and... (More)
Virtual reality (VR) offers promising opportunities for procedural learning, particularly in preserving intangible cultural heritage. Advances in generative artificial intelligence (Gen-AI) further enrich these experiences by enabling adaptive learning pathways. However, evaluating such adaptive systems using traditional temporal metrics remains challenging due to the inherent variability in Gen-AI response times. To address this, our study employs multimodal behavioural metrics, including visual attention, physical exploratory behaviour, and verbal interaction, to assess user engagement in an adaptive VR environment. In a controlled experiment with (n = 54) participants, we compared three levels of adaptivity (high, moderate, and non-adaptive baseline) within a Neapolitan pizza-making VR experience. Results show that moderate adaptivity optimally enhances user engagement, significantly reducing unnecessary exploratory behaviour and increasing focused visual attention on the AI avatar. Our findings suggest that a balanced level of adaptive AI provides the most effective user support, offering practical design recommendations for future adaptive educational technologies.
(Less)
- author
- Lau, Ka Hei Carrie ; Sen, Sema ; Stark, Philipp LU ; Bozkir, Efe and Kasneci, Enkelejda
- organization
- publishing date
- 2025-10-12
- type
- Chapter in Book/Report/Conference proceeding
- publication status
- published
- subject
- keywords
- generative artificial intelligence, intangible culture heritage, virtual reality
- host publication
- ICMI 2025 : Proceedings of the 27th International Conference on Multimodal Interaction - Proceedings of the 27th International Conference on Multimodal Interaction
- series title
- ICMI 2025 - Proceedings of the 27th International Conference on Multimodal Interaction
- editor
- Subramanian, Ram ; Nakano, Yukiko I. ; Gedeon, Tom ; Kankanhalli, Mohan ; Guha, Tanaya ; Shukla, Jainendra ; Mohammadi, Gelareh and Celiktutan, Oya
- pages
- 12 pages
- publisher
- Association for Computing Machinery (ACM)
- conference name
- 27th International Conference on Multimodal Interaction, ICMI 2025
- conference location
- Canberra, Australia
- conference dates
- 2025-10-13 - 2025-10-17
- external identifiers
-
- scopus:105022237872
- ISBN
- 9798400714993
- DOI
- 10.1145/3716553.3750760
- language
- English
- LU publication?
- yes
- additional info
- Publisher Copyright: © 2025 Copyright held by the owner/author(s).
- id
- a8faefd7-738d-41d4-8d28-f5186f775a0f
- date added to LUP
- 2026-01-06 11:07:39
- date last changed
- 2026-01-07 08:49:44
@inproceedings{a8faefd7-738d-41d4-8d28-f5186f775a0f,
abstract = {{<p>Virtual reality (VR) offers promising opportunities for procedural learning, particularly in preserving intangible cultural heritage. Advances in generative artificial intelligence (Gen-AI) further enrich these experiences by enabling adaptive learning pathways. However, evaluating such adaptive systems using traditional temporal metrics remains challenging due to the inherent variability in Gen-AI response times. To address this, our study employs multimodal behavioural metrics, including visual attention, physical exploratory behaviour, and verbal interaction, to assess user engagement in an adaptive VR environment. In a controlled experiment with (n = 54) participants, we compared three levels of adaptivity (high, moderate, and non-adaptive baseline) within a Neapolitan pizza-making VR experience. Results show that moderate adaptivity optimally enhances user engagement, significantly reducing unnecessary exploratory behaviour and increasing focused visual attention on the AI avatar. Our findings suggest that a balanced level of adaptive AI provides the most effective user support, offering practical design recommendations for future adaptive educational technologies.</p>}},
author = {{Lau, Ka Hei Carrie and Sen, Sema and Stark, Philipp and Bozkir, Efe and Kasneci, Enkelejda}},
booktitle = {{ICMI 2025 : Proceedings of the 27th International Conference on Multimodal Interaction}},
editor = {{Subramanian, Ram and Nakano, Yukiko I. and Gedeon, Tom and Kankanhalli, Mohan and Guha, Tanaya and Shukla, Jainendra and Mohammadi, Gelareh and Celiktutan, Oya}},
isbn = {{9798400714993}},
keywords = {{generative artificial intelligence; intangible culture heritage; virtual reality}},
language = {{eng}},
month = {{10}},
pages = {{305--316}},
publisher = {{Association for Computing Machinery (ACM)}},
series = {{ICMI 2025 - Proceedings of the 27th International Conference on Multimodal Interaction}},
title = {{Adaptive Gen-AI Guidance in Virtual Reality : A Multimodal Exploration of Engagement in Neapolitan Pizza-Making}},
url = {{http://dx.doi.org/10.1145/3716553.3750760}},
doi = {{10.1145/3716553.3750760}},
year = {{2025}},
}