Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Natural Language Response Formats for Assessing Depression

Gu, Zhuojun LU ; Kjell, Katarina LU and Kjell, Oscar LU (2023) 12th Conference of the International Society for Affective Disorders
Abstract
OBJECTIVE:
Recent advances in AI-based Large Language Models (LLM) can transform individuals’ descriptions of emotional health into numeric representations that predict rating scales with an accuracy approaching theoretical upper limits. This advancement enables examing various open-ended response formats. We develop and evaluate formats ranging from closed to more open: 1) selecting descriptive words from a pre-defined list (the select format), and generating own 2) descriptive words (the word format), 3) descriptive phrases (the phrase format), or 4) descriptive texts (the text format).
MATERIAL AND METHODS:
Participants (N = 963) were recruited online to answer questions about their depression using the four response... (More)
OBJECTIVE:
Recent advances in AI-based Large Language Models (LLM) can transform individuals’ descriptions of emotional health into numeric representations that predict rating scales with an accuracy approaching theoretical upper limits. This advancement enables examing various open-ended response formats. We develop and evaluate formats ranging from closed to more open: 1) selecting descriptive words from a pre-defined list (the select format), and generating own 2) descriptive words (the word format), 3) descriptive phrases (the phrase format), or 4) descriptive texts (the text format).
MATERIAL AND METHODS:
Participants (N = 963) were recruited online to answer questions about their depression using the four response formats. They also completed depression rating scales, including the Patient Health Questionnaire-9 (PHQ-9). Language responses were first transformed into numeric representations using an LLM and then trained to predict the PHQ-9 using cross-validated ridge regression.
RESULTS:
The highest accuracy for the PHQ-9 score (Pearson correlation between out-of-sample predictions and observed scores) is achieved by combining all four formats (r=.76). In single-format predictions of the PHQ-9, the select format yields the highest (r=.73), followed by the phrase (r=.68), text (r=.68), and word (r=.67) formats. The select format has the fastest completion time (median = 30 sec), followed by the word (median = 67 sec), the phrase (median = 80 sec), and the text (median = 122 sec) formats.
CONCLUSION:
The response formats have advantages, with varying predictive accuracy on rating scales and different completion times. This provides the potential to select a response format according to situations and needs. (Less)
Please use this url to cite or link to this publication:
author
; and
organization
publishing date
type
Contribution to conference
publication status
published
subject
keywords
artificial intelligence, natural language, natural language processing, assessment, mental health
conference name
12th Conference of the International Society for Affective Disorders
conference location
Milan, Italy
conference dates
2023-12-14 - 2023-12-16
language
English
LU publication?
yes
id
534a168c-f594-4713-a4bb-433ea53b9ab7
alternative location
https://www.isadconference.org/scientific-information/abstract-e-book/
date added to LUP
2024-03-13 12:06:38
date last changed
2024-03-20 16:03:29
@misc{534a168c-f594-4713-a4bb-433ea53b9ab7,
  abstract     = {{OBJECTIVE:<br/>Recent advances in AI-based Large Language Models (LLM) can transform individuals’ descriptions of emotional health into numeric representations that predict rating scales with an accuracy approaching theoretical upper limits. This advancement enables examing various open-ended response formats. We develop and evaluate formats ranging from closed to more open: 1) selecting descriptive words from a pre-defined list (the select format), and generating own 2) descriptive words (the word format), 3) descriptive phrases (the phrase format), or 4) descriptive texts (the text format).<br/>MATERIAL AND METHODS:<br/>Participants (N = 963) were recruited online to answer questions about their depression using the four response formats. They also completed depression rating scales, including the Patient Health Questionnaire-9 (PHQ-9). Language responses were first transformed into numeric representations using an LLM and then trained to predict the PHQ-9 using cross-validated ridge regression.<br/>RESULTS:<br/>The highest accuracy for the PHQ-9 score (Pearson correlation between out-of-sample predictions and observed scores) is achieved by combining all four formats (r=.76). In single-format predictions of the PHQ-9, the select format yields the highest (r=.73), followed by the phrase (r=.68), text (r=.68), and word (r=.67) formats. The select format has the fastest completion time (median = 30 sec), followed by the word (median = 67 sec), the phrase (median = 80 sec), and the text (median = 122 sec) formats.<br/>CONCLUSION:<br/>The response formats have advantages, with varying predictive accuracy on rating scales and different completion times. This provides the potential to select a response format according to situations and needs.}},
  author       = {{Gu, Zhuojun and Kjell, Katarina and Kjell, Oscar}},
  keywords     = {{artificial intelligence; natural language; natural language processing; assessment; mental health}},
  language     = {{eng}},
  month        = {{12}},
  title        = {{Natural Language Response Formats for Assessing Depression}},
  url          = {{https://www.isadconference.org/scientific-information/abstract-e-book/}},
  year         = {{2023}},
}