Natural Language Response Formats for Assessing Depression
(2023) 12th Conference of the International Society for Affective Disorders- Abstract
- OBJECTIVE:
Recent advances in AI-based Large Language Models (LLM) can transform individuals’ descriptions of emotional health into numeric representations that predict rating scales with an accuracy approaching theoretical upper limits. This advancement enables examing various open-ended response formats. We develop and evaluate formats ranging from closed to more open: 1) selecting descriptive words from a pre-defined list (the select format), and generating own 2) descriptive words (the word format), 3) descriptive phrases (the phrase format), or 4) descriptive texts (the text format).
MATERIAL AND METHODS:
Participants (N = 963) were recruited online to answer questions about their depression using the four response... (More) - OBJECTIVE:
Recent advances in AI-based Large Language Models (LLM) can transform individuals’ descriptions of emotional health into numeric representations that predict rating scales with an accuracy approaching theoretical upper limits. This advancement enables examing various open-ended response formats. We develop and evaluate formats ranging from closed to more open: 1) selecting descriptive words from a pre-defined list (the select format), and generating own 2) descriptive words (the word format), 3) descriptive phrases (the phrase format), or 4) descriptive texts (the text format).
MATERIAL AND METHODS:
Participants (N = 963) were recruited online to answer questions about their depression using the four response formats. They also completed depression rating scales, including the Patient Health Questionnaire-9 (PHQ-9). Language responses were first transformed into numeric representations using an LLM and then trained to predict the PHQ-9 using cross-validated ridge regression.
RESULTS:
The highest accuracy for the PHQ-9 score (Pearson correlation between out-of-sample predictions and observed scores) is achieved by combining all four formats (r=.76). In single-format predictions of the PHQ-9, the select format yields the highest (r=.73), followed by the phrase (r=.68), text (r=.68), and word (r=.67) formats. The select format has the fastest completion time (median = 30 sec), followed by the word (median = 67 sec), the phrase (median = 80 sec), and the text (median = 122 sec) formats.
CONCLUSION:
The response formats have advantages, with varying predictive accuracy on rating scales and different completion times. This provides the potential to select a response format according to situations and needs. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/534a168c-f594-4713-a4bb-433ea53b9ab7
- author
- Gu, Zhuojun LU ; Kjell, Katarina LU and Kjell, Oscar LU
- organization
- publishing date
- 2023-12-15
- type
- Contribution to conference
- publication status
- published
- subject
- keywords
- artificial intelligence, natural language, natural language processing, assessment, mental health
- conference name
- 12th Conference of the International Society for Affective Disorders
- conference location
- Milan, Italy
- conference dates
- 2023-12-14 - 2023-12-16
- language
- English
- LU publication?
- yes
- id
- 534a168c-f594-4713-a4bb-433ea53b9ab7
- alternative location
- https://www.isadconference.org/scientific-information/abstract-e-book/
- date added to LUP
- 2024-03-13 12:06:38
- date last changed
- 2024-03-20 16:03:29
@misc{534a168c-f594-4713-a4bb-433ea53b9ab7, abstract = {{OBJECTIVE:<br/>Recent advances in AI-based Large Language Models (LLM) can transform individuals’ descriptions of emotional health into numeric representations that predict rating scales with an accuracy approaching theoretical upper limits. This advancement enables examing various open-ended response formats. We develop and evaluate formats ranging from closed to more open: 1) selecting descriptive words from a pre-defined list (the select format), and generating own 2) descriptive words (the word format), 3) descriptive phrases (the phrase format), or 4) descriptive texts (the text format).<br/>MATERIAL AND METHODS:<br/>Participants (N = 963) were recruited online to answer questions about their depression using the four response formats. They also completed depression rating scales, including the Patient Health Questionnaire-9 (PHQ-9). Language responses were first transformed into numeric representations using an LLM and then trained to predict the PHQ-9 using cross-validated ridge regression.<br/>RESULTS:<br/>The highest accuracy for the PHQ-9 score (Pearson correlation between out-of-sample predictions and observed scores) is achieved by combining all four formats (r=.76). In single-format predictions of the PHQ-9, the select format yields the highest (r=.73), followed by the phrase (r=.68), text (r=.68), and word (r=.67) formats. The select format has the fastest completion time (median = 30 sec), followed by the word (median = 67 sec), the phrase (median = 80 sec), and the text (median = 122 sec) formats.<br/>CONCLUSION:<br/>The response formats have advantages, with varying predictive accuracy on rating scales and different completion times. This provides the potential to select a response format according to situations and needs.}}, author = {{Gu, Zhuojun and Kjell, Katarina and Kjell, Oscar}}, keywords = {{artificial intelligence; natural language; natural language processing; assessment; mental health}}, language = {{eng}}, month = {{12}}, title = {{Natural Language Response Formats for Assessing Depression}}, url = {{https://www.isadconference.org/scientific-information/abstract-e-book/}}, year = {{2023}}, }