Words Speak Louder Than Numbers: Post-intervention Language-Based Assessments Outperform Rating Scales' Accuracy in Classifying Different Affect Intervention Conditions.
(2023) PSYP01 20231Department of Psychology
- Abstract
- Language-based assessment (LBA) from open questions (e.g., how are you feeling?) analysed with natural language processing has predicted corresponding rating scales scores towards the theoretical upper limits. However, rating scales do not hold objective truth regarding psychological constructs. Therefore, the present pre-registered study set out to further validate LBA in a natural setting. Using an experimental design, participants (N = 153) completed a Mood Induction Procedure in different conditions (Church, shopping mall and a park). We aimed to compare the accuracy in classifying conditions based on post-intervention LBA of affect with the widely used Positive and Negative Affect Schedule (PANAS) rating scale. The results showed... (More)
- Language-based assessment (LBA) from open questions (e.g., how are you feeling?) analysed with natural language processing has predicted corresponding rating scales scores towards the theoretical upper limits. However, rating scales do not hold objective truth regarding psychological constructs. Therefore, the present pre-registered study set out to further validate LBA in a natural setting. Using an experimental design, participants (N = 153) completed a Mood Induction Procedure in different conditions (Church, shopping mall and a park). We aimed to compare the accuracy in classifying conditions based on post-intervention LBA of affect with the widely used Positive and Negative Affect Schedule (PANAS) rating scale. The results showed superior performance of LBA of post-intervention affect-phrases compared to PANAS scores. Specifically, the word embeddings exhibited significantly (p < .05) higher predictive power (AUC = .78) than the PANAS scores (AUC = .54) in classifying conditions. Additionally, data-driven word visualisation revealed underlying differences in affect from each post-intervention condition, showing the possibility of exploring nuances in emotions using LBA. Overall, all conditions revealed a general positive emotional state (e.g., happy and calm). However, in the church condition affect was described more in terms of positive and negative inner affective states (e.g., peaceful and uncomfortable) compared to the shopping mall and park where affect was more related to words that contextualise the affect (e.g., people and around). These findings emphasise the potential of incorporating LBA in psychological research to increase assessment validity. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/9133032
- author
- Bång, Oskar LU and De Leon Mendez, Flor LU
- supervisor
-
- Oscar Kjell LU
- organization
- course
- PSYP01 20231
- year
- 2023
- type
- H2 - Master's Degree (Two Years)
- subject
- keywords
- Language-based assessments, natural language processing, PANAS, mood induction procedure
- language
- English
- id
- 9133032
- date added to LUP
- 2023-08-02 10:28:59
- date last changed
- 2023-08-02 10:28:59
@misc{9133032,
abstract = {{Language-based assessment (LBA) from open questions (e.g., how are you feeling?) analysed with natural language processing has predicted corresponding rating scales scores towards the theoretical upper limits. However, rating scales do not hold objective truth regarding psychological constructs. Therefore, the present pre-registered study set out to further validate LBA in a natural setting. Using an experimental design, participants (N = 153) completed a Mood Induction Procedure in different conditions (Church, shopping mall and a park). We aimed to compare the accuracy in classifying conditions based on post-intervention LBA of affect with the widely used Positive and Negative Affect Schedule (PANAS) rating scale. The results showed superior performance of LBA of post-intervention affect-phrases compared to PANAS scores. Specifically, the word embeddings exhibited significantly (p < .05) higher predictive power (AUC = .78) than the PANAS scores (AUC = .54) in classifying conditions. Additionally, data-driven word visualisation revealed underlying differences in affect from each post-intervention condition, showing the possibility of exploring nuances in emotions using LBA. Overall, all conditions revealed a general positive emotional state (e.g., happy and calm). However, in the church condition affect was described more in terms of positive and negative inner affective states (e.g., peaceful and uncomfortable) compared to the shopping mall and park where affect was more related to words that contextualise the affect (e.g., people and around). These findings emphasise the potential of incorporating LBA in psychological research to increase assessment validity.}},
author = {{Bång, Oskar and De Leon Mendez, Flor}},
language = {{eng}},
note = {{Student Paper}},
title = {{Words Speak Louder Than Numbers: Post-intervention Language-Based Assessments Outperform Rating Scales' Accuracy in Classifying Different Affect Intervention Conditions.}},
year = {{2023}},
}