Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Natural language analyzed with AI-based transformers predict traditional subjective well-being measures approaching the theoretical upper limits in accuracy

Kjell, Oscar N E LU orcid ; Sikström, Sverker LU orcid ; Kjell, Katarina LU and Schwartz, H Andrew (2022) In Scientific Reports 12(1). p.3918-3918
Abstract

We show that using a recent break-through in artificial intelligence -transformers-, psychological assessments from text-responses can approach theoretical upper limits in accuracy, converging with standard psychological rating scales. Text-responses use people's primary form of communication -natural language- and have been suggested as a more ecologically-valid response format than closed-ended rating scales that dominate social science. However, previous language analysis techniques left a gap between how accurately they converged with standard rating scales and how well ratings scales converge with themselves - a theoretical upper-limit in accuracy. Most recently, AI-based language analysis has gone through a transformation as... (More)

We show that using a recent break-through in artificial intelligence -transformers-, psychological assessments from text-responses can approach theoretical upper limits in accuracy, converging with standard psychological rating scales. Text-responses use people's primary form of communication -natural language- and have been suggested as a more ecologically-valid response format than closed-ended rating scales that dominate social science. However, previous language analysis techniques left a gap between how accurately they converged with standard rating scales and how well ratings scales converge with themselves - a theoretical upper-limit in accuracy. Most recently, AI-based language analysis has gone through a transformation as nearly all of its applications, from Web search to personalized assistants (e.g., Alexa and Siri), have shown unprecedented improvement by using transformers. We evaluate transformers for estimating psychological well-being from questionnaire text- and descriptive word-responses, and find accuracies converging with rating scales that approach the theoretical upper limits (Pearson r = 0.85, p < 0.001, N = 608; in line with most metrics of rating scale reliability). These findings suggest an avenue for modernizing the ubiquitous questionnaire and ultimately opening doors to a greater understanding of the human condition.

(Less)
Please use this url to cite or link to this publication:
author
; ; and
organization
publishing date
type
Contribution to journal
publication status
published
subject
in
Scientific Reports
volume
12
issue
1
pages
3918 - 3918
publisher
Nature Publishing Group
external identifiers
  • scopus:85126232254
  • pmid:35273198
ISSN
2045-2322
DOI
10.1038/s41598-022-07520-w
language
English
LU publication?
yes
additional info
© 2022. The Author(s).
id
12755e98-789a-46c9-a90b-7186a52fc802
date added to LUP
2022-03-17 13:04:59
date last changed
2025-03-08 18:02:56
@article{12755e98-789a-46c9-a90b-7186a52fc802,
  abstract     = {{<p>We show that using a recent break-through in artificial intelligence -transformers-, psychological assessments from text-responses can approach theoretical upper limits in accuracy, converging with standard psychological rating scales. Text-responses use people's primary form of communication -natural language- and have been suggested as a more ecologically-valid response format than closed-ended rating scales that dominate social science. However, previous language analysis techniques left a gap between how accurately they converged with standard rating scales and how well ratings scales converge with themselves - a theoretical upper-limit in accuracy. Most recently, AI-based language analysis has gone through a transformation as nearly all of its applications, from Web search to personalized assistants (e.g., Alexa and Siri), have shown unprecedented improvement by using transformers. We evaluate transformers for estimating psychological well-being from questionnaire text- and descriptive word-responses, and find accuracies converging with rating scales that approach the theoretical upper limits (Pearson r = 0.85, p &lt; 0.001, N = 608; in line with most metrics of rating scale reliability). These findings suggest an avenue for modernizing the ubiquitous questionnaire and ultimately opening doors to a greater understanding of the human condition.</p>}},
  author       = {{Kjell, Oscar N E and Sikström, Sverker and Kjell, Katarina and Schwartz, H Andrew}},
  issn         = {{2045-2322}},
  language     = {{eng}},
  month        = {{03}},
  number       = {{1}},
  pages        = {{3918--3918}},
  publisher    = {{Nature Publishing Group}},
  series       = {{Scientific Reports}},
  title        = {{Natural language analyzed with AI-based transformers predict traditional subjective well-being measures approaching the theoretical upper limits in accuracy}},
  url          = {{http://dx.doi.org/10.1038/s41598-022-07520-w}},
  doi          = {{10.1038/s41598-022-07520-w}},
  volume       = {{12}},
  year         = {{2022}},
}