Skip to main content

LUP Student Papers

LUND UNIVERSITY LIBRARIES

Harnessing AI for Suicidal Ideation Detection: Thoroughly Evaluating and Fine-Tuning Transformer Models to Identify Suicidal Ideation in Social Media Posts

Altnäs, Johannes LU and Sompura, Keshav (2023) DABN01 20231
Department of Economics
Department of Statistics
Abstract
This thesis explored the application of pre-trained transformer models in detecting suicidal ideation in social media posts. We leveraged social media data from
platforms like Reddit and Twitter and applied a robust hyperparameter random
search strategy to fine-tune and evaluate existing transformer models. Despite noise
in the fine-tuning data, the models demonstrated high performance in identifying
posts about suicidal ideation. However, performance as measured by F1 and average precision scores decreased when the models where applied to more realistic
scenarios, indicating the need for inclusion of more general social media data during
fine-tuning. Despite this, most models retained high recall scores, capturing a majority of true... (More)
This thesis explored the application of pre-trained transformer models in detecting suicidal ideation in social media posts. We leveraged social media data from
platforms like Reddit and Twitter and applied a robust hyperparameter random
search strategy to fine-tune and evaluate existing transformer models. Despite noise
in the fine-tuning data, the models demonstrated high performance in identifying
posts about suicidal ideation. However, performance as measured by F1 and average precision scores decreased when the models where applied to more realistic
scenarios, indicating the need for inclusion of more general social media data during
fine-tuning. Despite this, most models retained high recall scores, capturing a majority of true suicidal ideation cases. Practical implications of these findings could influence the methods used by social media organizations and mental health professionals in identifying suicidal ideation, leading to faster interventions. To advance this research, future work should aim at obtaining more robustly annotated data, expanding the hyperparameter search range, and exploring other efficient model
architectures. (Less)
Please use this url to cite or link to this publication:
author
Altnäs, Johannes LU and Sompura, Keshav
supervisor
organization
course
DABN01 20231
year
type
H1 - Master's Degree (One Year)
subject
keywords
suicidal ideation, transformers, mental health, BERT, social media
language
English
id
9122311
date added to LUP
2023-11-21 12:53:21
date last changed
2023-11-21 12:53:21
@misc{9122311,
  abstract     = {{This thesis explored the application of pre-trained transformer models in detecting suicidal ideation in social media posts. We leveraged social media data from
platforms like Reddit and Twitter and applied a robust hyperparameter random
search strategy to fine-tune and evaluate existing transformer models. Despite noise
in the fine-tuning data, the models demonstrated high performance in identifying
posts about suicidal ideation. However, performance as measured by F1 and average precision scores decreased when the models where applied to more realistic
scenarios, indicating the need for inclusion of more general social media data during
fine-tuning. Despite this, most models retained high recall scores, capturing a majority of true suicidal ideation cases. Practical implications of these findings could influence the methods used by social media organizations and mental health professionals in identifying suicidal ideation, leading to faster interventions. To advance this research, future work should aim at obtaining more robustly annotated data, expanding the hyperparameter search range, and exploring other efficient model
architectures.}},
  author       = {{Altnäs, Johannes and Sompura, Keshav}},
  language     = {{eng}},
  note         = {{Student Paper}},
  title        = {{Harnessing AI for Suicidal Ideation Detection: Thoroughly Evaluating and Fine-Tuning Transformer Models to Identify Suicidal Ideation in Social Media Posts}},
  year         = {{2023}},
}