Harnessing AI for Suicidal Ideation Detection: Thoroughly Evaluating and Fine-Tuning Transformer Models to Identify Suicidal Ideation in Social Media Posts
(2023) DABN01 20231Department of Economics
Department of Statistics
- Abstract
- This thesis explored the application of pre-trained transformer models in detecting suicidal ideation in social media posts. We leveraged social media data from
platforms like Reddit and Twitter and applied a robust hyperparameter random
search strategy to fine-tune and evaluate existing transformer models. Despite noise
in the fine-tuning data, the models demonstrated high performance in identifying
posts about suicidal ideation. However, performance as measured by F1 and average precision scores decreased when the models where applied to more realistic
scenarios, indicating the need for inclusion of more general social media data during
fine-tuning. Despite this, most models retained high recall scores, capturing a majority of true... (More) - This thesis explored the application of pre-trained transformer models in detecting suicidal ideation in social media posts. We leveraged social media data from
platforms like Reddit and Twitter and applied a robust hyperparameter random
search strategy to fine-tune and evaluate existing transformer models. Despite noise
in the fine-tuning data, the models demonstrated high performance in identifying
posts about suicidal ideation. However, performance as measured by F1 and average precision scores decreased when the models where applied to more realistic
scenarios, indicating the need for inclusion of more general social media data during
fine-tuning. Despite this, most models retained high recall scores, capturing a majority of true suicidal ideation cases. Practical implications of these findings could influence the methods used by social media organizations and mental health professionals in identifying suicidal ideation, leading to faster interventions. To advance this research, future work should aim at obtaining more robustly annotated data, expanding the hyperparameter search range, and exploring other efficient model
architectures. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/9122311
- author
- Altnäs, Johannes LU and Sompura, Keshav
- supervisor
-
- Najmeh Abiri LU
- organization
- course
- DABN01 20231
- year
- 2023
- type
- H1 - Master's Degree (One Year)
- subject
- keywords
- suicidal ideation, transformers, mental health, BERT, social media
- language
- English
- id
- 9122311
- date added to LUP
- 2023-11-21 12:53:21
- date last changed
- 2023-11-21 12:53:21
@misc{9122311, abstract = {{This thesis explored the application of pre-trained transformer models in detecting suicidal ideation in social media posts. We leveraged social media data from platforms like Reddit and Twitter and applied a robust hyperparameter random search strategy to fine-tune and evaluate existing transformer models. Despite noise in the fine-tuning data, the models demonstrated high performance in identifying posts about suicidal ideation. However, performance as measured by F1 and average precision scores decreased when the models where applied to more realistic scenarios, indicating the need for inclusion of more general social media data during fine-tuning. Despite this, most models retained high recall scores, capturing a majority of true suicidal ideation cases. Practical implications of these findings could influence the methods used by social media organizations and mental health professionals in identifying suicidal ideation, leading to faster interventions. To advance this research, future work should aim at obtaining more robustly annotated data, expanding the hyperparameter search range, and exploring other efficient model architectures.}}, author = {{Altnäs, Johannes and Sompura, Keshav}}, language = {{eng}}, note = {{Student Paper}}, title = {{Harnessing AI for Suicidal Ideation Detection: Thoroughly Evaluating and Fine-Tuning Transformer Models to Identify Suicidal Ideation in Social Media Posts}}, year = {{2023}}, }