Skip to main content

LUP Student Papers

LUND UNIVERSITY LIBRARIES

Multi-Label Toxic Comment Classification Using Machine Learning: An In-Depth Study

Froste, Matilda LU and Hosseini, Mosa LU (2023) In LU-CS-EX EDAM05 20231
Department of Computer Science
Abstract
The classification of toxic comments is a well-researched area with many techniques available. However, effectively managing multi-label categorization still requires a considerable amount of work. In this thesis, we performed a classification experiment on over 200 thousand comments from the Jigsaw toxic comment competition data available on Kaggle. We aimed to optimize a model to identify six different categories of hate speech. Initially, we implemented a baseline model using a simple vectorization technique and logistic regression. Subsequently, we compared this model with more advanced approaches that employed elaborate vectorization techniques in conjunction with recurrent neural networks and transformers. After thorough analysis, we... (More)
The classification of toxic comments is a well-researched area with many techniques available. However, effectively managing multi-label categorization still requires a considerable amount of work. In this thesis, we performed a classification experiment on over 200 thousand comments from the Jigsaw toxic comment competition data available on Kaggle. We aimed to optimize a model to identify six different categories of hate speech. Initially, we implemented a baseline model using a simple vectorization technique and logistic regression. Subsequently, we compared this model with more advanced approaches that employed elaborate vectorization techniques in conjunction with recurrent neural networks and transformers. After thorough analysis, we found that a fine-tuned transformer-based model called RoBERTa yielded the best performance, achieving a mean macro average F1-score of 0.808. This model surpassed the previous state-of-the-art set by van Aken et al. (2018), which achieved an F1 score of 0.791. Finally, we integrated the optimized model in a web application to visualize the toxicity of messages. (Less)
Please use this url to cite or link to this publication:
author
Froste, Matilda LU and Hosseini, Mosa LU
supervisor
organization
alternative title
Multi-Label klassificering av hatiska kommentarer: en omfattande studie
course
EDAM05 20231
year
type
H2 - Master's Degree (Two Years)
subject
keywords
natural language processing, machine learning, offensive speech detection, transformers, multi-label classification
publication/series
LU-CS-EX
report number
2023-27
ISSN
1650-2884
language
English
id
9132776
date added to LUP
2023-09-01 10:01:51
date last changed
2023-09-01 10:01:51
@misc{9132776,
  abstract     = {{The classification of toxic comments is a well-researched area with many techniques available. However, effectively managing multi-label categorization still requires a considerable amount of work. In this thesis, we performed a classification experiment on over 200 thousand comments from the Jigsaw toxic comment competition data available on Kaggle. We aimed to optimize a model to identify six different categories of hate speech. Initially, we implemented a baseline model using a simple vectorization technique and logistic regression. Subsequently, we compared this model with more advanced approaches that employed elaborate vectorization techniques in conjunction with recurrent neural networks and transformers. After thorough analysis, we found that a fine-tuned transformer-based model called RoBERTa yielded the best performance, achieving a mean macro average F1-score of 0.808. This model surpassed the previous state-of-the-art set by van Aken et al. (2018), which achieved an F1 score of 0.791. Finally, we integrated the optimized model in a web application to visualize the toxicity of messages.}},
  author       = {{Froste, Matilda and Hosseini, Mosa}},
  issn         = {{1650-2884}},
  language     = {{eng}},
  note         = {{Student Paper}},
  series       = {{LU-CS-EX}},
  title        = {{Multi-Label Toxic Comment Classification Using Machine Learning: An In-Depth Study}},
  year         = {{2023}},
}