Evaluation of stance annotation of Twitter data
Simaki, Vasiliki LU ; Seitanidi, Eleni LU
and Paradis, Carita
LU
(2023)
In Research in Corpus Linguistics
11(1).
p.53-80
- Abstract
- Taking stance towards any topic, event or idea is a common phenomenon on Twitter and social media in general. Twitter users express their opinions about different matters and assess other people’s opinions in various discursive ways. The identification and analysis of the linguistic ways that people use to take different stances leads to a better understanding of the language and user behaviour on Twitter. Stance is a multidimensional concept involving a broad range of related notions such as modality, evaluation and sentiment. In this study, we annotate data from Twitter using six notional stance categories —contrariety, hypotheticality, necessity, prediction, source of knowledge and uncertainty— following a comprehensive annotation... (More)
- Taking stance towards any topic, event or idea is a common phenomenon on Twitter and social media in general. Twitter users express their opinions about different matters and assess other people’s opinions in various discursive ways. The identification and analysis of the linguistic ways that people use to take different stances leads to a better understanding of the language and user behaviour on Twitter. Stance is a multidimensional concept involving a broad range of related notions such as modality, evaluation and sentiment. In this study, we annotate data from Twitter using six notional stance categories —contrariety, hypotheticality, necessity, prediction, source of knowledge and uncertainty— following a comprehensive annotation protocol including inter-coder reliability measurements. The relatively low agreement between annotators highlighted the challenges that the task entailed, which made us question the inter-annotator agreement score as a reliable measurement of annotation quality of notional categories. The nature of the data, the difficulty of the stance annotation task and the type of stance categories are discussed, and potential solutions are suggested. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/52f7bfd1-f531-4b30-b3c7-f856dd9bdc72
- author
- Simaki, Vasiliki
LU
; Seitanidi, Eleni
LU
and Paradis, Carita
LU
- organization
- publishing date
- 2023
- type
- Contribution to journal
- publication status
- published
- subject
- in
- Research in Corpus Linguistics
- volume
- 11
- issue
- 1
- pages
- 38 pages
- publisher
- Spanish Association for Corpus Linguistics
- external identifiers
-
- scopus:85162841694
- ISSN
- 2243-4712
- DOI
- 10.32714/ricl.11.01.03
- project
- Language as a tool for understanding consumer attitudes and improving the social impact of sustainable products
- language
- English
- LU publication?
- yes
- id
- 52f7bfd1-f531-4b30-b3c7-f856dd9bdc72
- alternative location
- http://ricl.aelinco.es/first-view/245-Article%20Text-1798-1-10-20221207.pdf
- date added to LUP
- 2022-11-29 08:26:32
- date last changed
- 2024-08-13 14:08:47
@article{52f7bfd1-f531-4b30-b3c7-f856dd9bdc72,
abstract = {{Taking stance towards any topic, event or idea is a common phenomenon on Twitter and social media in general. Twitter users express their opinions about different matters and assess other people’s opinions in various discursive ways. The identification and analysis of the linguistic ways that people use to take different stances leads to a better understanding of the language and user behaviour on Twitter. Stance is a multidimensional concept involving a broad range of related notions such as modality, evaluation and sentiment. In this study, we annotate data from Twitter using six notional stance categories —contrariety, hypotheticality, necessity, prediction, source of knowledge and uncertainty— following a comprehensive annotation protocol including inter-coder reliability measurements. The relatively low agreement between annotators highlighted the challenges that the task entailed, which made us question the inter-annotator agreement score as a reliable measurement of annotation quality of notional categories. The nature of the data, the difficulty of the stance annotation task and the type of stance categories are discussed, and potential solutions are suggested.}},
author = {{Simaki, Vasiliki and Seitanidi, Eleni and Paradis, Carita}},
issn = {{2243-4712}},
language = {{eng}},
number = {{1}},
pages = {{53--80}},
publisher = {{Spanish Association for Corpus Linguistics}},
series = {{Research in Corpus Linguistics}},
title = {{Evaluation of stance annotation of <i>Twitter</i> data}},
url = {{http://dx.doi.org/10.32714/ricl.11.01.03}},
doi = {{10.32714/ricl.11.01.03}},
volume = {{11}},
year = {{2023}},
}