Fair Finance: Assessing Geopolitical Bias in Transformer-Based NLP Models
(2025) DABN01 20251Department of Economics
Department of Statistics
- Abstract
- Transformer-based sentiment analysis systems have become widespread in automated decision-making, particularly in the field of financial sentiment analysis. These models are easy to implement, inexpensive, and offer state-of-the-art performance in sentiment analysis. However, their rapid and broad adoption calls for careful evaluation of algorithmic bias in such models. Previous research has shown that transformer-based systems can exhibit bias related to gender and other forms of social bias. Despite this, limited attention has been paid to bias in NLP systems within the financial domain. This paper investigates whether transformer-based sentiment analysis models show directional bias in relation to the geographic framing of financial... (More)
- Transformer-based sentiment analysis systems have become widespread in automated decision-making, particularly in the field of financial sentiment analysis. These models are easy to implement, inexpensive, and offer state-of-the-art performance in sentiment analysis. However, their rapid and broad adoption calls for careful evaluation of algorithmic bias in such models. Previous research has shown that transformer-based systems can exhibit bias related to gender and other forms of social bias. Despite this, limited attention has been paid to bias in NLP systems within the financial domain. This paper investigates whether transformer-based sentiment analysis models show directional bias in relation to the geographic framing of financial news headlines. It presents a structured approach, adapted from previous fairness evaluations of sentiment analysis, which generates templates used to create headline variations that differ only in geographical framing. Using a manually curated set of financial news headlines, a large corpus suitable for bias testing is constructed. This corpus is used to test three sentiment analysis models: FinBERT, RoBERTa, and base-BERT. The results show statistically significant differences in sentiment scores based on geographic framing across all three models. FinBERT and RoBERTa both exhibit directional bias, but in opposite directions: FinBERT tends to assign more positive scores to Eastern headlines, while RoBERTa scores Western headlines more positively. In contrast, base-BERT remains neutral in terms of polarity. These findings suggest that such biases may emerge during fine-tuning rather than pre-training. In addition, the study proposes an interpretable ILR transformation for common sentiment model outputs, using a custom orthonormal basis. The findings highlight the importance of including bias testing in model development and evaluation. They also call for more research into biases beyond gender and social dimensions, including further work on bias mitigation. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/9198250
- author
- Fröderberg, Emil LU
- supervisor
- organization
- course
- DABN01 20251
- year
- 2025
- type
- H1 - Master's Degree (One Year)
- subject
- keywords
- Algorithmic Bias, Transformer Models, Financial NLP, Counterfactual Fairness, FinBERT
- language
- English
- id
- 9198250
- date added to LUP
- 2025-09-12 09:04:21
- date last changed
- 2025-09-12 09:04:21
@misc{9198250,
abstract = {{Transformer-based sentiment analysis systems have become widespread in automated decision-making, particularly in the field of financial sentiment analysis. These models are easy to implement, inexpensive, and offer state-of-the-art performance in sentiment analysis. However, their rapid and broad adoption calls for careful evaluation of algorithmic bias in such models. Previous research has shown that transformer-based systems can exhibit bias related to gender and other forms of social bias. Despite this, limited attention has been paid to bias in NLP systems within the financial domain. This paper investigates whether transformer-based sentiment analysis models show directional bias in relation to the geographic framing of financial news headlines. It presents a structured approach, adapted from previous fairness evaluations of sentiment analysis, which generates templates used to create headline variations that differ only in geographical framing. Using a manually curated set of financial news headlines, a large corpus suitable for bias testing is constructed. This corpus is used to test three sentiment analysis models: FinBERT, RoBERTa, and base-BERT. The results show statistically significant differences in sentiment scores based on geographic framing across all three models. FinBERT and RoBERTa both exhibit directional bias, but in opposite directions: FinBERT tends to assign more positive scores to Eastern headlines, while RoBERTa scores Western headlines more positively. In contrast, base-BERT remains neutral in terms of polarity. These findings suggest that such biases may emerge during fine-tuning rather than pre-training. In addition, the study proposes an interpretable ILR transformation for common sentiment model outputs, using a custom orthonormal basis. The findings highlight the importance of including bias testing in model development and evaluation. They also call for more research into biases beyond gender and social dimensions, including further work on bias mitigation.}},
author = {{Fröderberg, Emil}},
language = {{eng}},
note = {{Student Paper}},
title = {{Fair Finance: Assessing Geopolitical Bias in Transformer-Based NLP Models}},
year = {{2025}},
}