Using deep learning to predict ideology from facial photographs: expressions, beauty, and extra-facial information
(2023) In Scientific Reports 13(1).- Abstract
- Deep learning techniques can use public data such as facial photographs to predict sensitive personal information, but little is known about what information contributes to the predictive success of these techniques. This lack of knowledge limits both the public’s ability to protect against revealing unintended information as well as the scientific utility of deep learning results. We combine convolutional neural networks, heat maps, facial expression coding, and classification of identifiable features such as masculinity and attractiveness in our study of political ideology in 3323 Danes. Predictive accuracy from the neural network was 61% in each gender. Model-predicted ideology correlated with aspects of both facial expressions... (More)
- Deep learning techniques can use public data such as facial photographs to predict sensitive personal information, but little is known about what information contributes to the predictive success of these techniques. This lack of knowledge limits both the public’s ability to protect against revealing unintended information as well as the scientific utility of deep learning results. We combine convolutional neural networks, heat maps, facial expression coding, and classification of identifiable features such as masculinity and attractiveness in our study of political ideology in 3323 Danes. Predictive accuracy from the neural network was 61% in each gender. Model-predicted ideology correlated with aspects of both facial expressions (happiness vs neutrality) and morphology (specifically, attractiveness in females). Heat maps highlighted the informativeness of areas both on and off the face, pointing to methodological refinements and the need for future research to better understand the significance of certain facial areas. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/f31d6afd-3377-48f4-b35c-b7a1833e0472
- author
- Rasmussen, Stig HR ; Ludeke, Steven and Klemmensen, Robert LU
- organization
- publishing date
- 2023-03-31
- type
- Contribution to journal
- publication status
- published
- subject
- in
- Scientific Reports
- volume
- 13
- issue
- 1
- article number
- 5257
- pages
- 8 pages
- publisher
- Nature Publishing Group
- external identifiers
-
- scopus:85151374720
- pmid:37002240
- ISSN
- 2045-2322
- DOI
- 10.1038/s41598-023-31796-1
- language
- English
- LU publication?
- yes
- id
- f31d6afd-3377-48f4-b35c-b7a1833e0472
- alternative location
- https://www.nature.com/articles/s41598-023-31796-1
- date added to LUP
- 2023-04-02 21:41:59
- date last changed
- 2023-05-23 03:00:02
@article{f31d6afd-3377-48f4-b35c-b7a1833e0472,
abstract = {{Deep learning techniques can use public data such as facial photographs to predict sensitive personal information, but little is known about what information contributes to the predictive success of these techniques. This lack of knowledge limits both the public’s ability to protect against revealing unintended information as well as the scientific utility of deep learning results. We combine convolutional neural networks, heat maps, facial expression coding, and classification of identifiable features such as masculinity and attractiveness in our study of political ideology in 3323 Danes. Predictive accuracy from the neural network was 61% in each gender. Model-predicted ideology correlated with aspects of both facial expressions (happiness vs neutrality) and morphology (specifically, attractiveness in females). Heat maps highlighted the informativeness of areas both on and off the face, pointing to methodological refinements and the need for future research to better understand the significance of certain facial areas.}},
author = {{Rasmussen, Stig HR and Ludeke, Steven and Klemmensen, Robert}},
issn = {{2045-2322}},
language = {{eng}},
month = {{03}},
number = {{1}},
publisher = {{Nature Publishing Group}},
series = {{Scientific Reports}},
title = {{Using deep learning to predict ideology from facial photographs: expressions, beauty, and extra-facial information}},
url = {{http://dx.doi.org/10.1038/s41598-023-31796-1}},
doi = {{10.1038/s41598-023-31796-1}},
volume = {{13}},
year = {{2023}},
}