Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Aggregation, trade-offs, and uncertainties in AI wellbeing

Kim, Jiwon LU orcid (2025) In Asian Journal of Philosophy 4. p.1-25
Abstract
This paper examines how, if artificial agents are capable of wellbeing, their wellbeing should be compared and aggregated alongside human wellbeing. Building on arguments from Goldstein and Kirk-Giannini, who suggest that some AI systems may possess wellbeing, I explore the moral implications of this possibility. Rather than reinventing debates in population ethics, this paper adapts and extends them to the context of AI wellbeing. I analyse three major approaches to wellbeing aggregation: symmetric methods, which treat human and AI wellbeing as equally significant; uncertainty-responsive methods, which discount AI wellbeing due to ontological, temporal, or identity uncertainty; and constraint-based views, which impose categorical... (More)
This paper examines how, if artificial agents are capable of wellbeing, their wellbeing should be compared and aggregated alongside human wellbeing. Building on arguments from Goldstein and Kirk-Giannini, who suggest that some AI systems may possess wellbeing, I explore the moral implications of this possibility. Rather than reinventing debates in population ethics, this paper adapts and extends them to the context of AI wellbeing. I analyse three major approaches to wellbeing aggregation: symmetric methods, which treat human and AI wellbeing as equally significant; uncertainty-responsive methods, which discount AI wellbeing due to ontological, temporal, or identity uncertainty; and constraint-based views, which impose categorical constraints on trading off human wellbeing for AI gains. These approaches are tested against thought experiments involving classic problems, such as the repugnant conclusion, infinitarian paralysis, and fanaticism. While utilitarian approaches risk endorsing troubling consequences when AI wellbeing scales indefinitely, constraint-based views may underweight the wellbeing of AI. A distinctive finding is that our intuitions shift depending on whether a human or an AI is a welfare subject. This reveals a potential asymmetry in our intuitive judgments, suggesting that species identity may itself be a morally salient feature that future theories of AI wellbeing should address. I conclude that uncertainty-responsive approaches, particularly those combining ontological, temporal, and identity-based discounting, offer a promising middle path that incorporates AI wellbeing into our moral calculus without letting it disproportionately outweigh human wellbeing in aggregation. (Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Contribution to journal
publication status
published
subject
keywords
AI wellbeing, Wellbeing aggregation, Repugnant Conclusion, Moral uncertainty, Population ethics
in
Asian Journal of Philosophy
volume
4
article number
93
pages
1 - 25
publisher
Springer
external identifiers
  • scopus:105013965252
ISSN
2731-4642
DOI
10.1007/s44204-025-00318-3
language
English
LU publication?
yes
id
6ce00d88-5163-42ec-ab41-fc0dc8369f6c
date added to LUP
2025-09-03 11:56:29
date last changed
2025-09-16 11:02:55
@article{6ce00d88-5163-42ec-ab41-fc0dc8369f6c,
  abstract     = {{This paper examines how, if artificial agents are capable of wellbeing, their wellbeing should be compared and aggregated alongside human wellbeing. Building on arguments from Goldstein and Kirk-Giannini, who suggest that some AI systems may possess wellbeing, I explore the moral implications of this possibility. Rather than reinventing debates in population ethics, this paper adapts and extends them to the context of AI wellbeing. I analyse three major approaches to wellbeing aggregation: symmetric methods, which treat human and AI wellbeing as equally significant; uncertainty-responsive methods, which discount AI wellbeing due to ontological, temporal, or identity uncertainty; and constraint-based views, which impose categorical constraints on trading off human wellbeing for AI gains. These approaches are tested against thought experiments involving classic problems, such as the repugnant conclusion, infinitarian paralysis, and fanaticism. While utilitarian approaches risk endorsing troubling consequences when AI wellbeing scales indefinitely, constraint-based views may underweight the wellbeing of AI. A distinctive finding is that  our intuitions shift depending on whether a human or an AI is a welfare subject. This reveals a potential asymmetry in our intuitive judgments, suggesting that species identity may itself be a morally salient feature that future theories of AI wellbeing should address. I conclude that uncertainty-responsive approaches, particularly those combining ontological, temporal, and identity-based discounting, offer a promising middle path that incorporates AI wellbeing into our moral calculus without letting it disproportionately outweigh human wellbeing in aggregation.}},
  author       = {{Kim, Jiwon}},
  issn         = {{2731-4642}},
  keywords     = {{AI wellbeing; Wellbeing aggregation; Repugnant Conclusion; Moral uncertainty; Population ethics}},
  language     = {{eng}},
  month        = {{08}},
  pages        = {{1--25}},
  publisher    = {{Springer}},
  series       = {{Asian Journal of Philosophy}},
  title        = {{Aggregation, trade-offs, and uncertainties in AI wellbeing}},
  url          = {{http://dx.doi.org/10.1007/s44204-025-00318-3}},
  doi          = {{10.1007/s44204-025-00318-3}},
  volume       = {{4}},
  year         = {{2025}},
}