Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Position: Every Ground Truth is a Human Construction, not an Objective Truth

Högberg, Charlotte LU orcid ; Johnson, Ericka and Wagstaff, Kiri L. (2026) 43rd International Conference on Machine Learning, ICML 306. p.1-13
Abstract
Ground truth datasets play a fundamental role as reference values in the training and evaluation of machine learning models. This position paper argues that ground truths are not neutral objective measurements that are naturally given, but instead that they are constructed by arrangements of humans and technologies. We argue that the ML community will benefit from articulating and discussing these often invisible or unreported choices and acknowledging that reference data sets are contingent, not universal. Focusing on the situated and context-dependent nature of ground truths can improve reliability by enabling a better informed perspective on where, when, and how the datasets, and the models they have shaped, can best be used. We argue... (More)
Ground truth datasets play a fundamental role as reference values in the training and evaluation of machine learning models. This position paper argues that ground truths are not neutral objective measurements that are naturally given, but instead that they are constructed by arrangements of humans and technologies. We argue that the ML community will benefit from articulating and discussing these often invisible or unreported choices and acknowledging that reference data sets are contingent, not universal. Focusing on the situated and context-dependent nature of ground truths can improve reliability by enabling a better informed perspective on where, when, and how the datasets, and the models they have shaped, can best be used. We argue for increasing ‘situated reliability’ which includes articulating the limits and strengths of models and their truth claims. Finally, paying more attention to the construction of
ground truths can support transparency, accountability, and interdisciplinary work. (Less)
Please use this url to cite or link to this publication:
author
; and
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
in press
subject
keywords
Ground truth, Machine learning, benchmark dataset, data
host publication
Proceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea
volume
306
pages
13 pages
publisher
PMLR
conference name
43rd International Conference on Machine Learning, ICML
conference location
Seoul, Korea, Republic of
conference dates
2026-07-06 - 2026-07-11
DOI
10.48550/arXiv.2607.09668
project
When N equals 1: Ethics and epistemology of personalisation and group belongings in data-driven prediction medicine
language
English
LU publication?
yes
id
833b9fcf-5d41-41b0-ad8f-61bee0d79973
date added to LUP
2026-05-03 20:09:36
date last changed
2026-08-10 10:09:52
@inproceedings{833b9fcf-5d41-41b0-ad8f-61bee0d79973,
  abstract     = {{Ground truth datasets play a fundamental role as reference values in the training and evaluation of machine learning models. This position paper argues that ground truths are not neutral objective measurements that are naturally given, but instead that they are constructed by arrangements of humans and technologies. We argue that the ML community will benefit from articulating and discussing these often invisible or unreported choices and acknowledging that reference data sets are contingent, not universal. Focusing on the situated and context-dependent nature of ground truths can improve reliability by enabling a better informed perspective on where, when, and how the datasets, and the models they have shaped, can best be used. We argue for increasing ‘situated reliability’ which includes articulating the limits and strengths of models and their truth claims. Finally, paying more attention to the construction of<br/>ground truths can support transparency, accountability, and interdisciplinary work.}},
  author       = {{Högberg, Charlotte and Johnson, Ericka and Wagstaff, Kiri L.}},
  booktitle    = {{Proceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea}},
  keywords     = {{Ground truth; Machine learning; benchmark dataset; data}},
  language     = {{eng}},
  month        = {{04}},
  pages        = {{1--13}},
  publisher    = {{PMLR}},
  title        = {{Position: Every Ground Truth is a Human Construction, not an Objective Truth}},
  url          = {{https://lup.lub.lu.se/search/files/253015983/Hogberg_Johnson_Wagstaff_Position_Every_Ground_Truth_is_a_Human_Construction_not_an_Objective_Truth.pdf}},
  doi          = {{10.48550/arXiv.2607.09668}},
  volume       = {{306}},
  year         = {{2026}},
}