Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Non-Standard Errors

Menkveld, Albert J. ; Dreber, Anna ; Holzmeister, Felix ; Huber, Juergen ; Johannesson, Magnus ; Kirchler, Michael ; Neusüss, Sebastian ; Razen, Michael ; Weitzel, Utz and Wilhelmsson, Anders LU (2021) In Working Papers
Abstract
In statistics, samples are drawn from a population in a data generating process (DGP). Standard errors measure the uncertainty in sample estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence generating process (EGP). We claim that EGP variation across researchers adds uncertainty: non-standard errors. To study them, we let 164 teams test six hypotheses on the same sample. We find that non-standard errors are sizeable, on par with standard errors. Their size (i) co-varies only weakly with team merits, reproducibility, or peer rating, (ii) declines significantly after peer-feedback, and (iii) is underestimated by participants.
Please use this url to cite or link to this publication:
author
; ; ; ; ; ; ; ; and
author collaboration
organization
publishing date
type
Working paper/Preprint
publication status
published
subject
keywords
Non-standard, errors, B26
in
Working Papers
issue
2021:17
pages
57 pages
language
English
LU publication?
yes
id
84c08ec9-2e15-4559-ab7a-51a6edb9c2df
date added to LUP
2021-12-15 10:41:17
date last changed
2024-03-14 14:56:53
@misc{84c08ec9-2e15-4559-ab7a-51a6edb9c2df,
  abstract     = {{In statistics, samples are drawn from a population in a data generating process (DGP). Standard errors measure the uncertainty in sample estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence generating process (EGP). We claim that EGP variation across researchers adds uncertainty: non-standard errors. To study them, we let 164 teams test six hypotheses on the same sample. We find that non-standard errors are sizeable, on par with standard errors. Their size (i) co-varies only weakly with team merits, reproducibility, or peer rating, (ii) declines significantly after peer-feedback, and (iii) is underestimated by participants.}},
  author       = {{Menkveld, Albert J. and Dreber, Anna and Holzmeister, Felix and Huber, Juergen and Johannesson, Magnus and Kirchler, Michael and Neusüss, Sebastian and Razen, Michael and Weitzel, Utz and Wilhelmsson, Anders}},
  keywords     = {{Non-standard; errors; B26}},
  language     = {{eng}},
  note         = {{Working Paper}},
  number       = {{2021:17}},
  series       = {{Working Papers}},
  title        = {{Non-Standard Errors}},
  url          = {{https://lup.lub.lu.se/search/files/177122721/WP21_17.pdf}},
  year         = {{2021}},
}