Non-Standard Errors
(2021) In Working Papers- Abstract
- In statistics, samples are drawn from a population in a data generating process (DGP). Standard errors measure the uncertainty in sample estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence generating process (EGP). We claim that EGP variation across researchers adds uncertainty: non-standard errors. To study them, we let 164 teams test six hypotheses on the same sample. We find that non-standard errors are sizeable, on par with standard errors. Their size (i) co-varies only weakly with team merits, reproducibility, or peer rating, (ii) declines significantly after peer-feedback, and (iii) is underestimated by participants.
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/84c08ec9-2e15-4559-ab7a-51a6edb9c2df
- author
- Menkveld, Albert J. ; Dreber, Anna ; Holzmeister, Felix ; Huber, Juergen ; Johannesson, Magnus ; Kirchler, Michael ; Neusüss, Sebastian ; Razen, Michael ; Weitzel, Utz and Wilhelmsson, Anders LU
- author collaboration
- organization
- publishing date
- 2021
- type
- Working paper/Preprint
- publication status
- published
- subject
- keywords
- Non-standard, errors, B26
- in
- Working Papers
- issue
- 2021:17
- pages
- 57 pages
- language
- English
- LU publication?
- yes
- id
- 84c08ec9-2e15-4559-ab7a-51a6edb9c2df
- date added to LUP
- 2021-12-15 10:41:17
- date last changed
- 2024-03-14 14:56:53
@misc{84c08ec9-2e15-4559-ab7a-51a6edb9c2df, abstract = {{In statistics, samples are drawn from a population in a data generating process (DGP). Standard errors measure the uncertainty in sample estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence generating process (EGP). We claim that EGP variation across researchers adds uncertainty: non-standard errors. To study them, we let 164 teams test six hypotheses on the same sample. We find that non-standard errors are sizeable, on par with standard errors. Their size (i) co-varies only weakly with team merits, reproducibility, or peer rating, (ii) declines significantly after peer-feedback, and (iii) is underestimated by participants.}}, author = {{Menkveld, Albert J. and Dreber, Anna and Holzmeister, Felix and Huber, Juergen and Johannesson, Magnus and Kirchler, Michael and Neusüss, Sebastian and Razen, Michael and Weitzel, Utz and Wilhelmsson, Anders}}, keywords = {{Non-standard; errors; B26}}, language = {{eng}}, note = {{Working Paper}}, number = {{2021:17}}, series = {{Working Papers}}, title = {{Non-Standard Errors}}, url = {{https://lup.lub.lu.se/search/files/177122721/WP21_17.pdf}}, year = {{2021}}, }