Hypothesis-testing demands trustworthy data-a simulation approach to inferential statistics advocating the research program strategy
Krefeld-Schwalb, Antonia ; Witte, Erich H. and Zenker, Frank LU
(2018)
In Frontiers in Psychology
9(APR).
- Abstract
In psychology as elsewhere, the main statistical inference strategy to establish empirical effects is null-hypothesis significance testing (NHST). The recent failure to replicate allegedly well-established NHST-results, however, implies that such results lack sufficient statistical power, and thus feature unacceptably high error-rates. Using data-simulation to estimate the error-rates of NHST-results, we advocate the research program strategy (RPS) as a superior methodology. RPS integrates Frequentist with Bayesian inference elements, and leads from a preliminary discovery against a (random) H0-hypothesis to a statistical H1-verification. Not only do RPS-results feature significantly lower error-rates than... (More)
In psychology as elsewhere, the main statistical inference strategy to establish empirical effects is null-hypothesis significance testing (NHST). The recent failure to replicate allegedly well-established NHST-results, however, implies that such results lack sufficient statistical power, and thus feature unacceptably high error-rates. Using data-simulation to estimate the error-rates of NHST-results, we advocate the research program strategy (RPS) as a superior methodology. RPS integrates Frequentist with Bayesian inference elements, and leads from a preliminary discovery against a (random) H0-hypothesis to a statistical H1-verification. Not only do RPS-results feature significantly lower error-rates than NHST-results, RPS also addresses key-deficits of a "pure" Frequentist and a standard Bayesian approach. In particular, RPS aggregates underpowered results safely. RPS therefore provides a tool to regain the trust the discipline had lost during the ongoing replicability-crisis.
(Less)
- author
- Krefeld-Schwalb, Antonia
; Witte, Erich H.
and Zenker, Frank
LU
- organization
- publishing date
- 2018-04-24
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- Bayes' theorem, Inferential statistics, Likelihood, Replication, Research program strategy, T-test, Wald criterion
- in
- Frontiers in Psychology
- volume
- 9
- issue
- APR
- article number
- 460
- publisher
- Frontiers Media S. A.
- external identifiers
-
- pmid:29740363
- scopus:85045933568
- ISSN
- 1664-1078
- DOI
- 10.3389/fpsyg.2018.00460
- language
- English
- LU publication?
- yes
- id
- f634df0f-0582-4d73-b490-25bcebed1c44
- date added to LUP
- 2018-05-04 08:23:27
- date last changed
- 2024-04-15 06:20:14
@article{f634df0f-0582-4d73-b490-25bcebed1c44,
abstract = {{<p>In psychology as elsewhere, the main statistical inference strategy to establish empirical effects is null-hypothesis significance testing (NHST). The recent failure to replicate allegedly well-established NHST-results, however, implies that such results lack sufficient statistical power, and thus feature unacceptably high error-rates. Using data-simulation to estimate the error-rates of NHST-results, we advocate the research program strategy (RPS) as a superior methodology. RPS integrates Frequentist with Bayesian inference elements, and leads from a preliminary discovery against a (random) H<sub>0</sub>-hypothesis to a statistical H<sub>1</sub>-verification. Not only do RPS-results feature significantly lower error-rates than NHST-results, RPS also addresses key-deficits of a "pure" Frequentist and a standard Bayesian approach. In particular, RPS aggregates underpowered results safely. RPS therefore provides a tool to regain the trust the discipline had lost during the ongoing replicability-crisis.</p>}},
author = {{Krefeld-Schwalb, Antonia and Witte, Erich H. and Zenker, Frank}},
issn = {{1664-1078}},
keywords = {{Bayes' theorem; Inferential statistics; Likelihood; Replication; Research program strategy; T-test; Wald criterion}},
language = {{eng}},
month = {{04}},
number = {{APR}},
publisher = {{Frontiers Media S. A.}},
series = {{Frontiers in Psychology}},
title = {{Hypothesis-testing demands trustworthy data-a simulation approach to inferential statistics advocating the research program strategy}},
url = {{http://dx.doi.org/10.3389/fpsyg.2018.00460}},
doi = {{10.3389/fpsyg.2018.00460}},
volume = {{9}},
year = {{2018}},
}