Hypothesis-testing demands trustworthy data-a simulation approach to inferential statistics advocating the research program strategy
(2018) In Frontiers in Psychology 9(APR).- Abstract
In psychology as elsewhere, the main statistical inference strategy to establish empirical effects is null-hypothesis significance testing (NHST). The recent failure to replicate allegedly well-established NHST-results, however, implies that such results lack sufficient statistical power, and thus feature unacceptably high error-rates. Using data-simulation to estimate the error-rates of NHST-results, we advocate the research program strategy (RPS) as a superior methodology. RPS integrates Frequentist with Bayesian inference elements, and leads from a preliminary discovery against a (random) H0-hypothesis to a statistical H1-verification. Not only do RPS-results feature significantly lower error-rates than... (More)
In psychology as elsewhere, the main statistical inference strategy to establish empirical effects is null-hypothesis significance testing (NHST). The recent failure to replicate allegedly well-established NHST-results, however, implies that such results lack sufficient statistical power, and thus feature unacceptably high error-rates. Using data-simulation to estimate the error-rates of NHST-results, we advocate the research program strategy (RPS) as a superior methodology. RPS integrates Frequentist with Bayesian inference elements, and leads from a preliminary discovery against a (random) H0-hypothesis to a statistical H1-verification. Not only do RPS-results feature significantly lower error-rates than NHST-results, RPS also addresses key-deficits of a "pure" Frequentist and a standard Bayesian approach. In particular, RPS aggregates underpowered results safely. RPS therefore provides a tool to regain the trust the discipline had lost during the ongoing replicability-crisis.
(Less)
- author
- Krefeld-Schwalb, Antonia ; Witte, Erich H. and Zenker, Frank LU
- organization
- publishing date
- 2018-04-24
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- Bayes' theorem, Inferential statistics, Likelihood, Replication, Research program strategy, T-test, Wald criterion
- in
- Frontiers in Psychology
- volume
- 9
- issue
- APR
- article number
- 460
- publisher
- Frontiers Media S. A.
- external identifiers
-
- pmid:29740363
- scopus:85045933568
- ISSN
- 1664-1078
- DOI
- 10.3389/fpsyg.2018.00460
- language
- English
- LU publication?
- yes
- id
- f634df0f-0582-4d73-b490-25bcebed1c44
- date added to LUP
- 2018-05-04 08:23:27
- date last changed
- 2024-09-02 19:58:15
@article{f634df0f-0582-4d73-b490-25bcebed1c44, abstract = {{<p>In psychology as elsewhere, the main statistical inference strategy to establish empirical effects is null-hypothesis significance testing (NHST). The recent failure to replicate allegedly well-established NHST-results, however, implies that such results lack sufficient statistical power, and thus feature unacceptably high error-rates. Using data-simulation to estimate the error-rates of NHST-results, we advocate the research program strategy (RPS) as a superior methodology. RPS integrates Frequentist with Bayesian inference elements, and leads from a preliminary discovery against a (random) H<sub>0</sub>-hypothesis to a statistical H<sub>1</sub>-verification. Not only do RPS-results feature significantly lower error-rates than NHST-results, RPS also addresses key-deficits of a "pure" Frequentist and a standard Bayesian approach. In particular, RPS aggregates underpowered results safely. RPS therefore provides a tool to regain the trust the discipline had lost during the ongoing replicability-crisis.</p>}}, author = {{Krefeld-Schwalb, Antonia and Witte, Erich H. and Zenker, Frank}}, issn = {{1664-1078}}, keywords = {{Bayes' theorem; Inferential statistics; Likelihood; Replication; Research program strategy; T-test; Wald criterion}}, language = {{eng}}, month = {{04}}, number = {{APR}}, publisher = {{Frontiers Media S. A.}}, series = {{Frontiers in Psychology}}, title = {{Hypothesis-testing demands trustworthy data-a simulation approach to inferential statistics advocating the research program strategy}}, url = {{http://dx.doi.org/10.3389/fpsyg.2018.00460}}, doi = {{10.3389/fpsyg.2018.00460}}, volume = {{9}}, year = {{2018}}, }