Advanced

Hypothesis-testing demands trustworthy data-a simulation approach to inferential statistics advocating the research program strategy

Krefeld-Schwalb, Antonia; Witte, Erich H. and Zenker, Frank LU (2018) In Frontiers in Psychology 9(APR).
Abstract

In psychology as elsewhere, the main statistical inference strategy to establish empirical effects is null-hypothesis significance testing (NHST). The recent failure to replicate allegedly well-established NHST-results, however, implies that such results lack sufficient statistical power, and thus feature unacceptably high error-rates. Using data-simulation to estimate the error-rates of NHST-results, we advocate the research program strategy (RPS) as a superior methodology. RPS integrates Frequentist with Bayesian inference elements, and leads from a preliminary discovery against a (random) H0-hypothesis to a statistical H1-verification. Not only do RPS-results feature significantly lower error-rates than... (More)

In psychology as elsewhere, the main statistical inference strategy to establish empirical effects is null-hypothesis significance testing (NHST). The recent failure to replicate allegedly well-established NHST-results, however, implies that such results lack sufficient statistical power, and thus feature unacceptably high error-rates. Using data-simulation to estimate the error-rates of NHST-results, we advocate the research program strategy (RPS) as a superior methodology. RPS integrates Frequentist with Bayesian inference elements, and leads from a preliminary discovery against a (random) H0-hypothesis to a statistical H1-verification. Not only do RPS-results feature significantly lower error-rates than NHST-results, RPS also addresses key-deficits of a "pure" Frequentist and a standard Bayesian approach. In particular, RPS aggregates underpowered results safely. RPS therefore provides a tool to regain the trust the discipline had lost during the ongoing replicability-crisis.

(Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Contribution to journal
publication status
published
subject
keywords
Bayes' theorem, Inferential statistics, Likelihood, Replication, Research program strategy, T-test, Wald criterion
in
Frontiers in Psychology
volume
9
issue
APR
publisher
Frontiers
external identifiers
  • scopus:85045933568
ISSN
1664-1078
DOI
10.3389/fpsyg.2018.00460
language
English
LU publication?
yes
id
f634df0f-0582-4d73-b490-25bcebed1c44
date added to LUP
2018-05-04 08:23:27
date last changed
2018-11-21 21:39:40
@article{f634df0f-0582-4d73-b490-25bcebed1c44,
  abstract     = {<p>In psychology as elsewhere, the main statistical inference strategy to establish empirical effects is null-hypothesis significance testing (NHST). The recent failure to replicate allegedly well-established NHST-results, however, implies that such results lack sufficient statistical power, and thus feature unacceptably high error-rates. Using data-simulation to estimate the error-rates of NHST-results, we advocate the research program strategy (RPS) as a superior methodology. RPS integrates Frequentist with Bayesian inference elements, and leads from a preliminary discovery against a (random) H<sub>0</sub>-hypothesis to a statistical H<sub>1</sub>-verification. Not only do RPS-results feature significantly lower error-rates than NHST-results, RPS also addresses key-deficits of a "pure" Frequentist and a standard Bayesian approach. In particular, RPS aggregates underpowered results safely. RPS therefore provides a tool to regain the trust the discipline had lost during the ongoing replicability-crisis.</p>},
  articleno    = {460},
  author       = {Krefeld-Schwalb, Antonia and Witte, Erich H. and Zenker, Frank},
  issn         = {1664-1078},
  keyword      = {Bayes' theorem,Inferential statistics,Likelihood,Replication,Research program strategy,T-test,Wald criterion},
  language     = {eng},
  month        = {04},
  number       = {APR},
  publisher    = {Frontiers},
  series       = {Frontiers in Psychology},
  title        = {Hypothesis-testing demands trustworthy data-a simulation approach to inferential statistics advocating the research program strategy},
  url          = {http://dx.doi.org/10.3389/fpsyg.2018.00460},
  volume       = {9},
  year         = {2018},
}