Advanced

Analysis of size and power: Comparing the t-test, bootstrap, and permutation test

Haas, Brett and Yang, Dong (2009)
Department of Statistics
Abstract
This thesis compares three different tests by analyzing the difference in sample means between two different data samples, both with small sample sizes. The three tests being examined here are the t-test, nonparametric bootstrap, and permutation test. These tests are employed under several different circumstances, such as varying sample sizes, means, underlying distributions, and variances in the generated data. The goal is to examine the power and size of these tests and see if any of these tests in particular is more accurate under any of the circumstances set in the generated data. After examining all of these different cases, real data is used, and the previous results help determine which test is most effective in terms of size and... (More)
This thesis compares three different tests by analyzing the difference in sample means between two different data samples, both with small sample sizes. The three tests being examined here are the t-test, nonparametric bootstrap, and permutation test. These tests are employed under several different circumstances, such as varying sample sizes, means, underlying distributions, and variances in the generated data. The goal is to examine the power and size of these tests and see if any of these tests in particular is more accurate under any of the circumstances set in the generated data. After examining all of these different cases, real data is used, and the previous results help determine which test is most effective in terms of size and power. The conclusion will be that the permutation test yields more accurate results in terms of size and power when the distributions are lognormal or exponential. Bootstrap shows a pattern of over-rejection when the underlying distributions are lognormal or exponential, but accurate results when examining samples from normal distributions. The t-test shows an under-rejection when the distributions are lognormal. (Less)
Please use this url to cite or link to this publication:
author
Haas, Brett and Yang, Dong
supervisor
organization
year
type
H1 - Master's Degree (One Year)
subject
keywords
bootstrap, non-parametric tests, permutation test, size-power curve, Statistics, operations research, programming, actuarial mathematics, Statistik, operationsanalys, programmering, aktuariematematik
language
English
id
1644283
date added to LUP
2009-08-17 00:00:00
date last changed
2010-08-05 13:20:39
@misc{1644283,
  abstract     = {This thesis compares three different tests by analyzing the difference in sample means between two different data samples, both with small sample sizes. The three tests being examined here are the t-test, nonparametric bootstrap, and permutation test. These tests are employed under several different circumstances, such as varying sample sizes, means, underlying distributions, and variances in the generated data. The goal is to examine the power and size of these tests and see if any of these tests in particular is more accurate under any of the circumstances set in the generated data. After examining all of these different cases, real data is used, and the previous results help determine which test is most effective in terms of size and power. The conclusion will be that the permutation test yields more accurate results in terms of size and power when the distributions are lognormal or exponential. Bootstrap shows a pattern of over-rejection when the underlying distributions are lognormal or exponential, but accurate results when examining samples from normal distributions. The t-test shows an under-rejection when the distributions are lognormal.},
  author       = {Haas, Brett and Yang, Dong},
  keyword      = {bootstrap,non-parametric tests,permutation test,size-power curve,Statistics, operations research, programming, actuarial mathematics,Statistik, operationsanalys, programmering, aktuariematematik},
  language     = {eng},
  note         = {Student Paper},
  title        = {Analysis of size and power: Comparing the t-test, bootstrap, and permutation test},
  year         = {2009},
}