Advanced

Assessing vocabulary size through multiple-choice formats : Issues with guessing and sampling rates

Gyllstad, Henrik LU ; Vilkaite, Laura and Schmitt, Norbert (2015) In ITL: Institut Voor Toegepaste Linguistik 166(2). p.278-306
Abstract
In most tests of vocabulary size, knowledge is assessed through multiple-choice formats. Despite advantages such as ease of scoring, multiple-choice tests (MCT) are accompanied with problems. One of the more central issues has to do with guessing and the presence of other construct-irrelevant strategies that can lead to overestimation of scores. A further challenge when designing vocabulary size tests is that of sampling rate. How many words constitute a representative sample of the underlying population of words that the test is intended to measure? This paper addresses these two issues through a case study based on data from a recent and increasingly used MCT of vocabulary size: the Vocabulary Size Test. Using a criterion-related... (More)
In most tests of vocabulary size, knowledge is assessed through multiple-choice formats. Despite advantages such as ease of scoring, multiple-choice tests (MCT) are accompanied with problems. One of the more central issues has to do with guessing and the presence of other construct-irrelevant strategies that can lead to overestimation of scores. A further challenge when designing vocabulary size tests is that of sampling rate. How many words constitute a representative sample of the underlying population of words that the test is intended to measure? This paper addresses these two issues through a case study based on data from a recent and increasingly used MCT of vocabulary size: the Vocabulary Size Test. Using a criterion-related validity approach, our results show that for multiple-choice items sampled from this test, there is a discrepancy between the test scores and the scores obtained from the criterion measure, and that a higher sampling rate would be needed in order to better represent knowledge of the underlying population of words. We offer two main interpretations of these results, and discuss their implications for the construction and use of vocabulary size tests. (Less)
Please use this url to cite or link to this publication:
author
organization
publishing date
type
Contribution to journal
publication status
published
subject
keywords
sampling rate, testing, validation, vocabulary size, guessing, multiple-choice test, criterion-related validity, assessment
in
ITL: Institut Voor Toegepaste Linguistik
volume
166
issue
2
pages
278 - 306
publisher
John Benjamins Publishing Company
ISSN
0019-0829
DOI
10.1075/itl.166.2.04gyl
language
English
LU publication?
yes
id
dc288c1b-b998-40c3-8274-d3129fee0af3 (old id 5047033)
date added to LUP
2015-02-10 11:54:41
date last changed
2016-04-16 00:35:43
@article{dc288c1b-b998-40c3-8274-d3129fee0af3,
  abstract     = {In most tests of vocabulary size, knowledge is assessed through multiple-choice formats. Despite advantages such as ease of scoring, multiple-choice tests (MCT) are accompanied with problems. One of the more central issues has to do with guessing and the presence of other construct-irrelevant strategies that can lead to overestimation of scores. A further challenge when designing vocabulary size tests is that of sampling rate. How many words constitute a representative sample of the underlying population of words that the test is intended to measure? This paper addresses these two issues through a case study based on data from a recent and increasingly used MCT of vocabulary size: the Vocabulary Size Test. Using a criterion-related validity approach, our results show that for multiple-choice items sampled from this test, there is a discrepancy between the test scores and the scores obtained from the criterion measure, and that a higher sampling rate would be needed in order to better represent knowledge of the underlying population of words. We offer two main interpretations of these results, and discuss their implications for the construction and use of vocabulary size tests.},
  author       = {Gyllstad, Henrik and Vilkaite, Laura and Schmitt, Norbert},
  issn         = {0019-0829},
  keyword      = {sampling rate,testing,validation,vocabulary size,guessing,multiple-choice test,criterion-related validity,assessment},
  language     = {eng},
  number       = {2},
  pages        = {278--306},
  publisher    = {John Benjamins Publishing Company},
  series       = {ITL: Institut Voor Toegepaste Linguistik},
  title        = {Assessing vocabulary size through multiple-choice formats : Issues with guessing and sampling rates},
  url          = {http://dx.doi.org/10.1075/itl.166.2.04gyl},
  volume       = {166},
  year         = {2015},
}