Validation evidence for and against the Vocabulary Size Test – A closer look at the 3K, 6K and 9K bands

• author: Vilkaite, Laura ; Gyllstad, Henrik and Schmitt, Norbert
• organization: English Studies
• publishing date: 2012
• type: Contribution to conference
• publication status: unpublished
• subject: Languages and Literature
• conference name: Language Testing Forum (LTF)
• conference location: Bristol, United Kingdom
• conference dates: 2012-11-16 - 2012-11-18
• language: English
• LU publication?: yes
• additional info: Abstract The Vocabulary Size Test (VST) (Nation and Beglar, 2007) is increasingly being used for both research and pedagogy, despite the fact that there is still little evidence supporting its validity for these purposes. This study investigates a number of issues particular to the VST. L2 participants in Lithuania, Sweden, and the UK were given the original ten VST items from one of three one-thousand-word frequency levels (3,000, 6,000, or 9,000). They were also given another 20 newly-created VST items from the level. They were then interviewed one-on-one about their knowledge of these words, plus an additional 70 from the level (total 100). Thus, their results at various sampling rates (10/1,000 to 30/1,000) could be compared to the in-depth interview which sampled at a 100/1,000 rate. During the interview, the participants were also queried about their test-taking behaviour, guessing, and strategic approach to the VST. Sample Results: a) The 10-tem test has an insufficient sampling rate, correlating at only .80 with the interview measure. A 30-item test would be much more satisfactory, with a .93 correlation. b) Guessing is a serious threat to the validity of the VST. 57% of the participants were able to guess 2 or more of the original 10 items, and 30% were able to guess 3+. Given that each item represents 100 word families, this can lead to very substantial overestimation of vocabulary size. This guessing effect was particularly problematic with less-proficient learners, with overestimates of up to 50% for some participants. c) The guessing behavior was largely strategic, sometimes aided by weakly written items. Six major strategies were identified. d) Cognates were largely facilitative for participants who knew at least 1/3 of the words in a level, but learners with lower vocabulary sizes had difficulty utilizing cognates in the test.
• id: 78581243-5ff5-4440-ba00-51e8dc2f0a3e (old id 3218082)
• date added to LUP: 2016-04-04 14:22:40
• date last changed: 2018-11-21 21:19:59