Exercise type and L2 vocabulary retention

Keith Folse has published a tidy study in the most recent TESOL Quarterly. Here's the abstract:

The present study used a within-subjects design to examine the effect of the type of written exercise on L2 vocabulary retention. Using input for the meaning and usage of the new words from a specially prepared minidictionary, university intensive English program students (n = 154) practiced target vocabulary in three types of written exercises conditions: one fill-in-the-blank exercise, three fill-in-the-blank exercises, and one original-sentence-writing exercise. An unannounced posttest using a modified version of the vocabulary knowledge scale tested the meaning of the word (L1 translation or L2 synonym) and usage of the word in a student-written sentence. A repeated measures ANOVA revealed that mean scores for the three exercise types were significantly different from each other, with words practiced under the three fill-in-the-blank exercises condition retained much better than those practiced under either of the other two exercise conditions. The findings suggest the important feature of a given L2 vocabulary exercise is not depth of word processing but number of word retrievals required. This result has implications for language teachers, curriculum designers, and, in particular, materials writers of traditional workbooks and CALL materials.

He's done a few things particularly nicely.

1. He was very rigorous in the selection of target words. All were verbs; initial letters and syllable count were held constant across groups; cognates and known words were eliminated through pretesting; frequency was considered; and morphological affixes that might hint at meaning were avoided.
2. He's been very clear in his procedure such that replication should be quite easy.
3. He's controlled for time on task.
4. He's gathered data on any out-of-class learning that might have happened beween the pre- and post-tests.

One minor quibble is the use of ANOVA with a non-linear scoring system. He used a system where students got 1 point for an accurate synonym, definition, or L1 translation, and two points if they could also use it correctly in a meaningful sentence. Under such a system, a student who could do the first part for two words is deemed to be equivalent to a student who can do the second part, but only for one word. However, the results are almost the same if you use a 0 or 1 scoring system.

More target words would have been nice; only 15 were used.

The major problem I see with the study is the lack of a delayed post-test. I would be surprised to see the effectiveness of sentence writing and blank filling flip flop a month down the road, but the differences seem likely to be smaller.

Despite that, given the relative ease of creating and checking blank-fill activities, this is a study I'll keep in mind, especially if I get to work on any computer-based vocabulary study software.