An article in the most recent issue of TESOL Quarterly examines the efficacy of teaching Japanese university students the metaphors behind some phrasal verbs in comparison to simply explaining the verbs though Japanese. The treatment group (n=59), was given something like the following for each of the focussed prepositions (up, down, into, out, off):
In contrast, the control group was merely given translations with no grouping. For example:
- call off = 中止する
- calm down = 落ち着く
Fill in the blanks with the words given below so that the sentence will make sense.
- When she heard the news, she burst ( ) tears.
- No one can figure ( ) how the fire started.
Interestingly, the test included the 15 instructed words and 15 new words using the same metaphors. And here are the test results.
As you can see, the control group just beat out the experimental group in the studied words, but the experimental group did somewhat better in the new items. It turns out that this second difference is statistically significant. But what about the effect size? It's not published, but I calculate Cohen's d at .68, which is a moderate to large effect. If you add the two halves of the test together, though, you get a very modest 0.25 effect.
Here's how the author interprets the results:
"Thus the results of the experiment provide evidence for the claim that learning phrasal verbs can be greatly aided by increasing the awareness of orientational metaphors than (sic) by encouraging mere memorization." (emphasis added)What do you think? Is this spin or honest reporting?