Korean TOEFL shortage

Su-Hyun Lee, writing in the New York Times, has done a good job of reporting on the ongoing South Korean "TOEFL crisis".

"Forget the North Korean nuclear crisis. What has many South Koreans in an uproar these days is the “Toefl crisis.” The Educational Testing Service which administers the test, reduced the number of slots for test-takers. So with demand for the test far outstripping the available slots, and with scalpers demanding exorbitant prices, desperate South Koreans have been hunting for possible test sites from Japan to Southeast Asia, and even Australia.
Travel agencies have begun offering “Toefl tours” that include test preparation courses, a guaranteed test slot and sometimes even a bit of tourism on the side. One test preparation school estimates that about 500 Koreans a month all told travel to other countries to take the test."

While ETS surely deserves some blame for this, much should rest on the national testing fetish. The test is being (mis)used for everything from hiring, to high-school entrance. I'm sure that very few of those responsible for interpreting the tests have read the score user's manual.

I remember when I was teaching in Japan, a culture with a similar test preoccupation, the high school was using the paper-based TOEFL to track learning in their sr. high school students. On a test with a standard error of +/- 15, and a bottom score of 300, our students were averaging about 320. In other words, they were doing little better than guessing. It was a 6-month struggle to get the school to understand how useless and disheartening it was for students to have to take this test. There were also the inevitable test preparation classes which focussed on sample questions. These were so far above the students' level that it was a complete waste of time.

Elsewhere, Korea's Chosun newspaper bemoans the "fact" that Koreans are some of the worst English learners on the planet. How do they know? Just look at the rank of their average TOEFL score.

"According to a report released in 2003 by the Political and Economic Risk Consultancy in Hong Kong, which provides information on the management environment in East Asia, Korea ranked the lowest among 12 Asian countries when it comes to communication in English. From 2004 to 2005, the TOEFL scores of Korean applicants ranked 93rd among 147 nations. And last September, when a speaking section replaced the grammar component in the TOEFL exams, Korea’s rank dropped to 111th. In the speaking section, Korea ranked almost rock bottom, at 134th."

But if we put two and two together, its not hard to see that in a country where anyone from elementary school students to the local grocery uncle takes the test, you're going to have a lot of very low scores. Consider that South Korea, at about 50 million people [corrected, per comment], appears to have more TOEFL test takers than Japan, a country more than twice the size. They even have more test takers than China.

Clearly, then, the low average test score is not due to a national learning deficit, but to the fact that in more sane situations only people who intend to apply to a post-secondary institution in an English-speaking country and who feel they are likely to get a high mark apply. In other words, it is the result of a serious selection bias.