Sunday, November 13, 2011

Feedback on English Accent Coach

A few days ago, I posted about a site called English Accent Coach. I've had a bit of feedback, which I passed on to Ron Thomson. He's given me permission to post it here.

This is very useful info for me. Thanks so much. It will help me build an FAQ page for the site. It also confirms that the best way to make this effective is to really train teachers or better help them to understand the science behind the site as some of these comments reflect common misconceptions about how pronunciation develops – and I wouldn’t expect any teacher to know this. I’ll try answering some of these in turn if it might be of help to teachers/learners.

Paul: "I failed quite a few even knowing the symbols. I think it is partly a Brit/Am thing. I only tried it briefly, but some words were incomplete so I was expecting to hear a consonant to make a complete word, as in 'hit', not 'hi....' Strangely, that threw me, too." 
Yes, this is not unexpected. Teachers could try a higher level (e.g., Level 4) and see that they aren’t that bad if they’re familiar with the symbols. It IS tricky for native speakers to hear sounds in isolation. In fact, normal speakers use top down processing to disambiguate sounds. For learners, not having words is essential because, otherwise, they too will use top-down, only their top-down processing is based on faulty representations of the words. The approach in the site prevents that.

Interestingly, this conflict between top-down processing and phonetic processing causes some learners in my research to do more poorly on words than non-words. My suspicion is that this is because there are too many conflicting phonetic cues that don’t match their language. So for example, they might have a sound similar to the English word be, but not hear the vowel properly in a word like beat because they don’t have words that end in /t/. That t throws them off in the same way not having it might throw Paul off (as above).
Paul: "Just heard ha, which after 3 wrong goes matched head."
If Paul is Brit, then he’ll definitely not match the symbols easily. It’s a fallacy that there’s such a thing as a stable, IPA/Cardinal vowel system, as much as phoneticians say there is. When I listen to IPA produced by British vs. American experts, I hear very different categories. However, a British teacher, already having most of the American system won’t take long to learn the categories in the non-words.

If Paul is American, or if an American gets these wrong, then although everyone has slightly different category boundaries, consistent errors suggests the user isn’t very good at phonetic level processing – which tends to weaken at about 1 years of age – however this training will bring it back in a hurry, it just hasn’t been needed in natural communication. I can’t get into the detail of it here, but word level processing is not the same thing as phonetic level processing.

Native speakers aren’t aware of just how inaccurately we produce vowels in some words (usually related to frequency). For example, one speaker I recorded imitated my have prompt as /hɛv/ rather than /hæv/. He doesn’t sound like he has an foreign accent, and most of his words have what we’d expect. Nobody would pick up on his sloppy pronunciation of have because it’s not something totally off, like hove, and it’s a high frequency word, which listeners tend to process only superficially.

I’ve got some short, but rather technical research I could send to anyone who is interested that shows this effect. The key point is that categories aren’t particularly stable things, and so English Accent Coach is artificial in the sense that I’m actually trying to exclude obviously ambiguous or “wrong” items so that it really does allow for phonetic learning.
Paul: "I've just tried half of the vowels section again and did much better, though, if I'm honest I realise that sometimes I couldn't identify sounds automatically from the symbols."
Yes, not surprising, and I’d be surprised if Paul doesn’t quickly start reaching nearly perfect. Native speakers get there pretty quickly.
Paul: "I wonder why symbols are shown and not, say, short but complete words to which the vowel sound could be matched (e.g. for an 'oh' sound, click on 'hot'). Perhaps that would be a more effective way."
Words adversely affect/distort phonetic processing, as per above. Even symbols do since they may be previously associated with faulty/foreign phonological representations. In my previous work, I didn’t even used symbols, but nautical flags instead. The reason I use symbols now is because learners are less likely to want to learn something that seems totally bizarre (like nautical flags), unless a teacher is there to guide them. I’m hoping to create a switch eventually that will allow the use of other imagery, but that’s some time off.

Another issue is that English spelling is unpredictable, so not useful as a guide. That makes it difficult for non-native speakers to rely on since they don’t know the sound. Finally, these symbols are widely used in many dictionaries, so learners will benefit from knowing them. It’s obvious when learners have got the symbols as they quickly start getting 100% on vowels that are close to vowels in their language. When I trained Chinese speakers, after 2 or 3 sessions of 200 items, they’d get 100 on /o/, /e/, and /i/ all the time, meaning errors with the other vowels were the result of perceptual difficulties. Those three vowels that they got correct after a few trials are very similar to Mandarin counterparts.
Blair: "I've tried the site as well and I agree there are a few problems. It's not that there are no words, I think the site is clear that it's focusing on sounds only, but some of the sounds are not on the chart, to my ears. I'm native Canadian from Toronto so I don't think my accent would be perceptibly different from the people at Brock University, so I'll discount that. 
There is at least one diphthong in the sounds /ei/ (play/say/May) but it is not on the chart."
In fact almost all Canadian/American vowels are diphthongs. Books just don’t say that. For example /ɪ/, which is always called a monophthong in teacher texts is more accurately described as /ɪə/. What’s important to know is that I don’t use any diphthong symbols at all for the vowels currently taught on the site. So instead of /ei/ or /ej/ I just use /e/. This is to keep it simple for learners. Too many symbols just confuses them (unless they’ve already learned them), and really, it’s not primarily about learning a symbol, it’s about categorizing all instances of a given category the same way.

Regarding the English diphthongs /aj/, /oj/ and /aw/, I decided to exclude them for now because testing showed they are too easy because they are really extreme diphthongs that move across the entire vowel space. There may be some learners who confuse them, but I’m skeptical they will for very long. At the same time, if I get feedback that these are real problems, and know what the nature of those confusions look like, then I could create a special diphthong game. Incidentally, an iPhone app version will allow users to turn on those diphthongs and should be available soon.
Blair: "The /ae/ (mat/cat/flat) has in my opinion some variance, at times I could only hear it as the short "u" (hut/mut/shut)... The stressed "i" (she/he/me) is sometimes indistinguishable from the lax /e/ (met/set/fret). Those are the ones that I remember."
Again, if it’s isolated sounds we’re talking about, then it’s not surprising. Native speakers can’t do this automatically; it requires training. In experiments, native speakers can’t get 100% anyways. My rule of thumb is that for some vowel categories, that are inherently more ambiguous (e.g., many of the lax vowels), given my relatively unambiguous training set, native speakers should get 90-95% on those vowels, while some may get 100%. Many of my phonetics students and other teachers get 100%, but only after playing through 100 about 5 times. I note that this user says he only listened to 40 sounds. I’ve never heard of someone without phonetic training doing well. However, phoneticians, including those from the United States get 100% or close to that right off the bat. In other words, teachers need to use it a few times before believing that because they don’t get 100%, it’s not good. In the linguistics world, this methodology, as built into the site is very well-attested in many research labs. There is no doubt it is effective, learners (and teachers) just need to give it a chance.

Thanks to Blair and Paul for taking the time to send feedback, and to Brett for forwarding it. It’s very helpful to me, but in terms of design and in terms of understanding how teachers will react. It really will take teachers believing in it, giving it a chance, and seeing the results for learners to benefit as the approach may not seem obviously beneficial ... if you get learners to use it for 5 to 6 trials of 200, you will be very impressed, as will they. This is much more extensive than lab-demonstrated approaches however, so those first 5-6 trials will only be the beginning. However, that’s all it takes for learners to start seeing major dividends.


Ran said...

Is there some accent difference that would explain why I (an Ohioan) had difficulty with the /ʌ/'s in level 4? To me "tough" sounded like "toff" (I chose /ɑ/, then guessed /ʌ/ as a fallback) and "lush" didn't sound like any English vowel (I tried /ɑ/, /æ/, and /ɛ/ before it gave up on me). I had no difficulty with any other vowel in that level.

Brett said...

I'm afraid I don't have any answer for you. I'll pass this along to Ron though and see what he says.

GnomeTheChumpsky said...

Hi Ran,
Good questions. There are three possibilities I can think of:
1) There is an accent difference (quite likely for the /a/ - /ʌ/ contrast, which even studies of American vowel perception tend to result in accuracy scores by midwestern Americans (what you should be close to) of as low as 80%. So /ʌ/ is not really as unambiguous as phonologists suggest it is. However, although there may be an accent difference, if that's what's causing it, it wouldn't take a native speaker long to adjust. There is clearly a contrast there for the speakers in the database, you just might not be able to hear it in every case because the boundary isn't where yours is.
2) The items you got wrong could actually be bad items. This is one of the main reasons its in beta. I need to see what items people most often get wrong and then I'll be able to discard them from the database as being ambiguous. Since there are ultimately 100,000 recordings, it is impossible for me to screen them all independently. Even if I did, what's unambiguous for me might not be so for others.
3) I don't know you, but if you don't have a lot of phonetic experience, then perceiving the /ʌ/ may not be that straightforward. It varies by contrast, by speaker, and by listener, even within a dialect. Experiments I've conducted recently with vowel id rates by phonetics students in my university show that they only agree on /ʌ/ 80% of the time. Also, if "tough" sounded like "toff", then how did you know it was "tough"?

GnomeTheChumpsky said...

Solved, I think ... I just went and listened to all recordings of "tough" and "lush" that are in the system and there were some ambiguous/bad ones that have a vowel closer to /a/, even for me. Interestingly, it proves just how much unnoticeable variation there is in normal speech. I'm assuming Ran also got a bunch of the /^/s right, meaning the issue was really just with some isolated items, because of how a particular speaker or speakers produce them. Again, this will get better and better as these things are screened out.

Ran said...

Re: "Also, if 'tough' sounded like 'toff', then how did you know it was 'tough'?": I worked backwards. After /ɑ/ was rejected, I guessed /ʌ/ as a fallback, and that guess was confirmed. The /t/ and /f/ were a given, and obviously I know that /tʌf/ == "tough". :-)

GnomeTheChumpsky said...

So the moral is that "tough" is tough when it sounds like /tɑf/. Sorry about that. I'll remove those items post haste. Thanks for the feedback, Ran in Ohio. I think most of level 1, 2 and 3 are better screened and you'll run into less of that. Real words will take some time as there are so many of them. Incidentally, I played through a Level 4 round of 100 after your remark and never did hear bad items, so they are still relatively few and far between.