How well does Rosetta Stone's speech recognition instant feedback work?

January 12, 2012 8:36 AM

People who have used Rosetta Stone: how well does the software's speech recognition instant feedback work?

I learned that Rosetta Stone actually has speech recognition software that will listen to your pronunciation of a word and offer instant feedback. Based on what I know about speech recognition, this is actually a very difficult task, and I am amazed that it works well enough to be included in a major consumer product.

I have been able to find little information on how this actually works, i.e. what the software specifically can do, what types of feedback it offers, how accurate it is. The Rosetta Stone website does not make it easy to find this information. I did find a video on their website that gives the 30,000 ft. explanation and says it is proprietary technology, but doesn't go into detail about what it does and does not do.

I have found an image of a feedback session here.

From the things I have been able to read, it looks like the user can view a waveform of the correct pronunciation and compare their pronunciation with it. But how does this work exactly? It's not trivial for someone to just interpret a waveform. Does the software offer suggestions so that the user knows how to make their waveform be more like the correct one? Does the software point out what the user is doing wrong/right (e.g. "you are pronouncing 't' wrong", or whatever)?

Also, I am certain the software makes at least occasional mistakes, but I wonder just how often it makes mistakes, and how badly?

If anyone has descriptions from their own use of this technology, I would love to hear it (and if anyone actually has an in-depth article or paper on the topic, that would be amazing)!

posted by kosmonaut to Writing & Language (8 answers total)

I have the Rosetta Stone software for learning Portuguese, and the voice-feedback feature was gimmicky and terrible. My Brazilian wife would tell me my pronunciation of a word was perfect, and the software would tell me I was way off.

All it did was make you match the waveform. You could make unrelated sounds and get a perfect score. Maybe the same level as Rock Band in singing mode.

That being said, I know there's a newer version of the software out now, so maybe they've improved it.
posted by jozxyqk at 9:07 AM on January 12, 2012

I used Rosetta Stone briefly a few years ago (so, a different version of the software) but as I remember it a lot of the feedback was on specific pairs of sounds more than actual words - for English you might practice the difference between the "th" sound at the beginning of "this" and the one at the beginning of "thin," or the difference between "L" and "R". There is very little explanation in the software of *anything* - it's supposed to be an immersion kind of thing.

So you just keep trying it until it's right. I'm sure it does make mistakes, but as a language learner it's difficult to notice when it's making a mistake, since you are also making lots of mistakes. For the sounds I was already good at (I used it for German) it seemed to mostly work for me. I didn't try to "fool" it though. And I would guess that it would work better for some voices than others.

I don't think you're supposed to look at the waveform and say, "ah, yes, this means that I'm using a velar blablabla where I should be using a whatchacallit fricative," it's just supposed to give you a little more data about whether you're close or not.
posted by mskyle at 9:11 AM on January 12, 2012

when I tried it i didn't work well enough, and actually hindered the experience ... I turned it off.
posted by jannw at 9:12 AM on January 12, 2012

I think it must depend somewhat on accents, too. In jozxyqk's, Brazil has a dozen different accents, some of them pretty distinct. If the program is geared toward's one it may register major sound differences that are due to accent as mistakes.
posted by TheTingTangTong at 9:31 AM on January 12, 2012

My fiancee did the Dutch language course a couple of years ago. I'm a native Dutch speaker myself, and it seemed quite accurate to me. Just bear in mind that it doesn't deal with sentences, but mostly single words. There were a couple of instances where the software refused to recognize what to my ears was a perfectly fine pronunciation, but for the most part I thought it worked quite well.
posted by monospace at 9:53 AM on January 12, 2012

There is very little explanation in the software of *anything*

This is why I say Rosetta Stone is a good tool, one you might use as part of your language learning, but it isn't enough by itself. (Don't believe their advertising!) I teach ESL and I am a language learner. I have used Rosetta Stone as part of my Japanese learning, and we have used Rosetta Stone at my school in my English classes. For reasons unknown to me, we never used option E, the Speech Recognition feature, at school; but I've played around with it and I can see its utility, but as others have noted it's not Rosetta Stone's strong point.
posted by Rash at 11:55 AM on January 12, 2012

If you do get it, make sure you get a good microphone, and that the software is recognizing/using it. When I use it on my laptop, sometimes the program will default to the built-in microphone, which does NOT pick up what you are saying correctly. Using it with a USB mic works a lot better. You can check the waveform and see if it's being picked up correctly by playing back what it recorded you saying.
posted by theRussian at 3:16 PM on January 12, 2012

Kosmonaut asked about Rosetta Stone, and implied voice recognition was rare in language programs. Several other programs have it and seem to do it better: Tell Me More, Pronunciator, Babbel, eBerlitz, English Central and GoEnglish. All have reviews and links on my site TalkLikeANative.org , where you can try out the voice recognition. The coolest is Tell Me More, which shows the overall trend of pitch/intonation from the beginning to end of each word.
posted by riverbook at 8:14 PM on April 2, 2012