How accurate are CGI characters rendered?

September 14, 2007 8:37 PM

Have there been any CGI films rendered with enough fidelity that a skilled lip-reader could follow the dialogue?
posted by sonofslim to Grab Bag (5 answers total)

Best answer: I'm a pretty good lip reader, and I'd have to guess no. (I can't say for sure, and it's not something I've studied.)

Even non-CGI animation - like the Simpsons - often makes a pretty decent effort to synchronize mouth movements, and often even mouth shapes, with the audio track, to avoid the McGurk Effect. This is very helpful if you use lip reading to supplement auditory input, like most lip readers. Still, it's crude and general, and it's not usable without sound - it's just an added set of clues.

As far as wether there a film whose dialog you could follow without sound, just by reading lips ... it's really not a case of fidelity, in most situations - lip reading in the wild is complicated by individual differences (both in terms of anatomy, and in terms of the way someone speaks - which could be a goal of CGI, but it'd have to be deliberate), lighting, the orientation of the speaker towards the watcher, and so on. So except for a few specific instances (a CGI'd newscast? A deliberate attempt at what you're asking about?), reading a character's lips would be fairly impractical anyway - just like it's fairly impractical to read lips while watching a live TV show or movie. But then, even in an ideal situation, lip reading is fairly imperfect. It's doable, especially if you have context to work with.

So, to answer a similar question: could we create, with current technology, a CGI character that could be lip read at a level similar to lip reading a live actor? I suspect it could be done by hand, although it would be a very slow process. I very much doubt we're anywhere near being able to do this with any sort of automation or speed.
posted by spaceman_spiff at 8:53 PM on September 14, 2007

Er, one more relevant note. In some circles, it's popular to call it "speech reading". (For the record, I think this is a bit silly - people know what you mean when you say lip reading, but not speech reading.) This is because there's far more than the lips involved - you're also watching the cheeks, the throat, the tongue, the jaw, breathing ... basically the whole face. And if you really want to communicate, you're also going to be paying attention to context, body language, eye contact/gaze - all the non-verbal elements of communication, except they've suddenly become much more important since 'speech reading' has a lot of ambiguities compared with auditory/phonetic English (insert your language of preference here).
posted by spaceman_spiff at 8:58 PM on September 14, 2007

I would check out Polar Express or other movies that have used facial motion-capture. Check out the trailer for 'Beowulf', should do the trick.
posted by milinar at 9:07 PM on September 14, 2007

It's interesting that you point out Beowulf and Polar Express, milinar, because I could not watch either of those two movies (well, the trailer for Beowulf) because of crappy-looking mouth movements. I'm going to side with spiff on this one ... there have been zero CGI movies that render a human's face with enough fidelity for speech reading.
posted by yellowbkpk at 9:36 PM on September 14, 2007

Forrest Gump?
posted by A189Nut at 2:13 AM on September 15, 2007