Can I Convert Video Footage to Animation?

August 30, 2012 7:23 PM

Can I Convert Video Footage to Animation? We are making a video documentary about college students and their attitudes towards dating and sex. Much of the documentary involves one interviewer interviewing different people on the same set. As you might guess, many of the potential subjects are reluctant to speak candidly onscreen, BUT, most of them will consent to their voices being used…I am wondering if there is some not-crazily-expensive way to take a video interview and, using some kind of software, animate it, turning the interviewer and subject into animated avatars. The animation does not have to be super super high quality—a fairly simple avatar for each of the people onscreen will do…We have a lot of technical and post-production people on the crew, so someone involved probably has access to almost any software program available…Any help—especially specific programs or plug-ins—would be greatly appreciated. Thanks!
posted by laswingkid to Computers & Internet (13 answers total)

Erm - is there a reason you wouldn't just blur out/mosaic the interviewees?
posted by kavasa at 7:36 PM on August 30, 2012

Look into rotoscoping filters
posted by sanka at 7:37 PM on August 30, 2012 [2 favorites]

Yep, rotoscoping is what you're after. There's tutorials to do it on After Effects. Just keep in mind that Rotoshop (used for A Scanner Darkly and, I think, Waking Life)is not available to the publix.
posted by griphus at 8:00 PM on August 30, 2012

Response by poster: I should make it clear that our ideal would be to replace both the interviewer and interviewee with some kind of stylized avatar. Most important to us is that the interviewee not be recognizable (visually) by his/her friends.

To those that have recommended rotoscoping filters, can you suggest any specific filters or programs, or can you point me to some YouTube videos demonstrating these particular filters?
posted by laswingkid at 8:18 PM on August 30, 2012

It sounds like you're describing facial motion capture. These guys make a product that does it automagically with an XBox Kinect and they're giving out free 30-day trials of their beta right now. I found that by Googling for "kinect facial motion capture" and there seem to be lots of people, from hobbyists to professionals, working on this kind of thing.
posted by XMLicious at 8:43 PM on August 30, 2012

This is a stretch, perhaps, but what about Source Filmmaker? It comes with a bunch of very cartoony avatars, and the really neat bit: it will do lip syncing, so it'll actually match your recordings.

(You might perhaps not want TF2 and L4D2 avatars. You can import other characters, though.)
posted by zompist at 8:49 PM on August 30, 2012

You could use Xtra Normal's text-to-3D Animation and then replace the audio track with the audio track from the interview.
posted by Mad_Carew at 10:36 PM on August 30, 2012 [1 favorite]

Do you have any kind of a budget for this film? Is there an animation department at any of the schools in your area?

Any automated program is not going to turn out animation of usable quality. Although the process of modern computer animation has been streamlined and is less time consuming than it once was, it still requires a human artist who knows what they're doing to guide the animation process, often frame by frame. There's a joke in the animation business about how people often think you set up some basic parameters and then press an "animate" button on your computer -- that's really not how it works at all, even now, when we have much better tools than we used to.

You could probably find a student who will do this for you for very little money, although you run the risk of getting exactly what you paid for. If it's at all possible, setting aside the cash to hire a professional freelance animator will make your life MUCH easier in the long run. I'm sure there are plenty of people here on Metafilter who do that kind of work, so it might be worth your while to put a listing in Jobs.
posted by Narrative Priorities at 7:23 AM on August 31, 2012

Response by poster: I've checked out some of the rotoscoping programs suggested and those won't do--you can still discern who the original subject is, even through the rotoscoping.

I believe what I'm looking for is some program that replaces human figures with cartoon avatars. So, for instance, if I had a video interview between Charlie Rose and Warren Buffett, I could "convert" it to an interview between (seemingly) Fred Flintstone and Barney Rubble (although preserving the originals subjects' voices and gestures).

Is there such a thing? Or is this still in the realm of science fiction?
posted by laswingkid at 3:35 PM on August 31, 2012

Did you look at the "Face Shift" demo video on their home page I linked to above? You can't just convert existing video, you have to have something like the Kinect sensor present during the original interview to do the motion capture, but it appears to be doing what you're describing.
posted by XMLicious at 5:15 PM on August 31, 2012

I could "convert" it to an interview between (seemingly) Fred Flintstone and Barney Rubble (although preserving the originals subjects' voices and gestures).

Is there such a thing? Or is this still in the realm of science fiction?


Unless you had the subjects rigged up for motion capture when you did the initial interview, there's no way to do this automatically.

I know this isn't the answer you're hoping for, but you're probably going to need a human animator to get the results you describe, even if the animation is extremely simple. Hiring an animator seems like it would be less hassle than re-filming all of your interviews with a kinect setup, but obviously that's up to you.
posted by Narrative Priorities at 7:41 AM on September 1, 2012

Bit late but would something like this help to obscure the faces etc?

play videos in ASCII mode
posted by quaisi at 2:48 AM on September 14, 2012

Just caught a television news clip about a company making tools for this sort of animation:

http://www.reallusion.com/

Their product "CrazyTalk" does lip-synced animation just based on the audio, so it would work without any motion capture data. (Though presumably it's just guessing at the facial expressions.)
posted by XMLicious at 12:12 AM on August 26, 2013