Motion tracking video, data analysis edition
April 8, 2013 8:13 AM   Subscribe

Is there a program that will do what I'm imagining? Import a video. Automatically pick tracking points in the video and track their 2D movement. Output tracking data in a text format that I'll be able to import into other programs.

I've been playing with quantitative analysis of large quantities of video. I'd like to be able to analyze the motion within the video.

I was looking a bit at trackers in After Effects, but they either require more manual input than is practical for doing the quantities I'm talking about, or they won't export what I want. The 3D Camera tracker in AE has a stage where it will display dozens of tracking points, which would be perfect for what I want, and it proves that it's possible, but I can't get it to export those. It only exports the "solved" 3D camera movement, and if it fails to solve it I get nothing.

I guess overall, it's that a lot of motion tracking software is targeted at creating visual effects, not analysis.

I've been doing the other processing in R. Searching for motion tracking in R has so far resulted in only blob-detection algorithms that presume you have a still camera and a moving subject.

Any other solutions?
posted by RobotHero to Computers & Internet (9 answers total) 4 users marked this as a favorite
 
I don't know about the export formats, but Kinovea is a useful application for analyzing motion in video.
posted by dgran at 8:28 AM on April 8, 2013


How are you with C/C++/Python? OpenCV is the tool for these kinds of analysis problems, but you'll need to write some code to get it doing what you want. You might look at the docs for goodFeaturesToTrack, which will find contrasty points (part one of your problem), and then calcOpticalFlowPyrLK to track those points across frames. You'll have to roll your own exporter code, and deal with deciding things like "what happens when I lose a point partway through a shot" and the like, but that's the low-level way to build it up.
posted by Alterscape at 9:15 AM on April 8, 2013


Response by poster: The export formats for Kinovea look fine, but the kind of motion tracking might not be what I'm looking for.

All the examples I saw on the blog were tracking a small number of things very precisely. Whereas I'd be fine with not always being precise but I want it to track automatically many moving points.

My idea is kind of a film analysis, where I can distinguish still camera from hand-held, if the camera moves, which direction, if there is something moving in frame, how much of the frame does it occupy, is the movement smooth or variable, etc. If I can get a large number of automatically tracked points, I can search for patterns that would allow me to automatically classify visual info for each shot.
posted by RobotHero at 9:16 AM on April 8, 2013


Response by poster: The OpenCV might be a good plan, it would certainly let me customize it to my own needs.

Because of Bicklenack, I was intrigued by the Echonest remix which would mean learning some Python anyway.
posted by RobotHero at 9:29 AM on April 8, 2013


What you want is usually called "Structure From Motion", and in your case you don't care about the structure side of things and would only look at the computed camera positions. A popular platform for this would be Bundler, which outputs camera translation and rotation, allowing you to figure out if the camera is being panned or on a dolly.
posted by drwelby at 10:09 AM on April 8, 2013


We do exactly this at work for high-speed video analysis using a tool called Tracker (if I'm understanding what you're looking for—you want displacement data, right?). The site says "Export formatted or raw data to a delimited text file or the clipboard." It's written in Java and has a pretty straightforward interface and shallow learning curve. Hope this helps.
posted by jroybal at 4:57 PM on April 8, 2013 [1 favorite]


Response by poster: Tracker and Kinovea I've been able to install and use. They are not quite what I was looking for, but I'm going to keep them in mind as an expanded repertoire now that I know what they do.

It looks like OpenCV using Python has the power I want, but I'm having trouble working through the installation in Windows 7 and/or library dependencies. That might be next week's question.
posted by RobotHero at 1:20 PM on April 14, 2013


Response by poster: Okay, I've got my Python / OpenCV / NumPy / SciPy issues worked out, and can run the OpenCV samples.

Now I'll take a look at goodFeaturesToTrack and calcOpticalFlowPyrLK
posted by RobotHero at 2:47 PM on April 14, 2013


I'm currently researching ways to track moving objects in videos and here are the options I've found:
1. Motion 5 apple, tracks very well, easy to use, but can't for the life of me get the XY coordinates out. It appears the coordinates are stored in an XML file but I can't find an easy way to use it.
2. Image J (or FIJI) has many motion tracking plugins such as multitrack, mtrackj, trackmate etc
3. BioTrack Pack
4. Syntheyes
posted by dhruva at 8:21 AM on April 29, 2013


« Older So why are you studying Arabic?   |   Pop song about a girl who apologizes to nerdy high... Newer »
This thread is closed to new comments.