edit detection

October 19, 2009 1:41 AM

I'm interested in detecting edits in a given video stream. Is there any literature out there on edit detection?

The idea is to take in an arbitrary video stream as input, and detect 'edits', 'scenes' or 'cuts' in the stream. Further, to try and cluster similar kinds of edits/scenes/cuts.

How would you go about researching this?

Some examples:

http://cat.inist.fr/?aModele=afficheN&cpsidt=1579591

posted by yegga to computers & internet (3 comments total)

The term you're looking for is "Shot detection". There's quite a lot of computer vision work on this. Simple techniques just try to detect whether colours differ by a particular threshold; more sophisticated techniques can detect zoom / pan / tilt by analysis of optical flow patterns.

This review might be a good starting point for further research.
posted by handee at 4:20 AM on October 19, 2009

There are two ways to go about it. You can look, as handee suggests, at the video end result for "analog" indicators of tampering. Akin to seeing the clock on the wall in a shot change times incorrectly.

But you can also look at the more technical parts of the medium, or the actual transport. Does the data stream have a certain pattern of keyframes that changes at otherwise unpredictable points? Is there a timecode buried in the format? Does the audio show spots where the wave jumps phase or has some other kind of edit exposition?

The middle ground between the two might be looking for problems with the field interlacing, or if there is some kind of half of a frame long messup. (You used to be able to see this quite clearly on VHS edits, or even on a DVR where the station does a messy splice. Go frame by frame, and you'll see a normal frame, a frame with half of one picture and half of another, and then a frame with the other picture.) Or other analysis, like listening for background noise, or transport noise. Often, if you analyze the audio of a video, you can see weird spikes in the frequency distribution. Might be the 16khz noise, or a 60 cycle hum. Where there are hiccups in that, there may well be an edit. Sort of like if there is a clock in the room, and you hear "tick tock tick tock tick tock tock tick tock tick tock".

In other words, analyze the source for what it is supposed to be, and then find places where it isn't and eliminate reasons why it would legitimately happen.
posted by gjc at 7:08 AM on October 19, 2009

gjc and handee, mighty fine answers there.

gjc: how would you go about defining 'what the source should be'? i understand what you are hinting at, but how would you go about formalizing this? could you point me to some literature?
posted by yegga at 8:20 AM on October 19, 2009