What does everyone know about XVID settings that I don't?

February 24, 2006 9:50 AM

double-pronged question about working with XVID

I download TV shows using BitTorrent on occasion, and I find the bulk of the videos I download are encoded similarly so half-hour programs and hour-long programs are almost always ~175 Mb and ~350 Mb in size, respectively. (with commercials removed, of course.)

I understand the basics of codecs and rendering, have several low-to-high-end apps that can covert video, and have played with the XVID configuration panel, but I just don't know what settings to apply to create video files like the ones I download.

Considering that there are hundreds of identically-encoded videos added to BitTorrent every week, I feel like there must be some standard format or setting being used, and I feel like an idiot for not being able to find anything using Google. Can anyone tell me what I should be doing?

And since we're on the topic, I'm also wondering if there is any means by which someone could erase a few frames from within a XVID-encoded video without having to re-encode the video to save the changes. For example, if I download a show that has 25 seconds of blackness where a commercial break should have been, can I trim out that blackness without having to re-encode?

posted by chudmonkey to computers & internet (17 comments total)

it is all very obscure. i'd start with the mencoder documentation. and i'd use lavc instead of XVID, imho its better.

there is software called "avidemux2" which is a gui-based .avi editor which will let you make arbitrary edits to an .avi. if you stay on keyframe boundaries, no reencoding is necessary. if not, it will "patch" the file for you and only reencode at the cutpoints.

it works on linux and macosx, but i dont know about windows.
posted by joeblough at 9:54 AM on February 24, 2006

A lot of time, the videos you download from bittorrent will be tagged with some sort of abreviation signifying the release group responsible for encoding and releasing the video. My understanding is that these groups tend to have proscrbed tools, versions and settings for doing encodings. That said, I don't know where you'd look to find those settings. Probably an fserv on an IRC channel somewhere.

VideoHelp.com. Has a number of guides for converting videos, an overview of tools, including AVI editors, and active and sometimes helpful forums.
posted by Good Brain at 10:16 AM on February 24, 2006

I've been able to get pretty decent XVID compression results with Auto Gordian Knot. It basically automates the various steps involved using several different pieces of software.
posted by nixxon at 10:26 AM on February 24, 2006

Based on observation, I've noticed the following:

Encoding is done at a variable rate to fit a full 2 hour movie onto a single CD (~700 megs) cause it was/is cheap, convienent, and capable for commerce.

Now, most of these people then use those settings...for a one hour show (~350) and half hour (~175).
posted by filmgeek at 10:28 AM on February 24, 2006

Filmgeek makes a good point. It's also interesting to consider that people have been tuning settings and probably even the code, to get the most pleasing picture on the widest range of content at that the 350MB/hr bitrate, so the incentive to break ranks and go with, say, a 50% higher bitrate for more quality is offset by the work it would take to get things dialed in again.
posted by Good Brain at 10:39 AM on February 24, 2006

If you're not doing 2-pass VBR that is probably the reason. This allows the encoder to vary the bitrate as the scenes dictate, so that scenes with a lot of motion can use a higher bitrate, and scenes with relatively little motion can use a lower bitrate. This requires two passes, one to analyze the entire clip and distribute the "bitrate resevoir" and another to do the actual encoding. You only supply to the codec a single overall average effective bitrate, calculated based on the length to give the desired file size. There are many calculator applets out there to do this.

Also you didn't mention what software you are using but if you're not using virtualdub (or one of its forks like virtualdubmod which handles VBR audio much better) then you're probably doing something wrong. Vdub is free/open source and the standard for xvid encoding. It can do everything you need to do. There are many tools that provide a friendlier front end (such as Auto Gordian Knot) but they all use vdub to do the actual work.

There are countless tutorials on doing 2 pass Xvid encoding with virtualdub. Check sites like doom9 or videohelp.com.
posted by Rhomboid at 10:41 AM on February 24, 2006

e.g. 2-pass Xvid encoding with Virutaldub guide.
posted by Rhomboid at 10:45 AM on February 24, 2006

Oh and if you want your Xvid to be playable on standalone (i.e. set-top) DVD players you should not use qpel (quarter pixel motion estimation) or GMC (global motion compensation) as these are advanced features that many hardware decoders cannot handle.
posted by Rhomboid at 10:48 AM on February 24, 2006

You must have top quality source material! Noise and other artifacts don't compress well.
posted by Chuckles at 11:39 AM on February 24, 2006

Rhomboid: Thanks for the info about qpel and GMC. And thanks to you and Good Brain for the videohelp.com link.

Since no one so far seems to know what specific quality/encoding settings I'm looking for, perhaps someone knows how I could pull this information out of one of the files I already have?
posted by chudmonkey at 1:59 PM on February 24, 2006

Popping open about half a dozen Divx files (I used quicktime player as it gave more info than VLC player did.)

Most of them are around 900-1100kb/s.

This is no great suprise. VCDs were encoded at this level, splitting at the hour, perfect for movies - 2 VCDs covers the lenght of most films.

All sorts of res (640 x 352, 576x320, etc) ; find a movie you like and mirror those settings. Make sure you use 2 pass.
posted by filmgeek at 2:26 PM on February 24, 2006

i've got what i think is a reasonable set of args for mencoder, but it sounds like you want to use virtualdub...

it would be nice if the encoding tools could stash what options were used somewhere in the resultant file, but i dont think there is any such metadata. AFAIK there is just the framerate, bitrate, codec name, WxH size, aspect ratio (in ODML .avi anyway) and color depth in the AVI header.
posted by joeblough at 2:30 PM on February 24, 2006

joeblough: I have no restrictions on software at all! Tell me more about your mencoder solution?
posted by chudmonkey at 3:25 PM on February 24, 2006

As I said you calculate the bitrate based on the length. Say you have a clip that's 23:30, and you want the file to be 175MB. You plug those two numbers (and the type of audio you're going to use, typically 128 kbit VBR mp3) and it will give you a video bitrate which is simply (total bits)/(total seconds). You enter that in the xvid settings panel, set up the two passes in virtual dub, let it run for a few hours, and out pops a nice 175mb AVI file.
posted by Rhomboid at 4:11 PM on February 24, 2006

The sizes are based off CDs. 350mb = 1/2 an 80 min CDR.
posted by tiamat at 5:02 PM on February 24, 2006

i always do 2-pass encoding, using lavc mpeg4. IMHO this gave me better results than xvid (using the package called "transcode") but i wasnt very experienced with video coding at the time, and transcode was also kind of rough then. (~2yrs ago). perhaps xvid is better now, or i just didnt know how to use it. at the time transcode had problems keeping the video and audio in sync so i dropped it like a hot rock.

i suspect that the windows-based stuff (like virtual dub, gordian knot, etc.) probably analyze the input video format and handle a lot of the corner cases automatically for you, but with mencoder you have to plan it all out ahead.

the 2 big nasty things are framerate conversion and conversion from interlaced to progressive video.

movie DVDs, though they are stored as interlaced video, are actually progressive @ 23.976 frames/sec (each frame is stored as two interlaced fields, but they belong to the same moment in time.) for display on an NTSC TV, the dvd player has to convert this to 59.94 fields per second, interlaced, using a process called 3:2 pulldown. but for your computer (since your display is progressive), you really want progressive video @ 23.976hz. fortunately, the mpeg demuxer in mencoder understands that the DVD is actually 23.976 progressive, and if you tell it the output frame rate is 23.976, it just does the right thing.

the TV shows you're seeing around the web probably started life as ATSC tv broadcasts. although the ATSC spec includes all sorts of framerates (23.976, 24, 29.97, 59.94) at a variety of resolutions (480interlaced, 480progressive, 720p, 1080i, 1080p), in practice only three are ever used: 480i@59.94 fields per second, 720p@59.94 frames per second and 1080i@59.94 fields per second.

for all of these formats, if the material was shot on video, it was probably shot at 59.94 fields or frames/sec. in the case of interlaced video then, you just need to apply some kind of deinterlacing filter to get progressive output. i usually use "-vf lavcdeint" in mencoder, and i dont specify an input or output framerate, as they should be the same.

if the material was shot on film @ 24fps (most of the high-production shows are these days), then things get kind of messy.

- if the station is broadcasting 720p@59.94, then they have repeated the original film frames in the following pattern: F1 F1 F2 F2 F2 F3 F3 F4 F4 F4 ... to convert 23.976 to 59.94. to return this to 23.976, you need to delete the repeated frames. the challenge arises because, due to a variety of reasons, the first F1 and the 2nd F1 are not exact digital copies of one another. i have been using the -vf decimate filter in mencoder to do this, but its not foolproof. some repeat frames sneak through, and some non-repeats are turfed, but its usually nearly correct. of course in this case you have to give -ofps 24000/1001.

- if the station is broadcasting 1080i@59.94, then they have done the 3:2 pulldown conversion at the station to convert framerates. in this case, there are several mencoder filters that can be used, but i've had good luck with -vf pullup,softskip (which is the newest so called "inverse telecine" filter. there's also filmdint and ivtc. also, again you have to specify -ofps 24000/1001

of course, in either of these situations, you could just transcode to the original framerate. for the 720p material it looks okay (though to my eye, the juddering of the 3/2 pattern is very apparent). for the 1080i material, you'll end up with F1 F2 F3 F4 F4 F5 F6 F7 F8 F8 ... in the output, which is also pretty noticible at least to me. there is an mencoder filter designed to undo this bad deinterlacing (-vf divtc) in case you have messed up and improperly deinterlaced telecined content.

this is a pretty good guide to using lavc + mencoder.

generally my 1st pass has the following args:

-lavcopts vcodec=mpeg4:v4mv:vqmin=2:vqscale=2:vpass=1:trell:mbd=2:autoaspect:keyint=50:turbo

my 2nd pass has the following args:

-lavcopts vcodec=mpeg4:v4mv:vqmin=2:vbitrate=5500:vpass=2:trell:mbd=2:autoaspect:keyint=50

i use keyint=50 so i can edit out stuff; if there's enough keyframes around you won't have to re-encode across the edits (since you can find keyframes near enough the places you want to make cuts)

vbitrate=XXXX is the knob that's going to give you the filesize. its specified in kbits/sec. generally speaking i use a bit less than 1/2 the original bitrate of the ATSC broadcast, unless i have scaled the video down from 1080i -> 720p, in which case i use about 1/3 the bitrate. that's where the 5500 in my example comes from. the stuff you're finding on the net is usually nowhere near HD resolution, so the bitrate is much less, probably in the neighborhood of 1000-1200 kbit/sec.

in the 1st pass, vqscale=2 with no bitrate basically says don't really try hard on the quantization. the 1st pass is just to write the logfile which says which frames need more/less bits, so the 2nd pass can allocate the available filesize intelligently. similarly, :turbo to speed up the 1st pass.

also, note that you should use 30000/1001 when you mean 29.97 and 24000/1001 when you mean 23.976.

phew.
posted by joeblough at 5:20 PM on February 24, 2006

Thanks to Rhomboid for that brief and clarifying answer, and to you andjoeblough for the technical directions as well!
posted by chudmonkey at 6:52 PM on February 24, 2006