Comments on: decoding 10bit video

Question: decoding 10bit video

shino-boy — Mon, 30 Jun 2008 11:14:58 -0800

Decoding a 10bit video file ain't easy...

I own an HD camera mod that records uncompressed 8bit and 10bit video files directly to a Mac's HDD. The camera company provided software to capture and render the video files. The rendering software never worked correctly with the 10bit files and we were waiting for an updated release. The company was sold and the rendering software was *not* updated and is now no longer supported.

:(

I wondered how hard it would be to write a Quicktime component under OS X 10.4 to directly read both the 8bit and 10bit files.

I bought some red, green and blue felt from the art store and recorded 8bit and 10bit clips of each color on an overcast day.

Starting with the 8bit files, I used a hex viewer to examine the "red" video file, and saw numbers like this (converted to decimal):

114 31 32 112 29 32 [...]

Well, that was easy. A quick check of the "green" video file confirmed that the byte order of each pixel is stored R G B. And indeed, my codec can now open successfully open an 8bit file. Hooray! But wait...

Using the same hex viewer I peek at the 10bit "red" video file, which was recorded using the same exposure as the 8bit file so I kind of know what numbers to expect, and I get this (again converted to decimal):

212 245 129 137 120 123 44 130 156 32 142 136 71 200 34 [...]

The italics are mine, they are (kinda) the R G B values I'm looking for, I just didn't expect them to show up like that. I've seen examples of how 10bit DPX image files are packed but the pattern I see in my files just doesn't make sense to me.

Do they make sense to anyone else? I really want to use these 10bit files, and if I can figure out the R G B pattern I'll be good to go.

For the record I did email the rendering software's developer asking for a hint. He did not reply.

By: pombe

pombe — Mon, 30 Jun 2008 11:41:22 -0800

It's not clear from your post how you did the 10 bit -> decimal conversion, but I can imagine a couple different ways to store the 10 bit data.

1. Packed so that every 10 bits is a value; no alignment between bytes and 10 bit values (i.e. the first 10 bits are red, the next 10 green, etc.

2. Zero padded so that the 10 bit chunks align with byte boundaries (i.e. 6 zeros followed by 10 bits).

3. Zero padded the other way (10 bits followed by 6 zeros).

4. Some more complicated encoding....

If you look at the binary you should be able to tell if it's one of the first three possibilities pretty quickly. Maybe posting some of the binary data here would help?

By: demiurge

demiurge — Mon, 30 Jun 2008 11:46:10 -0800

I've dealt with 12 bit scanner data and all of that was encoded using a 16 bit format that only had some of the bits being used. If this data is uncompressed and you know the number of frames and size of each frame, you can estimate the file size and see if this is true.

By: rajbot

rajbot — Mon, 30 Jun 2008 12:47:31 -0800

It looks like you messed up when you did your hex->decimal conversion for the 10-bit sequence. You should be seeing peaks around 1023 for the red channel, and it looks like you just pasted your hex editor's 8-bit decimal representation. If you expand out what you pasted (assuming RGB packing order) into 10 bits, you get something like ('1101010011b', '1101011000b', '0001100010b') or (851, 856, 98). That's probably not right, but if you post your files we can peak at them. BTW, are these .mov files, or raw bitstreams? If they are mov files, are you sure you are looking in the right place (mdat atom)?

In answer to your question "I wondered how hard it would be to write a Quicktime component", as someone who used to write QT compentents for a living, I would say it if farily complicated. The learning curve is *huge*. I imagine it would take at least a couple weeks for someone who doesn't know the QT ecosystem to just get up to speed.

By: shino-boy

shino-boy — Mon, 30 Jun 2008 13:55:12 -0800

Thank you for your answers so far. I can post some data from the "red" file tonight, after I get home from work.

rajobt:
The video files are raw bitsreams; there's no header or other information in there at all. It did take me several months to get the 8bit component working, being a hobbyist and all.

By: shino-boy

shino-boy — Mon, 30 Jun 2008 13:55:51 -0800

Pardon me, it should be "rajbot".

By: shino-boy

shino-boy — Tue, 01 Jul 2008 00:13:19 -0800

To do the hex->decimal conversion I just read the decimal values from the hex viewer that I've been using.

I uploaded the "red" 10bit video file to my website :

red.dvx1
(right click to download)

I'm trying your suggestions now but am making a lot of mistakes because it's late and I've had a long day.

By: filmgeek

filmgeek — Tue, 01 Jul 2008 03:43:28 -0800

I'd absolutely email two guys; first the guy who does the Glue tools for quicktime, and the second would be Graham Nattress.

Both these guys might be able to provide strong insights on how to develop a codec (both have done serious decoding for film, the Phantom Camera and the Red camera.)

(I'd have posted any other file other than the 'red' one, so there'd be no confusion of what the file is that you have up.)

By: jepler

jepler — Tue, 01 Jul 2008 08:00:47 -0800

Looking at the file in 15-byte chunks in binary there is a clear repetitive structure. (15 bytes is the smallest number of bytes which is a multiple of 30 bits).

110101001111010110000001100010010111100001111011001011001000001010011100001000001000111010001000010001111100100000100010

110011110000010110110010110010000111100010000000000011000100001011011101001000011000011001101100111001111000011100100001

110010101111100100110001110010000111100001111011000100001001001001011101001000001001000001011100101101111100011100100000

111001101111100111100001110010000111000110000110001101001100001000011101000111111000010001000000110001111100011100100000

111000001111100100010001010010010111010101111000111001000100000100011110001000001000100001101100000001110000100000100101

110110110000000100000010010010000111110010000011010011001010001011011101001000011000110101101100100101111000100000100011

110101111111000111010001010010000111110110000011000011001011001011011011000111011000100001011100000101111100100000100000

110100011110110101110001010010000111010001111101010011001101001010011100000111110111110110100100110001111000011100100100

101110001111010110010001100010000111011001111110000010001101001010011100001000011001000111011000111001111100011100100000

^^                               ^^^                               ^^                                ^^^

15*8=120 bits should be 4 pixels, but looking at 10-bit or 30-bit chunks didn't immediately jump out like pixels for me.

110101001111010110000001100010 010111100001111011001011001000 001010011100001000001000111010 001000010001111100100000100010

110011110000010110110010110010 000111100010000000000011000100 001011011101001000011000011001 101100111001111000011100100001

110010101111100100110001110010 000111100001111011000100001001 001001011101001000001001000001 011100101101111100011100100000



1101010011 1101011000 0001100010 0101111000 0111101100 1011001000 0010100111 0000100000 1000111010 0010000100 0111110010 0000100010

1100111100 0001011011 0010110010 0001111000 1000000000 0011000100 0010110111 0100100001 1000011001 1011001110 0111100001 1100100001

1100101011 1110010011 0001110010 0001111000 0111101100 0100001001 0010010111 0100100000 1001000001 0111001011 0111110001 1100100000

1110011011 1110011110 0001110010 0001110001 1000011000 1101001100 0010000111 0100011111 1000010001 0000001100 0111110001 1100100000

1110000011 1110010001 0001010010 0101110101 0111100011 1001000100 0001000111 1000100000 1000100001 1011000000 0111000010 0000100101

1101101100 0000010000 0010010010 0001111100 1000001101 0011001010 0010110111 0100100001 1000110101 1011001001 0111100010 0000100011

1101011111 1100011101 0001010010 0001111101 1000001100 0011001011 0010110110 1100011101 1000100001 0111000001 0111110010 0000100000

1101000111 1011010111 0001010010 0001110100 0111110101 0011001101 0010100111 0000011111 0111110110 1001001100 0111100001 1100100100

1011100011 1101011001 0001100010 0001110110 0111111000 0010001101 0010100111 0000100001 1001000111 0110001110 0111110001 1100100000

The bits of each pixel must be interleaved in some way that is not yet apparent.

How many pixels do you expect in the image? The size of the file is enough for 2505289.8666666667 pixels, which is more than 1920*1200. That the file size is not a multiple of 15 bytes is interesting, too (in fact, the file length of 9394837 bytes is a prime number)

By: shino-boy

shino-boy — Tue, 01 Jul 2008 09:33:58 -0800

I'm sorry, I should have posted more information about the video file:

red.dvx1
770x392 pixels / frame
R G B channels / pixel (no alpha)
progressive (should not be interleaved)

I'm going to try the following method when I get home tonight -
At 10bits I'm expecting the *typical* red pixel value to be, in decimal and R G B order:

458 125 128

I'll convert these values to binary and pack them with my best guess, and see if I can spot a pattern in the real file.

Thanks again for all your suggestions.

By: rajbot

rajbot — Tue, 01 Jul 2008 13:39:39 -0800

Something still seems wrong. The size of a frame in bits should be (770*392*10*3). The file you posted doesn't seem to have an integer number of frames:

>>> 9394837.0*8/(770*392*10*3)
8.3000591925081721..

By: bdc34

bdc34 — Wed, 02 Jul 2008 09:06:57 -0800

I don't want to hold you up on your project and the fun/learning it will provide, but have you considered MEncoder?

Here's an option from the man page:

−rawvideo
This option lets you play raw video files. You have to use −demuxer rawvideo as well.
Available options are:

fps=
rate in frames per second (default: 25.0)

sqcif|qcif|cif|4cif|pal|ntsc
set standard image size

w=
image width in pixels

h=
image height in pixels

i420|yv12|yuy2|y8
set colorspace

format=
colorspace (fourcc) in hex or string constant. Use −rawvideo format=help for a list of possible strings.

size=
frame size in Bytes

EXAMPLE:

mplayer foreman.qcif -demuxer rawvideo -rawvideo qcif
Play the famous "foreman" sample video.

mplayer sample-720x576.yuv -demuxer rawvideo -rawvideo w=720:h=576
Play a raw YUV sample.

By: shino-boy

shino-boy — Wed, 02 Jul 2008 10:06:23 -0800

!

I use mplayer at work but never knew it could play raw files. I'm hoping I can use it to test different raw settings and will try it tonight, thanks.