inexperienced geek seeks opengl expert for comfort, support

June 1, 2008 10:29 PM

Any OpenGL coders out there? I want to use my GPU to speed up some video processing, but I know very little opengl. Can you tell me where to focus my research efforts? Here's the application...

I want to perform bilinear demosaicing using the graphics card. I'm pretty sure my algorithm is suitable, since it's all element-wise multiplication and addition of matrices, the stuff that GPUs are supposedly good at. The algorithm goes something like this:

1) get an image from the camera's buffer and send three copies to video memory
2) multiply each copy element-wise by a pre-computed masks of 1s and 0s, so that each "colour matrix" contains only the values of red, green or blue pixels. All other pixels are 0
3) interpolate the missing colour values by averaging the neighboring pixels. This can be done without iteration by adding cropped copies of the colour matrices (element-wise) then doing scalar division.
4) Create a texture from the interpolated colour matrices.
5) Create a 2d primitive to cover the camera's field of view and map the texture to it.
6) Render to screen.

I think I can do 1 with glpushmatrix, and I can probably figure out 4, 5 and 6 (though it looks more difficult than I thought it would be). 2 and 3 are harder to figure out. Any suggestions?

After an afternoon of reading, I thought I might be able to process each colour (and each cropped copy using gltexsubimage2d) as separate textures, then add/multiply them together with multitexturing. But ye gods that looks complicated. Surely there's an easier way?

posted by Popular Ethics to computers & internet (9 comments total)

Pixel shaders using GLSL?
posted by Talez at 10:53 PM on June 1, 2008

I thought about GLSL, but I was hoping to have this run on older and integrated graphics cards. Would that make my life a bunch easier though?
posted by Popular Ethics at 11:10 PM on June 1, 2008

Probably? Do you happen to like the thought of doing GPGPU programming using a C-a-like language?

Integrated graphics chipsets have had pixel shaders for quite a while now and GPUs have had pixel shaders since the R200 and NV20 days. I don't think there'd be any problem using shaders to do your dirty work.
posted by Talez at 11:38 PM on June 1, 2008

If you don't want to use GLSL or Cg, then you'll be doing old-fashioned texture combining. Either you do it in a single pass using the ARB_multitexture and ARB_texture_env_combine extensions, or you'll be doing it with multiple passes and specifying the blend mode. The functions you need to read about are glBlendEquation, glBlendFunc and glBlendColor. That's going to be a slow approach but probably the most portable - all the way back to first-version GeForce and earlier.

Don't forget to use Texture Objects.

I would strongly suggest that you prototype your demosaicing algorithm in C first so that you know exactly what additions/multiplications/etc you need to apply to your images.
posted by polyglot at 11:55 PM on June 1, 2008

I think that using Cg or GLSL will make your life a lot easier. The speedup for older graphics cards won't be that good, but you would probably have to do something fairly sophisticated in order to get it working using just OpenGL 1.2 features. And like polyglot said, doing it that way will be slower.
posted by demiurge at 12:23 AM on June 2, 2008

CUDA. Seriously, if you want to reach the true power of what GPU's are capable of, this is the framework to use right now.
posted by effugas at 1:10 AM on June 2, 2008

CUDA would work nicely too, but if you're worried about the code working on multiple platforms, simple fragment shaders would work fine for this particular application. CUDA only works on new NVIDIA cards. I've found CUDA to be more complicated to program with than GLSL as well, but if your goal is performance, you may want to look into it.
posted by demiurge at 5:48 AM on June 2, 2008

Thanks all. I'm going to take another look at fragment shaders using glsl. Failing that, I'll try ARB_multitexture and ARB_texture_env_combine. Way to go team!
posted by Popular Ethics at 8:12 AM on June 2, 2008

Here's a follow up for anyone who happens along this post. I finally got multitexturing working, and while it was a great learning experience, the results were less than spectacular. The time it takes to overlay two textures onto one quad is about the same time it takes my CPU (and its vector units) to add two matrices, so there wasn't much speed boost in using the GPU for this particular case. OpenGL was noticeably faster at resizing the image though.
posted by Popular Ethics at 11:58 AM on June 15, 2008