Windows Transparency Problems

February 20, 2006 7:38 PM

OK, so I figured out, through lots of painful browsing of MSDN, how to make windows (partially) transparent, and also how to chroma-key windows. And it works great, but only in windows 2000. In windows XP it works only in certain cirumstances. Need windows programmer to help me...

Here's the basic code snippet, where title and red/green/blue are defined upstream of this.

HWND hwnd = FindWindow(NULL, title);
COLORREF crkey = RGB(red, green, blue);

SetWindowLong(hwnd, GWL_EXSTYLE, GetWindowLong(hwnd, GWL_EXSTYLE) | WS_EX_LAYERED);
SetLayeredWindowAttributes(hwnd, crkey, alpha, LWA_COLORKEY | LWA_ALPHA);

This is like textbook, out of the windows example pages.

* It works on windows 2000
* It works on windows XP, EXCEPT it seems that I can only do it to windows that are NOT related to main application that is running the code above. That is, I can pick, say, my email window and make it transparent, but not another toplevel in my application.

I actually suspect that it may be a problem with how me/windows is "finding" the window. That first line where I get the HWND, I suspect that maybe it is not finding the window, under windows XP, if it's part of my application. The windows MSDN pages indicate that a different method is used if the window is/is not part of the application, so I suspect that may be the source of the problem.

I'm a unix programmer, doing this in windows out of necessity and I am in way over my head. I either need a better way to "find" window handles, or I need a way to verify whether FindWindow is finding anything, or if it's finding the "right" window.

The C code above is in a module, called by a scripting language called Tcl/Tk, so I don't really have low-level access to the toplevel windows in other ways (that is, I'm not explicitly creating them myself)

posted by RustyBrooks to computers & internet (15 comments total)

Can you debug or rewrite the C module code so that you know where the point of failure is? There are at least 4 possibilities, from what I can see. For example (replacing OutputDebugString with the equivalent TCL debug output command):


COLORREF crkey = RGB(red, green, blue);
HWND hwnd = FindWindow(NULL, title);

if ( !hwnd )
{
 OutputDebugString("FindWindow returned NULL");
 return -1;
}

LONG lWnd = GetWindowLong(hwnd, GWL_EXSTYLE);
if ( !lWnd )
{
 OutputDebugString("GetWindowLong failed");
 return -1;
}

LONG lRes = SetWindowLong(hwnd, GWL_EXSTYLE, lWnd | WS_EX_LAYERED);
if ( !lRes )
{
 OutputDebugString("SetWindowLong failed");
 return -1;
}

BOOL fSuccess = SetLayeredWindowAttributes(hwnd, crkey, alpha, LWA_COLORKEY | LWA_ALPHA);
if ( !fSuccess )
{
 OutputDebugString("SetLayeredWindowAttributes failed");
 return -1;
}

return 0;

posted by helios at 8:06 PM on February 20, 2006

Why not just make all the windows 'non-main' and hide the main application window? I realize that would be a bit of a pain in the ass, but its something you could do if no one else has any suggestion.
posted by delmoi at 8:07 PM on February 20, 2006

OK, I've got a little more info from debugging.

It's not a problem with FindWindow. FindWindow is getting a handle, and it's the right one. Also, I use FindWindow for some other window manipulating (like setting the window to be "on top") and that works.

SetLayeredWindowAttributes returns an error when it succeeds. The error is something like "Operation Completed Successfully". Hey, thanks.

I'll check and see if I can tell if SetWindowLong or GetWindowLong is the problem. GetWindowLong should be OK since I use the same code in other places, but SetWindowLong may have some kind of problem.

delmoi: I don't have control over the windows in question. And I think I might have phrased it badly. Windows that belong to other applications, I can set the alpha on those. Windows that belong to my application, I can not. (Under windows XP. In windows 2000, it works fine)
posted by RustyBrooks at 8:13 PM on February 20, 2006

SetLayeredWindowAttributes is not returning an error, nor is GetLayeredWindowAttributes, nor is SetWindowLong or GetWindowLong.
posted by RustyBrooks at 8:21 PM on February 20, 2006

OK, this is weird.

The whole reason that I even started trying this myself is I found a (Delphi) application that lets you control the transparency and chroma keying of all your windows. It was like 10 lines long so I figured it couldn't be that hard in C.

I just hauled out that program and tried it out under windows XP. For Tcl/Tk windows, setting the transparency to ANYTHING makes it totally transparent. Also, setting the chroma-key color to ANYTHING makes the whole window transparent. I'm starting to think there's something odd about how Tcl/Tk is creating it's windows.

Is there some combination of attributes the windows might have that would cause this? Where to look and what to look for?

It's all so frustrating because it frikkin works great under win2k.
posted by RustyBrooks at 8:28 PM on February 20, 2006

Argh. it's a problem with how tcl creates windows
posted by RustyBrooks at 8:31 PM on February 20, 2006

You cannot layer child windows. Use the WS_POPUP attribute when creating the windows to layer, instead of the WS_CHILD attribute.

To change existing child windows in your application to pop-up style, use the SetParent and follow the MSDN: "For compatibility reasons, SetParent does not modify the WS_CHILD or WS_POPUP window styles of the window whose parent is being changed. Therefore, if hWndNewParent is NULL, you should also clear the WS_CHILD bit and set the WS_POPUP style after calling SetParent.".

I've not tried it, but seems like it should work.
posted by mdevore at 8:57 PM on February 20, 2006

These aren't child windows. They're toplevel windows. It does work under windows 2000. FindWIndow itself only finds toplevel windows, not child windows, and it does find these, so I'm pretty sure they're not child windows.
posted by RustyBrooks at 9:00 PM on February 20, 2006

OK,a bit hard to say what might be going on from the 4-line snippet, I didn't know whether you could be FindWindowEx'ing for children in another location. As you alluded to, Windows 2000 doesn't necessary suffer identical restrictions because layered windows rules were changed a bit in XP.

Anyway, as you probably know, class styles of CS_OWNDC or CS_CLASSDC are disallowed for WS_EX_LAYERED. There's something about restrictions on different sessions too, as I recall, but can't find the reference right now.
posted by mdevore at 9:19 PM on February 20, 2006

Do you have access to visual studio? You could figure out what's different between your TCL/TK windows and other windows by using the Spy++ utility and checking the window styles/classes.

My guess would be that 1) transparent/chroma keyed windows require a registered class name, 2) you have to remove or set the WS_CLIPCHILDREN flag, or 3) the TCL/TK window has child windows with unusual properties. But these are just guesses based on similar issues I've seen.
posted by helios at 9:47 PM on February 20, 2006

I do have visual studio, but not installed on an XP box. I'll have to see if I can dig up the CDs at work and install it on my laptop.
posted by RustyBrooks at 5:38 AM on February 21, 2006

You can try creating an AutoIt script that uses the WinSetTrans function to see if it's a problem in your code or not.

If it works correctly you can copy the source code from the AutoIt source files (AutoIt_Script::F_WinSetTrans in src/script_win.cpp)
posted by Sharcho at 10:48 AM on February 21, 2006

I might as well paste the code:
///////////////////////////////////////////////////////////////////////////////
// WinSetTrans
//
// Sets the transparency of a window (Windows 2000/XP or later)
// Takes 3 parameters:
// vParam[0] - Window Title
// vParam[1] - Window Text
// vParam[2] - Integer that controls the transparency
///////////////////////////////////////////////////////////////////////////////
AUT_RESULT AutoIt_Script::F_WinSetTrans(VectorVariant &vParams, Variant &vResult)
{
#ifndef WS_EX_LAYERED // Only defined on Windows 2000+
#define WS_EX_LAYERED 0x00080000
#endif
#ifndef LWA_ALPHA
#define LWA_ALPHA 0x00000002
#endif

typedef BOOL (WINAPI *SLWA)(HWND, COLORREF, BYTE, DWORD); // Prototype for SetLayeredWindowAttributes()

uint value = vParams[2].nValue() < 0 ? 0 : vparams[2].nvalue(); // valid range is 0 - 255br>
Win_WindowSearchInit(vParams);
if (Win_WindowSearch() == false)
return AUT_OK; // No window

HMODULE hMod = LoadLibrary("user32.dll");
if (!hMod)
return AUT_OK; // If this happens, we have major OS issues.

SLWA lpSetLayeredWindowAttributes = (SLWA)GetProcAddress(hMod, "SetLayeredWindowAttributes");
if (lpSetLayeredWindowAttributes)
{
LONG style = GetWindowLong(m_WindowSearchHWND, GWL_EXSTYLE);
if (value >= 255 && (style & WS_EX_LAYERED))
SetWindowLong(m_WindowSearchHWND, GWL_EXSTYLE, style ^ WS_EX_LAYERED); // Remove
else
{
SetWindowLong(m_WindowSearchHWND, GWL_EXSTYLE, style | WS_EX_LAYERED);
lpSetLayeredWindowAttributes(m_WindowSearchHWND, 0, value, LWA_ALPHA);
}
vResult = 1;
}
else
SetFuncErrorCode(1); // This means the OS isn't supported since the function wasn't loaded from the DLL
FreeLibrary(hMod);

return AUT_OK;

} // WinSetTrans()

posted by Sharcho at 10:51 AM on February 21, 2006

So I checked a tcl window compared to a regular/blendable window (mozilla) and here are some settings that are in the Tcl one but not the Mozilla one:

WS_OVERLAPPED | WS_BORDER | WS_SYSMENU | WS_THICKFRAME | WS_MINIMIZEBOX | WS_MAXIMIZEBOX | 0x04194312

Any idea what that last one is? 0x04194312

Of all of these, WS_OVERLAPPED seems like a likely candidate to be screwing it up. I'll be looking all these up later today to see if I can figure it out.
posted by RustyBrooks at 5:56 AM on February 22, 2006

I take that back, I was accidentally looking at a Mozilla Child Window. The only difference is WS_CLIPCHILDREN -- which is the one mentioned above. I'll try unsetting it,
posted by RustyBrooks at 5:59 AM on February 22, 2006