[bug] TortoiseSVN overlay icons & Vista

[Klug]

After upgrading to Windows Vista RTM I faced a bug with overlay icons in TC. I was using TortoiseSVN-1.3.3.6219, which displays overlay icons for files and folders. With Windows XP this program displays overlay icons both in Explorer and in TC, while with Vista they are displayed only in Explorer (and in Speed Commander), but not in TC . Screenshot: astonshell dot com/tmp/tc_svn_bug.jpg.
TortoiseSVN upgrade to 1.4.1.7992 has changed nothing, neither did TC 6.54a upgrade to 7.0 beta 2.

Please help!

[horizon]

btw: are you sure, you the option of showing overlay icons in tcmd enabled? (cfg-options-icons) (just to be sure )

[Klug]

yea, look at screenshot...

[ghisler(Author)]

Are you using Vista 64? TC is a 32-bit program, so it can only show 32-bit shell extensions. That's not a limitation of TC, it's a limitation of Windows.

[Klug]

We have tested both Vista versions: x32 and x64 - overlay icons are gone. In all our x64 OS, we have installed both TortoiseSVN versions, x32 & x64. BTW TC works correctly with overlay icons in Windows XP SP1 x64. In Vista x32-x64 overlay icons are visible in all OpenFileDialogs (x32) - but not in the TC
(Tried to disable UAC but no luck)

[ghisler(Author)]

I will try to find out what is going on. Are just the Tortoise overays gone, or also the LNK overlays?

[Klug]

Only Tortiose overlays disappeared, LNK overlays are displayed correctly. Moreover, Tortiose overlays are not displayed in TC even in the "Open" dialog of the built-in Lister.
In the exterior Lister.exe the overlay icons are displayed all right. Please have a look at the astonshell dot com/tmp/tc7bug.jpg (Vista RTM x32).

[Klug]

=============
init Log
hooking... C:\WinCmd\TOTALCMD.EXE
hope this helps!
CoCreateInstance:
{30351346-7B7D-4FCC-81B4-1E394CA267EB}, "TortoiseSVN";
{0C6C4200-C589-11D0-999A-00C04FD655E1}, ""
error 0x800401F0 CoInitialize has not been called.
CoCreateInstance:
{30351347-7B7D-4FCC-81B4-1E394CA267EB}, "TortoiseSVN";
{0C6C4200-C589-11D0-999A-00C04FD655E1}, ""
error 0x800401F0 CoInitialize has not been called.
CoCreateInstance:
{30351348-7B7D-4FCC-81B4-1E394CA267EB}, "TortoiseSVN";
{0C6C4200-C589-11D0-999A-00C04FD655E1}, ""
error 0x800401F0 CoInitialize has not been called.
CoCreateInstance:
{3035134B-7B7D-4FCC-81B4-1E394CA267EB}, "TortoiseSVN";
{0C6C4200-C589-11D0-999A-00C04FD655E1}, ""
error 0x800401F0 CoInitialize has not been called.
CoCreateInstance:
{3035134C-7B7D-4FCC-81B4-1E394CA267EB}, "TortoiseSVN";
{0C6C4200-C589-11D0-999A-00C04FD655E1}, ""
error 0x800401F0 CoInitialize has not been called.
CoCreateInstance:
{3035134D-7B7D-4FCC-81B4-1E394CA267EB}, "TortoiseSVN";
{0C6C4200-C589-11D0-999A-00C04FD655E1}, ""
error 0x800401F0 CoInitialize has not been called.
CoCreateInstance:
{3035134E-7B7D-4FCC-81B4-1E394CA267EB}, "TortoiseSVN";
{0C6C4200-C589-11D0-999A-00C04FD655E1}, ""
error 0x800401F0 CoInitialize has not been called.
CoCreateInstance:
{6705C307-E2A4-499A-B6D9-8DC0CA8B2089}, "CMyOverlayIcon Object";
{0C6C4200-C589-11D0-999A-00C04FD655E1}, ""
error 0x800401F0 CoInitialize has not been called.
First-chance exception at 0x75fa3843 (kernel32.dll) in TOTALCMD.EXE: 0x000006BA: The RPC server is unavailable.
CoCreateInstance:
{77F419AA-771A-45FF-AC66-7567FA3243D3}, "SMB Share Engine";
{247BBBD9-9C50-4E7E-A055-68A1E15A3C32}, "IShareEngine"
CoCreateInstance:
{EDB5F444-CB8D-445A-A523-EC5AB6EA33C7}, "Share Manager";
{E3373025-4D9A-4676-B37F-C2C531F7FD87}, "IShareManager"
error 0x00000000 The operation completed successfully.
error 0x00000000 The operation completed successfully.
=============

Is hope enough?

[ghisler(Author)]

It's a bug in Tortoise to assume that all calling apps have called CoInitialize. Total Commander calls InitializeOLE2 only after creating its windows, to make it start faster. I guess that Tortoise wants to hook its dlls before...

[Klug]

>>It's a bug in Tortoise to assume that all calling apps have called CoInitialize.

Wrong!
Tortoise has nothing to do with it.

The previously mentioned CMyOverlayIcon is a test example, which can be found here:
http://www.codeproject.com/shell/overlayicon.asp
and it also doesn't work.

Without CoInitialize any overlay dll does not have a single chance to be loaded in the TC process.
The matter is that overlay icon extension is a COM-server, i.e. a dll, which can be loaded in the client process only during the CoCreateInstance function call, and no other way.
If CoCreateInstance detects that CoInitialize call was not executed beforehand, it will not even try to load the dll.

Please refer to the basics of the COM technology.

If CoInitialize was not called before OverlayManager initialization, NO OVERLAY EXTENSIONS will work!

Using debugger it is easy to find out where OverlayManager is initialized:
TC calls SHGetFileInfo:
==================
api_hook.dll!New_CoCreateInstance(const _GUID & rclsid={30351346-7B7D-4FCC-81B4-1E394CA267EB}, IUnknown * pUnkOuter=0x00000000, unsigned long dwClsContext=0x00000401, const _GUID & riid={0C6C4200-C589-11D0-999A-00C04FD655E1}, void * * ppv=0x000bef74) Line 82 C++
shell32.dll!_SHCoCreateInstance() - 0x292e5 bytes
shell32.dll!_SHExtCoCreateInstance@24() + 0x1e bytes
shell32.dll!_DCA_SHExtCoCreateInstance@16() + 0x2a bytes
shell32.dll!CFSIconOverlayManager::_s_LoadIconOverlayIdentifiers() + 0x10c bytes
shell32.dll!CFSIconOverlayManager::_InitializeHdsaIconOverlays() + 0xb bytes
shell32.dll!CFSIconOverlayManager::CreateInstance() + 0x4c bytes
shell32.dll!_IconOverlayManagerInit@0() + 0x256 bytes
shell32.dll!_GetIconOverlayManager@4() + 0x17 bytes
shell32.dll!_FileIconInit@4() + 0x15e bytes
shell32.dll!_SHGetImageList@12() + 0x90 bytes
shell32.dll!_GetFileInfoSections() + 0xaa bytes
shell32.dll!_SHGetFileInfoW@20() + 0x11b bytes
shell32.dll!_SHGetFileInfoA@20() + 0x70 bytes
TOTALCMD.EXE!00578333()
==================

despite of MSDN:
==================
You must initialize Component Object Model (COM) with CoInitialize or OleInitialize prior to calling SHGetFileInfo.
==================
This is what causes unpredictable behavior, for example, any overlay COM-dll are not loaded into the process.


>>Total Commander calls InitializeOLE2 only after creating its windows, to make it start faster.

This could work for Pentium1-133, but now such an "optimization" is useless, (first if all, because CoInitializeEx call goes very fast with Pentium3 and later, secondly, because Windows2000-XP call CoInitializeEx before the window creation anyway (in contrast to Vista, which lets the developer explicitly set CoInitializeEx parameters)), and even dangerous and harmful.

If you are so eager to keep it - please make it optional!

TC also should call ::CoInitializeEx(NULL, COINIT_APARTMENTTHREADED | COINIT_DISABLE_OLE1DDE)
BEFORE the first addressing to SHGetFileInfo.
Then, TC can create its windows and call OleInitialize...

P.S.
Please look at this function
IShellIconOverlayManager::LoadNonloadedOverlayIdentifiers

[zeflash]

I hope this fix will be in beta3 or rc1 - I'm switching to vista and not having tortoise working properly in TC is a big issue to me!

[ghisler(Author)]

I have moved the initialize function in front of the window creation, but I haven't had the time to test it with Vista yet. I have to attach a different harddisk to test with Vista because it doesn't work under Vmware (my PC has 512 MB of RAM and cannot be upgraded because the chipset doesn't support more).

[good_alias]

It could perhaps be interesting to know that the overlay icons for TortoiseSVN (ver 1.4.1.7992) doesn't show with TotalCommander 6.55 in Vista either
But normally I'm running 7.0 Beta 2.

[zeflash]

I have moved the initialize function in front of the window creation, but I haven't had the time to test it with Vista yet. I have to attach a different harddisk to test with Vista because it doesn't work under Vmware (my PC has 512 MB of RAM and cannot be upgraded because the chipset doesn't support more).

Well then release a beta 3 and I'll tell you if it did the trick or not

[ghisler(Author)]

I have tested it now on Vista Final from MSDN, and it does work. The fix will be in beta 3.