Slower CD/flash browse after entering specific archive

[MVV]

Few days ago I noticed strange slow work with flash but I didn't find any reason (TC restart helped).

Now I found reason, it is strange but I'll describe it. You need DiskDirExtended plugin.

0. I can browse my flash fast. I create empty test folder on it, I:\#Other\0 and I:\#Other\0\1. I'm in I:\#Other\0 folder. I can press Enter multiple times per second and TC will go to subfolder "1" and back.
1. I'm entering attached archive (it may be on flash or HDD) on another panel.
2. Now I need 5-10 seconds for each folder change on flash even between folders described in step 0, menu works slowly and entire TC interface hangs for seconds.
3. I'm executing command cm_UnloadPlugins.
4. Now I can browse my flash fast again just like in step 0.

On HDD I can't notice timeouts (maybe because HDD works much faster).

Here contents of archive:

Code: Select all

D:\RECYCLER\
S-1-5-21-1335581083-1701920442-1024176194-0007\	0	2010.6.10	20:43.38
Alpha Beta, The.mkv	6447517243	2010.5.24	18:42.46
Abbarara Betgfdrt.avi	3343410894	2010.5.27	16:25.6
Abbarara Betgfdrt.ENG.srt	313452	2010.5.27	16:25.4
Abbarara Betgfdrt.RUS.srt	225093	2010.5.27	16:25.4

BTW now I tested some other archives created by DiskDirExtended and I can confirm same behaviour.

I don't know why TC works with my flash much slower when DiskDirExtended is loaded.
Why packer plugin slows TC down even when it is not used (after it was used)?

BTW2, TC 7.50a works fine with my flash even if DiskDirExtended is loaded!

TC 7.55pb3 - TC 7.55 final slow down with DiskDirExtended, TC 7.55pb1 also works slowly. So it seems that something was added/changed between TC 7.50a and TC 7.55pb1...

Tested with 2 USB sticks, same behaviour...

If I close TC 7.55 final when it hangs sometimes it shows access violation message but it closes quickly, I can't copy text. I see something like 'stack overflow'. But it is hard to catch message, I saw them only twice (messages were different).

This was tested on Windows XP SP3.

[ghisler(Author)]

Sounds like a threading problem to me: As you may know, TC 7.55 now supports packing and unapcking in the background if the plugin supports it. My guess is that DiskDirExtended detects TC 7.55 and tries to pack or unpack in the background, but doesn't stop its thread when done - the thread probably waits in some kind of busy loop. when TC closes the thread crashes (that's why the exception cannot be caught by TC).

So the author of DiskDirExtended should check his code. He can use something like WaitForSingleObject to let the thread wait without slowing down the system, and then wake it up with some Event object or so. That's what I do with the various TC background threads.

[MVV]

TC eats less than 0.3% of CPU when TC hangs - it is not high value.

Exception windows were in TC style - so TC caught them but closed then.

BTW plugin wasn't updated since 8.08.2009 so I think it can't tell to TC that it supports background operations...

[ghisler(Author)]

I have tried to reproduce your problem now, but without success. Here is what I tried:
1. Downloaded and installed the latest diskdir extended from totalcmd.net
2. Plugged in the slowest USB flash drive I could find
3. Created folders #Other\0 and #Other\1 on it, they are otherwise empty
4. Created lst file with the above contents
5. Created the files in that list file (just a few bytes in it)
6. Switched between #Other\0 and #Other, no problem
7. Entered the lst file -> contents were shown
8. Switched between #Other\0 and #Other, no problem, as fast as before

I watched what TC did with "Process Explorer". No extra threads were created, the existing one didn't go mad.

So what the hell is going on with your system? Any ideas? Am I doing something wrong? I used XP SP3 as you did.

[MVV]

Sorry, I should check it before w/o another plugins. Bug was in GAUP plugin v. 0.6.0.1 (earlier I reported another bug with same archive type and GAUP but I didn't link them now).


You may get buggy plugin here - maybe you'll get some reason why this plugin causes such bugs.


Again, I don't understand why TC loads this plugin if DiskDirExtended supports this archive type and has higher priority. And, I have associated this file extension with DiskDirExtended, why TC loads GAUP plugin for this file?

BTW, if I have GAUP loaded when I opened such archive, all works fine!!!

Christian, can you tell me how TC processes archives?

By Enter:
E.g. I open archive by Enter, TC checks if associated plugin exists and opens archive using that plugin... I can't confirm this logic by tests since in this case TC shouldn't load GAUP plugin when I open archive associated with DiskDirExtended by Enter...

By Ctrl+PgDn:
E.g. TC checks for associated plugin. If no one associated, it loads plugins in specified order and calls CanYouHandleThisFile function with this archive... If no one plugin can't handle archive, TC doesn't open archive. Such logic confirmed by tests.

[ghisler(Author)]

Can you send me a sample archive for this plugin so I can reproduce the error, please?

[MVV]

Can you send me a sample archive for this plugin so I can reproduce the error, please?

Well, I did it already:) you don't need to have archive for GAUP, you need archive for DiskDirExtended to get bug.

New steps to reproduce:

1. Install DiskDirExtended and GAUP plugins from totalcmd.net, assign it to e.g. extensions DIR and GAUP.
2. Save piece of text from first post of this topic as DIR file.
3. Try to open saved file by Enter.

TC loads GAUP and.........

[ghisler(Author)]

I have installed the GAUP plugin now, but I cannot see any slowdown when browsing flash drives. Maybe it needs some other condition too, e.g. a background virus scanner?

[MVV]

I have installed the GAUP plugin now, but I cannot see any slowdown when browsing flash drives. Maybe it needs some other condition too, e.g. a background virus scanner?

I have no any virus scanners or other tools (just firewall in background). As I said, GAUP plugin is loaded when I try to open DIR archive and then TC begins hanging. I don't know what exactly causes this but on clean TC (just UseIniInProgramDir=7) with theese two plugins installed (DiskDirExtended, then GAUP) I get such behaviour. I don't know why TC loads GAUP plugin if DiskDirExtended processes file and don't know what happens but it happens...

I can alwaya reproduce it with clean TC installation and following INI:

Code: Select all

[configuration]
useiniinprogramdir=7
[PackerPlugins]
dir=31,E:\TOTALCMD\Plugins\Packer\DiskDirExtended\DiskDirExtended.wcx
gaup=223,E:\TOTALCMD\Plugins\Packer\GAUP\GAUP_Pro.wcx

BTW I can't reproduce it on my another computer. So I'll try to find differences in Windows configurations (both PCs have XP SP3).
Maybe it will be easier if you try to remember affected changes between 7.50a and 7.55 because TC 7.50a works fine in the same situation.


I noticed that on my PC when I enter archive.dir TC loads DiskDirExtended and GAUP (GAUP then unloaded) but on another PC GAUP isn't loaded at all when I enter same archive! I checked it using Process Explorer's lower pane with loaded DLLs. But C tries to load GAUP only when I'm opening archive from flash, it doesn't happen on local drive. And TC tries to load GAUP on every folder change causing freezes! Tested it with 3 USB sticks (Transcend JF V60 / 8 GB, JF V30 / 2 GB, JF V35 / 4 GB).

Starting TC under admin account changes nothing. I tried to unload Outpost too.

[ghisler(Author)]

I noticed that on my PC when I enter archive.dir TC loads DiskDirExtended and GAUP (GAUP then unloaded) but on another PC GAUP isn't loaded at all when I enter same archive!

Please check the order of the plugins in the ini. The GAUP plugin is implementing the "by content" detection method, which means that TC calls the plugin for each double clicked file to check whether the plugin supports it. If the GAUP plugin comes before the DiskDirExtended plugin, it will be asked first, and if it responds "do not support", TC will pass the name to DiskDirExtended.

TC tries to load GAUP on every folder change

That makes no sense, once the plugin is loaded it will not be loaded again...

[MVV]

Please check the order of the plugins in the ini.

I listed plugin order in INI sample - dir and gaup then.

That makes no sense, once the plugin is loaded it will not be loaded again...

For me it is strange also, but I see it... It seems that TC tries to load GAUP, it crashes, TC tries to load it again and again... When GAUP plugin is loaded, it is immediately unloaded, and while GAUP is in memory, TC hangs. When GAUP is unloaded, I can control TC again.

If I could find any reason why it is so not everywhere...

[ghisler(Author)]

I have checked it now in the debugger, with the GAUP plugin first: LoadLibrary is NOT called here when I press ENTER on file which has same extension as defined for diskdir extended. Here is what I have in my wincmd.ini now:

[PackerPlugins]
gaup=223,c:\totalcmd\wcx\GAUP_PRO\gaup_pro.wcx
ls2=21,c:\totalcmd\wcx\diskdir_extended\DiskDirExtended.wcx

So when I press ENTER on file with extension .ls2, only the diskdir plugin is loaded!

Do you have any color filters or custom hints defined which may be using the GAUP plugin?

[MVV]

So when I press ENTER on file with extension .ls2, only the diskdir plugin is loaded!

I don't know why but it doesn't work for me this way.

Do you have any color filters or custom hints defined which may be using the GAUP plugin?

No, I haven't. I posted above my wincmd.ini used for tests - it contains only two packer plugins in specified order and UseIniInProgramDir=7 option.

Maybe I should try to log TC activity using Process Monitor?

[ghisler(Author)]

Yes, that could indeed be helpful. I just tried it myself, with the following steps and result:
1. Created new wincmd.ini with your contents in TC dir (paths of course changed to my locations of the plugins)
2. Started procmon, filter for TOTALCMD
3. Started TC
4. Renamed diskdir file to test.dir, with above contents
5. Pressed ENTER on it
6. Switched to procmon
7. Searched for word gaup -> no result
8. Searched for word diskdir -> loading of diskdir extended found

EDIT: I found something: It seems like the Diskdir Extended plugin must be responsible for loading the GAUP plugin on your system. If you look in the readme, you find:

Also lists all the other archive files recognised by installed TC plugins.
You can choose which archive types to list and whether to try to determine unknown archive type using installed TC plugins.

However, according to my tests with procmon, the GAUP plugin is only loaded by diskdir extended when I try to add a file with extension .gaup to the diskdir file...

[MVV]

You're right! I completely forgot that DiskDirExtended can use other packer plugins so this can cause some effects. So I need to show contents of DiskDirExtended config.

But it is interesting that GAUP is only loaded on TC 7.55, when I tested same configuration and repeat same test on TC 7.50a, GAUP is not loaded. And, GAUP is loaded only when I'm entering archive on USB stick, not on local drives.

Christian, I've sent you e-mail with corresponding PM logs.