[BUG] TotalCmd crashes while moving files

[Innuendo]

I have a folder full of files...stats on the folder are 17,845 files weighing in at 1.57 GB.

Windows will move the files just fine. TotalCmd starts out moving okay, but then gets very, very slow moving each file until finally TotalCmd.exe crashes with the following dialog box error message:

Application Error

Exception EOutOfMemory in module TOTALCMD.EXE at 0005E9EA.
Out of memory.

[ghisler(Author)]

One user had the exact same problem, and it was caused by his virus scanner. Do you use one, and if yes, which one?

[Innuendo]

No, I thought that might be the problem so when I reinstalled Windows the other day I tried the above exercise without any anti-virus programs at all running.

All that runs down in my system tray is:

eMule
GetRight
PMMail2000
RemotelyAnywhere
O&O CleverCache

And the first three weren't running when I tried the file move. Total Commander works brilliantly in every other case except when trying this "monster" folder move.

[pdavit]

What happens if you split the procedure into let’s say three segments? i.e. select and move 1/3 of the list then the remaining half and then what’s left.

Is the “break-up” point always close to a specific file? (I guess you can only observe that if the files are big coz otherwise TC is too fast for your eyes!)

[Innuendo]

Oh, breaking it up works...just trying to copy them all at once is the problem.

Something else happened to me earlier this evening while moving some files (A LOT of files).

The moving started and partially through WinXP popped up a dialog box about not enough memory & it was going to increase the swap file size. Then, a while later Total Commander crashed with the same Application Error I posted earlier.

I think there's a memory leak in TotalCmd, but it's only evident when copying a humongous amount of files.

[Innuendo]

I moved a BUNCH of files....and now Total Commander is sitting idle doing nothing & Task Manager showed TotalCmd.exe increasing memory usage all through the file move & now idle TotalCmd.exe is showing 189,244K Memory Usage according to Windows XP's Task Manager.

Total Commander's been sitting idle for about 5 minutes and shows no sign of releasing the RAM, either.

Now...started moving more files. Mem usage briefly dropped down to 120,000K, but quickly spiked back up to 160,000K mem usage and rising.

[Valentino]

Looks like a cancerous growth

[Innuendo]

Spiked over 202,000K and finally gave up with the Application Error I posted above. Total Commander gives up a bit of RAM as the copying goes along, but never enough to reverse the RAM consumption...120K, 150K, 170K, 190K, 150K, 160K, 180K, etc. and will keep going till it consumes all the RAM in the computer.

Contrast that with Windows Explorer. I'm moving the exact same directories and Explorer's RAM usage is staying between 14,600K and 14,900K.

There's either a slow memory leak or a faulty memory management routine in our beloved Total Commander.

But...there's no one I'd rather have work on it than Christian. He can be quite the miracle worker sometimes.

[ghisler(Author)]

I cannot repeat here what you report, sorry. I'm using two tools (memory sleuth and Numega BoundChecker) to make sure that TC has absolutely no meory leaks!

However, some device drivers seem to accept data for copying and report that it's copied, while they just put it in their internal cache (which then overflows). To avoid this, try the following settings in your wincmd.ini:

Under [Configuration], please add
CopyBlockSize=1024
CopyHugeBlockSize=10240
CopyBlockSizeOther=128
CopyHugeBlockSizeOther=1024

This will bypass the Windows disk cache and write directly to the disk. Please be careful because some bad device drivers don't seem to work with this, or work very slowly.

[Innuendo]

Thanks for the suggestion, Christian, but those settings are already present in wincmd.ini. That was one of the first tweaks I did to the ini file.

Any other ideas?

[pdavit]

Let’s follow some more troubleshooting Qs and As to help Christian:

1. Do you get the same behaviour if you copy the files instead of moving them? (I presume this will cause less memory consumption)

2. It that folder you are talking about a network folder by any chance?

3. I guess we can withdraw the chance of defective files because you said that Explorer does the job without any problems and TC works fine if you split that colossal task into smaller segments. Are you using a beta version of TC and if not are you using the latest one?

4. You said your OS is WinXP. Are you running in FAT32 or NTFS?

5. Is there enough free space on your hard drive to handle this colossal moving task? (and that is on the source location not the destination)

6. What are your swap file configurations?


Remark: Why on earth do you keep 17,845 files on a single folder? You don’t like the idea of directories?

[Innuendo]

I'm all for troubleshooting. I just want to get this fixed.

1. Yes, same problem if I just copy instead of move.

2. No, no network folders. All local drives.

3. Using the latest non-beta...5.50 Silent Update 1

4. Yes...WinXP SP1. NTFS on both the source and destination drives.

5. This last go around the source drive had 160 GB free...destination 70 GB free...should be enough, yes?

6. Swap file is set to 512-1024 MB.

I love the idea of directories, but this was sort of a catch-all directory for a bunch of downloaded/lumped-together files and I was moving a bunch of stuff to one hard drive before I undertook the task of sorting all those files into smaller, more manageable directories.


Thanks for the assist, pdavit...judging from Christian's replies, I don't think it's a TotalCmd problem, but a Windows one & hopefully with help, he can work out a work-around for the problem.

[Renard_Roux]

2Innuendo
Let me give a wild guess on your prob. Maybe some part of your bunch of files are archives. In this case if you use zip magic you should turn it off:

wincmd ini:
[Configuration]
...
ZipMagic=1

[Hacker]

Innuendo,
O&O CleverCache

Could this be your problem?

HTH
Roman

[Innuendo]

Renard_Roux, no...I don't use ZipMagic & all the files are JPGs. Good suggestion, though.

Hacker, I installed O&O CleverCache *after* I experienced the problem hoping that CleverCache would help with the problem. It didn't.

I guess this problem will go away once I sorted all the files out of the directory, but it just annoys me that this is a problem currently without a real solution.