TC 7.5 Release hangs on large directory move

Bug reports will be moved here when the described bug has been fixed

Moderators: white, Hacker, petermad, Stefan2

Post Reply
davydm
Junior Member
Junior Member
Posts: 2
Joined: 2009-09-15, 07:24 UTC

TC 7.5 Release hangs on large directory move

Post by *davydm »

I wanted to move a directory containing over 100k files and folders (weighing in at about 12 gig) to another location through TC. I initiated the move, but after popping up the "Rename/Move" dialog, TC seemed to become partially unresponsive. I say partially, because the app doesn't seem to have officially hung -- it's just in a dead state.

I can click the Pause, Cancel and Background buttons on the raised dialog -- they depress (so the app isn't fully hung) but do nothing. The Pause button does toggle its label between Pause and Start but doesn't seem to actually affect anything. The progress bar has stuck at 0%.

The real kickers are that:
(1) examining through explorer, the directory seems to have been moved (and it should have been, in a second or so, since it was moving on the same filesystem). The source directory no longer exists, and the destination seems to have the correct files in it (size and file count is what I expect -- I haven't verified, but I will since this is a large SVN checkout)
(2) Examining the totalcmd.exe process with ProcessHacker shows that TC is systematically reading all of the files in the destination dir. I assume it's doing some kind of check or test -- though I don't know against what since the original directory doesn't exist any more.

All of the above is on WinXP SP3 with TC 7.5 release version.
There is no shame in not knowing -- the shame is in not finding out (Russian proverb)
User avatar
fenix_productions
Power Member
Power Member
Posts: 1979
Joined: 2005-08-07, 13:23 UTC
Location: Poland
Contact:

Post by *fenix_productions »

2davydm
TC needs to calculate the size of all folders/files before starting proper operation (for later progress bar indication). This sometimes gives the feeling of freeze.

To skip such procedure hit Escape once and TC will start moving files immediately.
"When we created the poke, we thought it would be cool to have a feature without any specific purpose." Facebook...

#128099
davydm
Junior Member
Junior Member
Posts: 2
Joined: 2009-09-15, 07:24 UTC

Post by *davydm »

Thanks for the response fenix

Please re-read the report. The directory move *has already happened* at the time of the report, even thou TC appears to be just "sitting there".

I had, in fact, pressed escape at the outset to avoid the progress bar calculation, though the entire process shouldn't have really required a progress bar or calculation. It was a local move from c:\tmp\foo to c:\code\foo, which is one dir move on a local fs -- should be accomplished in about 1 or 2 seconds (on a bad day) since it's just an unlink/relink operation. Try it in explorer or via the commandline and you'll see what I mean.

The bug report is because the interface buttons remain responsive (but don't do anything) and TC is touching the output files (as someone here pointed out, it may be touching last modified times or permissions or something of the like).

I couldn't even background the process (because of the non-working buttons) so I had to kill TC to carry on with other operations.
There is no shame in not knowing -- the shame is in not finding out (Russian proverb)
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

The problem may be caused by a virus scanner: This can hang the actual copy thread for several seconds, while the user interface thread remains active (but cannot stop the hanging copy thread).
Author of Total Commander
https://www.ghisler.com
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

I found out the reason now! Total Commander is trying to set the access rights of moved files and directories to the access rights of the target. In older versions, I was using GetFileSecurityW/SetFileSecurityW, which didn't handle inheritance - so it was very fast.

In TC 7.5, I'm now using SetNamedSecurityInfoW to set an access control list (ACL) which just says "inherit from parent". Strangely, Windows then seems to check the ACLs of ALL directories and files within that directory too! I don't really understand why it does that, but it's very slow.

So the workaround to avoid this is to turn off the copying of the ACL:

Please add the following string to the file wincmd.ini, section [Configuration]:
XPMoveMethod=0

Then the permissions of the folder will be left untouched.

The strange thing is that when I move a file with XP Explorer, the permissions are unchanged too! So maybe I should remove this feature, or use it only when the option to copy permissions is checked? What do you think?
Author of Total Commander
https://www.ghisler.com
ouzoWTF
Junior Member
Junior Member
Posts: 93
Joined: 2009-04-20, 21:03 UTC

Post by *ouzoWTF »

ghisler(Author) wrote:...
use it only when the option to copy permissions is checked
...
8)
Single User Licence #201763
Slofje
Junior Member
Junior Member
Posts: 15
Joined: 2006-08-01, 18:27 UTC
Location: Nederland

Post by *Slofje »

ghisler(Author) wrote: So the workaround to avoid this is to turn off the copying of the ACL:

Please add the following string to the file wincmd.ini, section [Configuration]:
XPMoveMethod=0

Then the permissions of the folder will be left untouched.
Am I right in assuming that this workaround is anyway the sensible thing to do, for the moment?
There are no drawbacks? And the operation is then compatible with previous versions of TC?

BTW, thank you for this great product that I've been using for some years! TC7.5 made me pay attention to this forum again and discovering all kind of unknown features and plugins.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

TC will move the files and directories "as they are", without trying to give them the permissions of the target directory. It seems that the Explorer now moves files and dirs the same way too.

I will probably revert to TC 7.04a behaviour in the next release.
Author of Total Commander
https://www.ghisler.com
User avatar
ado
Senior Member
Senior Member
Posts: 445
Joined: 2003-02-18, 13:22 UTC
Location: Slovakia, Pezinok

Post by *ado »

ghisler(Author) wrote:I will probably revert to TC 7.04a behaviour in the next release.
I'd say it would make sense - even in unix when you move file, it will retain permissions and ownership. When you copy file, it will get permissions of current user/setup (but not target directory)

ado
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

Actually that's not what I wrote - TC 7.04a DID set the permissions of a moved directory to those of the parent, but not the permissions of all the child directories and files (which is actually correct, but can take a long time).

Therefore the next version will do the following:
XPMoveMethod=0: Moved files/dirs keep their permissions
XPMoveMethod=1: as in TC 7.04a: Moved files/dirs get the permissions of the target folder as if they were newly created, but files/dirs in subdirs will not be affected
XPMoveMethod=2: As in TC 7.50 now, but only when the permissions are different: Moved files/dirs get the permissions of the target folder as if they were newly created, also files/dirs in the moved folders

I will make XPMoveMethod=1 the default as in TC 7.04a. This way those who need the new method can still activate it. TC will show a warning in the progress dialog then.
Author of Total Commander
https://www.ghisler.com
Post Reply