+Problems with CopyLinks option

[Sob]

I got excited when I found CopyLinks option in history.txt, but unfortunately, it didn't last me very long. Because as it works now, I must wonder what is it good for.

Example (repeating problems and other unnecessary parts removed):

Code: Select all

F:\source>dir
08.09.2013  20:44    <JUNCTION>     dirjunction [F:\source\testdir]
08.09.2013  20:36    <SYMLINKD>     dirsoftlink [testdir]
08.09.2013  20:42    <JUNCTION>     drivejunction [\??\Volume{eb1569c0-8e28-11e0-b367-806e6f6e6963}\]
08.09.2013  20:37    <SYMLINK>      filesoftlink.txt [testfile.txt]
08.09.2013  20:47    <JUNCTION>     outsidedirjuction [F:\outsidedir]
08.09.2013  21:20    <DIR>          target1
08.09.2013  21:20    <DIR>          target2
08.09.2013  20:35    <DIR>          testdir
08.09.2013  20:37                 6 testfile.txt

F:\source\target1>dir // CopyLinks=1, own subdirectory target
08.09.2013  21:21    <JUNCTION>     dirjunction [F:\source\testdir] // correct
08.09.2013  21:21    <JUNCTION>     dirsoftlink [testdir] // wrong (*2)
08.09.2013  21:21    <JUNCTION>     drivejunction [Volume{eb1569c0-8e28-11e0-b367-806e6f6e6963}\] // wrong (*3)
08.09.2013  20:37                 6 filesoftlink.txt // wrong (*1)
08.09.2013  21:21    <JUNCTION>     outsidedirjuction [F:\outsidedir] // correct

F:\target1>dir // CopyLinks=1, same drive target
08.09.2013  20:49    <JUNCTION>     dirjunction [F:\source\testdir] // correct
08.09.2013  20:49    <JUNCTION>     outsidedirjuction [F:\outsidedir] // correct

E:\target1>dir // CopyLinks=1, different drive target
08.09.2013  20:59    <JUNCTION>     dirjunction [F:\source\testdir] // correct
08.09.2013  20:59    <JUNCTION>     outsidedirjuction [F:\outsidedir] // correct

F:\source\target2>dir // CopyLinks=2, own subdirectory target
08.09.2013  21:21    <JUNCTION>     dirjunction [F:\source\target2\testdir] // correct (*4)
08.09.2013  21:21    <JUNCTION>     outsidedirjuction [F:\outsidedir] // correct

F:\target2>dir // CopyLinks=2, same drive target
08.09.2013  20:48    <JUNCTION>     dirjunction [F:\target2\testdir] // correct (*4)
08.09.2013  20:48    <JUNCTION>     outsidedirjuction [F:\outsidedir] // correct

E:\target2>dir // CopyLinks=2, different drive target
08.09.2013  20:59    <JUNCTION>     dirjunction [E:\target2\testdir] // correct (*4)
08.09.2013  20:59    <JUNCTION>     outsidedirjuction [E:\outsidedir] // correct? (*5)

It doesn't support symlinks at all. File symlinks are ignored (*1) and directory symlinks are incorrectly changed to non-working junctions (*2). Copied drive junctions are also invalid (*3). For CopyLinks=2, changing the target when original one was inside copied directory (*4) is fine. But I'm not sure about outside targets (*5). It's good for copying whole system drive somewhere else, but not necessarily for other uses. And most importantly, unless the option can be changed on the fly in copy dialog, it's close to useless, I'm affraid.

[ghisler(Author)]

TC tries to create junctions instead of symlinks whenever possible, because creating symlinks requires elevation. You should still get symlinks if the source symlink points to a network share.

Can you tell me how to create this "drivejunction", e.g. with the Windows tool mklink? And what is "SYMLINKD"?

[Lefteous]

2ghisler(Author)
I think it's okay to ask for elevation in this case.

Elevation is only required if the SE_CREATE_SYMBOLIC_LINK_NAME privilege is not set for the current user.

[ghisler(Author)]

Unfortunately the SE_CREATE_SYMBOLIC_LINK_NAME privilege isn't even set for administrators on Windows 7/8.

What would be the benefit to create symlinks instead of junctions when the target is just a normal directory?

[MVV]

Privilege isn't set but it is enabled. You only need to set it:

Code: Select all

bool GetCreateSymlinkPrivileges() {
	HANDLE hToken; TOKEN_PRIVILEGES tp;
	bool result=0;

	if (OpenProcessToken(GetCurrentProcess(), TOKEN_ADJUST_PRIVILEGES, &hToken)) {
		tp.PrivilegeCount=1;
		tp.Privileges[0].Attributes=SE_PRIVILEGE_ENABLED;
		if (LookupPrivilegeValue(0, SE_CREATE_SYMBOLIC_LINK_NAME, &tp.Privileges[0].Luid)) {
			result=AdjustTokenPrivileges(hToken, 0, &tp, sizeof(TOKEN_PRIVILEGES), 0, 0)!=0 && !GetLastError();
		}
		CloseHandle(hToken);
	}

	return result;
}

[Sob]

TC tries to create junctions instead of symlinks whenever possible, because creating symlinks requires elevation. You should still get symlinks if the source symlink points to a network share.

Sorry, but that's just wrong. When I want to copy something, I want to end up with the same thing I had before, only somewhere else. What will be next, will TC convert my documents to some better format while copying them? Just kidding of course, but in principle it's the same thing.

Can you tell me how to create this "drivejunction", e.g. with the Windows tool mklink? And what is "SYMLINKD"?

"drivejunction" is whole partition mounted to empty directory on NTFS partition, instead of having standard drive letter. Windows supports it for ages, but no one uses it. You can set it in Disk Management. And I don't actually suppose that anyone is going to copy those, but since I already tested other things..
SYMLINKD is directory symlink (mklink /d link directory), currently changed to unworking junctions by TC with CopyLinks>0. SYMLINK is file symlink (mklink link file), currently ignored by TC (target file is copied instead).

[SWZ]

"drivejunction" is whole partition mounted to empty directory on NTFS partition, instead of having standard drive letter. Windows supports it for ages, but no one uses it. You can set it in Disk Management. And I don't actually suppose that anyone is going to copy those, but since I already tested other things..

That's called (Volume) Mount Point. I used it all the time until very recently. IIRC, it's indeed very similar to Junction but slightly different from it. You can create it via SetVolumeMountPoint API or mountvol.exe commandline tool.

SYMLINKD is directory symlink (mklink /d link directory), currently changed to unworking junctions by TC with CopyLinks>0. SYMLINK is file symlink (mklink link file), currently ignored by TC (target file is copied instead).

I believe the correct terminology in Windows is Symbolic Link, handled by CreateSymbolicLink API.

[ghisler(Author)]

OK, beta 3 can now copy drive mount points (where the link points to a GUID). The problem was that there MUST be a trailing backslash, while there must not be one for directories (except for the root).

Beta 3 will now also copy symlinks to relative paths (without drive letter) as symlinks, which needs elevation.

Privilege isn't set but it is enabled. You only need to set it:

OK, tried that, but on Windows 7 I get an error 1300:
System error code 1300 means "Not all privileges or groups referenced are assigned to the caller."

[MVV]

ghisler,
Have you tried it with elevation? UAC restricts such privileges of course. This code used in my NTLinks plugin and it allows changing symlink targets.

[ghisler(Author)]

Yes, it's not needed to request the privilege when creating the links with elevation - that's what I'm currently doing.

[Sob]

1) Copying mount points works.

There is small difference between original and copy, target of original is listed as: \??\Volume{eb1569c0-8e28-11e0
-b367-806e6f6e6963}\ while copy has Volume{eb1569c0-8e28-11e0-b36
7-806e6f6e6963}\ without \??\. But it does not seem to affect functionality.

2) Copying directory symlinks works with UAC enabled, but does not work with UAC disabled. Ends up with:

Copy:
Could not access 1 folder(s):
x:\target1\dirsoftlink

Test environment:

Code: Select all

cd \
mkdir source
mkdir source\testdir
mkdir target1
cd source
mklink /d dirsoftlink testdir

Then in TC with clean ini (except for CopyLinks=1/2) try to copy dirsoftlink from x:\source to x:\target1.

3) Still no support for file symlinks. Just saying, I know you did not promise to add it, but now that it will work for directories, it would be nice for files too. :)

[ghisler(Author)]

1) OK, I checked this - there are two strings in a file system link:
SubstituteName: The actual target
PrintName: The display name, which is shown by the dir command

I added \??\ to the SubstituteName, but not to the PrintName. It seems that mklink adds it also to the PrintName, so I will change that too.

2) You are right - it is still creating the target directory, but this is necessary only when creating links via DeviceIoControl(FSCTL_SET_REPARSE_POINT but not for symlinks...

3) Sorry, currently only directory symlinks are supported. While it would be quite easy to support the newer symbolic links, the older hard links are quite a nightmare (there is no main file and links pointing to it). But supporting only the newer symbolic links would be inconsistent.

[Sob]

2) If you could make it work also with disabled UAC, it would be nice. Because the whole point of UAC off is to be god with unlimited power and this kind of ruins it.

3) I don't like hardlinks. In short, I find them confusing and impractical (go get me, hardlink lovers!). But if they can be detected - and they can, e.g. ntlinks plugin shows reference count - they should not be hard to copy. As I understand it, you don't need any main file, they all should be equal, so you'd create a link to the one you have.
Except when copying to different partition. I think I see the problem. If someone copied two hardlinked files from one partition to another, they would probably want them to be hardlinked on target too. And because the files can be anywhere in directory structure, you'd have to keep track all the time.
If that's a problem or there are other ones I missed, I would just skip hardlinks for now. Yes, it would be a little inconsistent, but supporting one kind of symlink (directory) and not other (file) is inconsistent too. Even more maybe, because both are still symlinks, in principle the same thing, while hardlinks are more different.

[ghisler(Author)]

2) I have already fixed it here, it will be in beta 4.

3) Hard links can be very tricky to copy, and cause a lot of confusion. For example, on the source, 1.txt and 2.txt point to the same data. On the target, 1.txt points to its own data, while 2.txt and 3.txt point to the same data.

Now you copy 1.txt and 2.txt to the target. What should happen? Should the content of 1.txt be overwritten? Or the content of 2.txt and 3.txt? And should 2.txt and 3.txt still point to the same data? It gets even more confusing if only some of the files are copied.

[Sob]

Again, I don't see supporting file symlinks and not supporting hardlinks as such big problem. Better something than nothing.

Some thoughts about hardlinks:

If copy target exists and it's hardlink, the only right solution is to ask user if they want to simply overwrite the content (and change all linked files) or to unlink specified file from others (i.e. create new) and then overwrite it (so all other previously hardlinked files would not change). Current TC silently chooses first option, which can lead to data loss. If I somehow got two hardlinked files which I'd think are regular non-linked files, then after overwriting one of them, oops, original content of the other file is gone (I just tried that with current beta).

I think it's unreasonable to assume any relation between source and destination, at least for copying (synchronization is exact opposite, so it's more fun). If source contains 1.txt hardlinked to 2.txt and I want to copy them both in one go, copies should be hardlinked too (with CopyLinks>0). It would probably require keeping list of already copied hardlinks and checking all following hardlinks, if they belong to already copied one, and if so, then creating them as hardlinks too. But if I copy 1.txt and then later try to copy 2.txt separately, I can not expect them to be hardlinked on target. There's simply no way for TC to know that this is what I want. That was when target is another partition and copy can't be linked to original. But that's another option for same partition. If I copy c:\a\1.txt together with c:\a\2.txt to c:\b\, I might want just copies linked together but not to originals, but I might also want all four linked together.

And if that wasn't enough, there's synchronization, which has potential to be quite a nightmare...