Page 1 of 1

Server-Side copy on network shares

Posted: 2016-04-27, 15:57 UTC
by formtapez
Hi,
recently i discovered this beautiful feature of the samba protocol, which allows server-side copy of files on the network share:
https://wiki.samba.org/index.php/Server-Side_Copy

It allows copying files without transferring them over the network. And in combination with a good filesystem (btrfs, zfs, ...) you even dont have to write all data to the harddisk anymore.

From the page above:
Client Support
  • Windows Server 2012 and later: via Windows Explorer or Robocopy
    Windows 8 and later: via Windows Explorer or Robocopy
    Windows Server 2008: via Robocopy only
    Windows 7: via Robocopy only
For other tools, ask the respective vendor for SMB2 FSCTL_SRV_COPYCHUNK support.
Is it possible to get this working with Total Commander, too?

Posted: 2016-04-28, 08:57 UTC
by ghisler(Author)
This would be great, but I'm not aware of any command to do this. FSCTL_SRV_COPYCHUNK is a low level SMB packet, I doubt that I can send that from Windows API...

Posted: 2018-04-08, 03:36 UTC
by alnmawi
To refresh 2yo thread - maybe this will help: aHR0cHM6Ly9tc2RuLm1pY3Jvc29mdC5jb20vZW4tdXMvbGlicmFyeS9nZzU0NzY0NCh2PXZzLjg1KS5hc3B4
There you'll find description of IOCTL_COPYCHUNK and FSCTL_SRV_REQUEST_RESUME_KEY - two control calls that can be used to perform server side copy. Documentation states that those control calls and structures are not defined in any header file, but still - this is development information from MS. Those calls seems to be available at least from Vista SP1, you can read about it in here: aHR0cHM6Ly9ibG9ncy50ZWNobmV0Lm1pY3Jvc29mdC5jb20vbWFya3J1c3Npbm92aWNoLzIwMDgvMDIvMDQvaW5zaWRlLXZpc3RhLXNwMS1maWxlLWNvcHktaW1wcm92ZW1lbnRzLw==

Links are base64 encoded.

Posted: 2018-04-08, 03:38 UTC
by alnmawi
BTW I can't post even if there's a word u_r_l in text. That's quite hard restriction. Just to clarify - I used underscores, cause I couldn't post this.

Posted: 2018-04-09, 14:39 UTC
by ghisler(Author)
Here is the first link (without the version number):
https://msdn.microsoft.com/en-us/library/gg547644.aspx

And here the second:
https://blogs.technet.microsoft.com/markrussinovich/2008/02/04/inside-vista-sp1-file-copy-improvements/

IOCTL_COPYCHUNK looks interesting, but I cannot find any sample code how to call it correctly, especially for bigger files.

But according to this post, CopyFileEx should be using IOCTL_COPYCHUNK:
https://lists.samba.org/archive/samba-technical/2013-February/090667.html

Total Commander does use CopyFileEx already. But this thread only refers to W2K08 (Windows Server 2008), no idea whether it's supported on any other Windows versions...

Posted: 2018-04-11, 03:23 UTC
by alnmawi
I did perform some tests:

server: Ubuntu 14.04.5, Samba 4.3.11, share on btrfs
client: TC on Windows 7

It copies files nearly instantly using server-side copy-on-write available on btrfs. When copying files on same share, but crossing filesystem boundaries on server side, copy-on-write is not available, but still copy is performed server side. But copying between different shares on save server ends up in not using server side copy. I guess it's CopyFileEx implementation or SMB protocol that make it that way.

Anyway, good to know that this works, and needs no further effort on your side. It's just another reason to finally change server filesystems to something that supports copy-on-write :)