Server-Side copy on network shares

Here you can propose new features, make suggestions etc.

Moderators: white, Hacker, petermad, Stefan2

Post Reply
formtapez
Junior Member
Junior Member
Posts: 48
Joined: 2004-02-13, 19:41 UTC

Server-Side copy on network shares

Post by *formtapez »

Hi,
recently i discovered this beautiful feature of the samba protocol, which allows server-side copy of files on the network share:
https://wiki.samba.org/index.php/Server-Side_Copy

It allows copying files without transferring them over the network. And in combination with a good filesystem (btrfs, zfs, ...) you even dont have to write all data to the harddisk anymore.

From the page above:
Client Support
  • Windows Server 2012 and later: via Windows Explorer or Robocopy
    Windows 8 and later: via Windows Explorer or Robocopy
    Windows Server 2008: via Robocopy only
    Windows 7: via Robocopy only
For other tools, ask the respective vendor for SMB2 FSCTL_SRV_COPYCHUNK support.
Is it possible to get this working with Total Commander, too?
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48028
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

This would be great, but I'm not aware of any command to do this. FSCTL_SRV_COPYCHUNK is a low level SMB packet, I doubt that I can send that from Windows API...
Author of Total Commander
https://www.ghisler.com
alnmawi
Junior Member
Junior Member
Posts: 3
Joined: 2018-04-08, 03:02 UTC

Post by *alnmawi »

To refresh 2yo thread - maybe this will help: aHR0cHM6Ly9tc2RuLm1pY3Jvc29mdC5jb20vZW4tdXMvbGlicmFyeS9nZzU0NzY0NCh2PXZzLjg1KS5hc3B4
There you'll find description of IOCTL_COPYCHUNK and FSCTL_SRV_REQUEST_RESUME_KEY - two control calls that can be used to perform server side copy. Documentation states that those control calls and structures are not defined in any header file, but still - this is development information from MS. Those calls seems to be available at least from Vista SP1, you can read about it in here: aHR0cHM6Ly9ibG9ncy50ZWNobmV0Lm1pY3Jvc29mdC5jb20vbWFya3J1c3Npbm92aWNoLzIwMDgvMDIvMDQvaW5zaWRlLXZpc3RhLXNwMS1maWxlLWNvcHktaW1wcm92ZW1lbnRzLw==

Links are base64 encoded.
alnmawi
Junior Member
Junior Member
Posts: 3
Joined: 2018-04-08, 03:02 UTC

Post by *alnmawi »

BTW I can't post even if there's a word u_r_l in text. That's quite hard restriction. Just to clarify - I used underscores, cause I couldn't post this.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48028
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

Here is the first link (without the version number):
https://msdn.microsoft.com/en-us/library/gg547644.aspx

And here the second:
https://blogs.technet.microsoft.com/markrussinovich/2008/02/04/inside-vista-sp1-file-copy-improvements/

IOCTL_COPYCHUNK looks interesting, but I cannot find any sample code how to call it correctly, especially for bigger files.

But according to this post, CopyFileEx should be using IOCTL_COPYCHUNK:
https://lists.samba.org/archive/samba-technical/2013-February/090667.html

Total Commander does use CopyFileEx already. But this thread only refers to W2K08 (Windows Server 2008), no idea whether it's supported on any other Windows versions...
Author of Total Commander
https://www.ghisler.com
alnmawi
Junior Member
Junior Member
Posts: 3
Joined: 2018-04-08, 03:02 UTC

Post by *alnmawi »

I did perform some tests:

server: Ubuntu 14.04.5, Samba 4.3.11, share on btrfs
client: TC on Windows 7

It copies files nearly instantly using server-side copy-on-write available on btrfs. When copying files on same share, but crossing filesystem boundaries on server side, copy-on-write is not available, but still copy is performed server side. But copying between different shares on save server ends up in not using server side copy. I guess it's CopyFileEx implementation or SMB protocol that make it that way.

Anyway, good to know that this works, and needs no further effort on your side. It's just another reason to finally change server filesystems to something that supports copy-on-write :)
Post Reply