Delayed file copy/move? Or suggestions on how to approach a large file migration

English support forum

Moderators: Stefan2, white, sheep, Hacker

Post Reply
GISUser
New Member
New Member
Posts: 1
Joined: 2019-09-18, 17:33 UTC

Delayed file copy/move? Or suggestions on how to approach a large file migration

Post by *GISUser » 2019-09-18, 17:51 UTC

I am at a small company that does GIS and has a lot of data files, we just got a new NAS drive and I am tasked with migrating the files from our existing network NAS to the new NAS, however we would also like to restructure the folder organization along the way to consolidate a few drive letters into one location, fix improper placement, etc.

I am not sure if TotalCommander can work how I am imagining this happening, so asking for advice here. There may also be a better approach/tool for this process so I am open to suggestion.

What I need to be able to do is let people keep working off the old NAS while I prepare the data on the new NAS, and then everyone switches to the new NAS when everything is in place. However, I need to be able to make sure any changed files on the old NAS migrate to the new NAS before it goes live. The amount of data moving is a few terrabytes of data, but no idea exactly how much since currently the data is spread out across a few locations and not everything is going to move.

So is there a way that i can set up a mock-move/copy type situation where I define source files and where they should go on a destination, but then trigger the move/copy at a later date? In essence script everything in advance and then hit "GO!" and let all the files copy overnight, then the next morning everyone accesses the data on the new NAS in a new location? I would think then there would likely be less changed files to track down, as the copy will all happen at once. Ideally then I would do a file compare/sync operation but that's tricky as all the files will be in new paths, so not sure what to do on that besides have employees track files they have added since the copy/move script was defined for their work folder, and then manually copy those files to the new NAS locations once the rest of the data in the script has moved.

I posted a similar request for input on Stack Exchange and someone there suggested making a .cmd script in Notepad for the file moves, so all the copying to the new NAS would happen with its existing structure, employees would use the new NAS with old folder structure, then at end of day I run the .cmd script to move the files on the new NAS into their new locations. Sounds like a doable plan, so if that is also something TotalCommander could script and then run, it would solve the time between copying files and employees using new NAS, and hopefully give me a better interface for defining the move of files rather than copy/pasting file paths into Notepad.

Unfortunately the re-sorting of the files into new locations is going to take a while as there are a lot of misplaced files to hunt down and decide on a destination for.

As I mentioned, if there is a better tool/approach to this, I am all ears.

User avatar
Stefan2
Power Member
Power Member
Posts: 2611
Joined: 2007-09-13, 22:20 UTC
Location: Europa

Re: Delayed file copy/move? Or suggestions on how to approach a large file migration

Post by *Stefan2 » 2019-09-18, 18:47 UTC

Hi GISUser and welcome!

No time here to read prose, so can you please explain in a few short words what you are after?


 
Inofficial FAQs || WIKI (Deu/Eng) || TC Home (What's new? // FAQ // Download // Order // Addons // Tools // Plugins)
Erst wenn der letzte Baum gefällt oder die letzte Biene verendet ist, werden die Leute verstehen warum Umweltschutz wichtig ist.

User avatar
Hacker
Moderator
Moderator
Posts: 11272
Joined: 2003-02-06, 14:56 UTC
Location: Bratislava, Slovakia

Re: Delayed file copy/move? Or suggestions on how to approach a large file migration

Post by *Hacker » 2019-09-18, 19:18 UTC

GISUser,
You could always use Commands - Background Transfer Manager - Pause, then add all the necessary operations to the BTM and when you want to start it you could Start it.
There is also Net - FTP Download From List which can also be used for non-FTP transfers:
TC Help File wrote:FTP Download from list
Downloads all files from a user-created list file to the given directory. The list file must contain a list of URLs to files or subdirs (like ftp://ftp.server.com/subdir/file.zip). It may also contain a relative or absolute destination name, separated by an arrow (ftp://ftp.server.com/subdir/file.zip -> c:\local\file.zip). You can add a file to the download list by right clicking on it during an ftp connection, and choosing 'add to download list'. This function also allows to download from WEB servers (http://www.server.com). The list must be saved either as ANSI (plain text) or UTF-8 (with byte order mark BOM at the beginning). If you enter only a list name with no path, it will be searched in the current directory.

Supported commands in the list:
ftp://ftp.server.com/subdir/file.zip download file to base path
ftp://ftp.server.com/subdir/file.zip -> c:\target\file.zip download file to custom destination
put:c:\source\file.zip -> ftp://ftp.server.com/subdir/file.zip upload file
copy:c:\source\file.zip -> c:\destination\file.zip copy file
c:\source\file.zip -> c:\destination\file.zip copy file
move:c:\source\file.zip -> c:\destination\file.zip move file
sleep: 1000 sleep the specified number of milliseconds
header: Referer: http://www.source.com send additional headers like a referer
copyflags: x set overwrite flags (see overwrite options in F5 dialog)
Anyways, neither of these options is really intuitive if you find out that 20 steps back you need to do something different than you already planned.

Stefan2,
can you please explain in a few short words what you are after?
Prepare a large copy/move operation using TC's GUI, then later run it.

HTH
Roman
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.

User avatar
Stefan2
Power Member
Power Member
Posts: 2611
Joined: 2007-09-13, 22:20 UTC
Location: Europa

Re: Delayed file copy/move? Or suggestions on how to approach a large file migration

Post by *Stefan2 » 2019-09-18, 19:39 UTC

Hacker wrote:
2019-09-18, 19:18 UTC
...

Stefan2,
can you please explain in a few short words what you are after?
Prepare a large copy/move operation using TC's GUI, then later run it.

HTH
Roman

Thanks =/ (<< thumpsup)

 
Inofficial FAQs || WIKI (Deu/Eng) || TC Home (What's new? // FAQ // Download // Order // Addons // Tools // Plugins)
Erst wenn der letzte Baum gefällt oder die letzte Biene verendet ist, werden die Leute verstehen warum Umweltschutz wichtig ist.

Post Reply