Multithreaded copy + replace smaller newer files option

[temp0]

Hello ghisler
Thanks for great work on TotalCmd program!
If I can have question. Is it possible to implement multithreaded copy of files (configurable) and also possibility to replace smaller and newer file?
Background: If you will connect cloud drive, you are limited with two things. When any copy process is interrupted, you will get unfinished files with smaller size than original but with newer timestamp because timestamp is the time of copy process and when finished successfully, timestamp is changed to original one. So it will be great to have possibility to replace similar unfinished files - smaller size with newer timestamp.
Second feature is multithreading. In most cases cloud service is very slow for one threaded copy which will not gain full speed of possible download/upload.
It will be great to have possibility to copy e.g. one folder with many files/folders as one job but totalcmd will start multithreaded copies to max download/upload speeds.
Right now it is very slow and complicated to manually copy part of folder in the backgroud, start second part of folder in the background, third part, 4th part, 5th part,...10th part so speed is maxed out (instead of e.g. 150KB/s you will get 2MB/s)
Many thanks for reply and the opinion if this feature can be possible and useful globally, not only for me.
Many thanks!

[MVV]

0. You should create one topic per suggestion to prevent mixing them.
1. It is already possible to copy larger files, just click 'More options' button in overwrite dialog and chose corresponding option.
2. I'm afraid TC is not a download/upload manager and its main purpose is managing directories on local drives where multithreading is usually a bad thing 'cause HDDs don't support parallel operations. Partially such functionality may be implemented as a FS plugin but I don't think it should be in TC itself.

[temp0]

Hello @MVV.
Yes I know, only larger files but without possibility of combination of larger and newer files in case file is created with same size but with actual timestamp (most of the time in my case). What I remember FTP creates increasing size when file is copied but not copy from CD/DVD, Cloud disk or another local disk. Nice to have it as a tickbox so user can combine more options together.
I agree with point 2. Whatever option will be great. Yes, for general usage with classic HDD it is not useful but for "modern" PC/users with cloud drive attached as a drive letter where you don't see difference from local disk it is very useful to have it in any way implemented (as separate FS plugin or directly). Also it is useful for SSD and many small files. Quite faster when multithreaded than as a single copy instance.
I will be very happy if already some FSplugin exists but found nothing.
And sorry for mixing two suggestions together. Next time will do it right.
Thanks

[MVV]

And how do you suggest to decide which file is 'newer (if it is smaller)' and which one is 'larger (if it is older)' when you have smaller files with newer timestamps? If you need to copy older 20-byte file over newer 10-byte file, how TC should know it is interrupted (and should be overwritten) or it is outdated (and should be skipped)?

I don't think SSDs will work faster in a multithreaded way, they will work with same overall speed.

[temp0]

Because when you copy some folders from some source where you know you want these versions of files and you already copied some and you interrputed it, PC get frozen, external disk disconnected, external shared folder disconnected, or you copied only part of source. In this case you want to replace all from source but not already existing but be sure incomplete files will be replaced, files with same size as original but with newer timestamp(time of copying)
In this case it is very useful so you will be sure not necessary overwrites will be done and also incomplete files will be overwritten.

[MVV]

When you chose 'overwrite smaller', TC will do exactly what you want: copy new files and overwrite incomplete ones (with less size) regardless of their timestamps.

[temp0]

In case of FTP etc. but not what I described. When you will copy from cloud disk, external hdd, shared folder, CD/DVD, size in the filesystem is full size also in case whole file is not transfered and it was interrputed. Only difference is timestamp which is newer in case of incomplete file because timestamp is changed after successful copy.
Yes, in case incomplete file has partial size (FTP), it is is easy but not in case size is same and only timestamp differ...and there is no option to "replace only newer" files

[MVV]

Have you tried sync tool? it may be used to delete all newer files from target panel. Just compare dirs, then unclick buttons except ones that TC suggest to copy from cloud drive, then select all and delete...

[temp0]

I wasn't talking about cloud drive only but also about any other attached disk. Yes you can synchronize dirs with "ignore different date" and "compare content" unticked to see different files. Then you must click on every different file and choose copy operation in correct direction. Very uncomfortable and show process. Similar can be done with Ctrl+B to show all files in root folder, sort it by date and manually delete all files with today's date. But when copying more different folders it is unmanageable and uncomfortable but it works. But why not doing things more easily and automated if ghisler will agree it is useful also for others? If not, it is pity, suggestion will be closed, but I asked:-)
For multithreaded copy there is no workaround unfortunately.

[MVV]

There is no need to chose action for every single file, just select them all and open context menu. You need to delete files that are newer in cloud folder, so (if it is right one) uncheck buttons ==, =/= and -> and check only <-, then select all and use delete command from context menu. You can save such sync preset. I know that it isn't useful if you have such problems too often but it is available now.

[temp0]

I know Ctrl+A & Right-Click combo. But synchronize folders doesn't work in this case as I should require.
Imagine one folder named PICS on local HDD. You have five DVD with pics from one action as a wedding. You will copy first DVD to PICS folder, then second or better second at the same time from another dvd drive and when third DVD will be copied it will somehow interrupt. You should merge all five DVDs to this folder.
OK you will try synchronize third DVD. Some files are already copied, some not and some are incomplete. Incomplete with same size but newer timestamp on HDD. In this case "synchronize folders" with unticked "ignore date" and "compare content" will not show files as different ones. When you will check <- it will not show them again because it shows only files with newer timestamp to be copied so older files will be replaced. We need the opposite:-)
When checking opposite direction -> (copying files from HDD to DVD) totalcmd will show them with no preferred action. Great, we see them but also other files are there which are not present on DVD because folder is merged. Not easy way how to get rid of it:-)
I used bad example but tried to describe that it is not so easy in all cases but I know there is a complicated workaround:-)
Right now main problem is for me with cloud drive, I agree. To speed up things I copy few folders divided to more multithreaded copy processes (about 20) and it is quite boring job which complicates in case any of them is interrupted.
Same I do also with local noncloud disks when I am trying to make things organized and cleaned-up. Don't want to wait when one copy job is finished to plan another one so copying some jobs queued and some in the background (unzipping can't be queued) so there are 25 copy processed at one time so it can run whole night without my additional action. Here bad things happen when any job is interrupted. Quite impossible to detect which ones are incomplete, which missing so what is needed to be planned again. Copying all with mentioned suggestion will ease whole copy-repair process.
With my English I little bit overcomplicated the example and description. Sorry for this spam.

[MVV]

Well, I think in such case your choice is 'copy to right' instead of 'delete right'. So, you filter out files except newer on the right side (<- button) and then override copy direction via context menu.