File disappear after copy from server to local

[Feriman]

Hi,

I use sftpplug/SFTP plugin with the latest version of TC (10.00), and I can reproduce this annyoing bug.

The bug: If the remote connection disconnected OR the remote file has been removed, and you copy the already copied file to local, you will lost the local file.

Steps to reproduce the bug:

1., Open Total Commander with already installed sftpplug plugin
2., Open one server inside of Network/Secure FTP on left side
3., Copy any file from the server (left side) to the local machine (right side)
4., Delete this file from the server (DONT through the Total Commander, use console instead)
5., Copy this file again from the server (left side) to the local machine (right side)
6., Error message appear, then the local file disappear

As I remember I can reproduce this bug in TC 9.5x as well, but not sure.

Can you do the same?


Feriman

[ghisler(Author)]

Well, that cannot be avoided - something similar happens when the connection is lost in the middle of a transfer. Then you will have a partially downloaded file which has overwritten the local file.

[Feriman]

Well, I though on the possible solution, and I found two way. They are compatible with each other.

1., Check the connection and file avalibility before starting the copy process. If one of them failed, let's do nothing, just print the error message (Connection lost or file/folder is not available anymore).

2., Copy file into temp file. If the connection lost, remove this temp file. If successful, move this temp file to overwrite the original one. Another possible solution is delete the file into Recycle bin, and restore it from here if connection lost. If successful, you can remove this file from the Recycle bin.

[Dalai]

1., Check the connection and file avalibility before starting the copy process. If one of them failed, let's do nothing, just print the error message (Connection lost or file/folder is not available anymore).

That's what I was going to suggest. Just check if the file is still there before doing the transfer (overwriting the file), and abort if it's not.

2., Copy file into temp file. If the connection lost, remove this temp file. If successful, move this temp file to overwrite the original one. Another possible solution is delete the file into Recycle bin, and restore it from here if connection lost. If successful, you can remove this file from the Recycle bin.

That won't work, when

• the temp directory has not enough free space left for the file in question, or even runs out mid-transfer
• the destination is a network (UNC) share/drive because such locations don't have a recycle bin

As for the last bit of this suggestion: Are applications able to delete files from the recycle bin selectively, is there an API function for this? Emphasis on selectively here.

I see another possible solution: Rename the target file before initiating the transfer and delete the renamed file only after successful transfer. But this also has its drawbacks:

• if the target file has special permissions set, they are lost.
• it requires some space on the destination, likely double the size of the transferred file (maybe even more), just like the way above that goes via recycle bin.

I recommend to manually refresh the source directory right before doing the transfer, and perhaps rename the target file beforehand. Sure, this has its own drawbacks, mainly in the area of convenience.

Regards
Dalai

[ghisler(Author)]

I have already considered the idea of the TEMP file, but the problem is that you wouldn't inherit the permissions of the overwritten file - it would be as if there had been no local file at all.

Checking for the presence of the remote file for each single download is also a problem:
Method 1: Initiate the download, and once it has started, try to create the local file and prompt the user for overwriting. This is a problem if the user doesn't react quickly enough (connection can time out), or the local file cannot be opened for writing (e.g. locked by another app), so we now need to somehow abort the operation.
Method 2: Make an extra call to the server before each download just to check whether the file is still there. This would be slower, and wouldn't ensure that the remote file can actually be opened.

Currently I'm considering a third method, where I open the target file for writing, but don't truncate it yet. This is a different CreateFile option, so I will have to check first whether it has any negative effects. For example, the opening could work but the truncation could fail when there are partial locks in the middle of the file (e.g. a database).

[ghisler(Author)]

I have added method three now to the SFTP plugin -> new version 2.90:
https://www.ghisler.com/plugins.htm#filesys

Please let me know what you think. I do exactly what I suggested above:
1. Open the target file for writing, but don't truncate it yet.
2. Start the download
3a. Once we have received at least one byte, truncate the target
3b. If the download fails, keep the target unchanged

Please let me know what you think! The Cloud and Webdav plugins use the same (old) logic, so I will modify them too if you think that it's a good solution.

[Feriman]

Nice job, works as expected! Thank you for fixing this bug!