New SFTP plugin available now

[Jack!]

There's a new cURL version available (7.19.6). With the old version (7.19.5) I couldn't transfer files containing one or more spaces in its filename with the SFTP-plugin in SCP mode. The new cURL version (7.19.6) doesn't work at all with the SFTP plugin in SCP mode. Total Commander (7.5RC1) seems to hang (doesn't respond). Without the SFTP plugin in SCP mode everything works fine. Somebody else having the same issue?

OS: Windows XP SP3 - English
TC: 7.5RC1
SFTP plugin: 0.94 beta
cURL: 7.19.6

[Boofo]

2Jack!,

It works fine in SCP mode. It just takes a little time to come up the first time is all. But... it won't allow you to download any files with it enabled with the new versions of the DLLs. So you are partially right.

2Christian,

Is there any way to add a setting that will let you pick which pane to have the listing show up in (like the FTP setting)?

[Boofo]

Can anyone tell me exactly where the DLLs are supposed to go? I have them in the actual plugin directory but others are saying they need to go elsewhere.

[Jack!]

Can anyone tell me exactly where the DLLs are supposed to go? I have them in the actual plugin directory but others are saying they need to go elsewhere.

According to the SFTP plugin readme.txt you must place files libeay32.dll and libssh2.dll in the same directory as sftpplug.wfx.

[Boofo]

That's the way I understood it but others have said different so I wanted to make sure. Thanks!

[myfreexp]

Very strange indeed, I will check that.

Any idea, WHEN the key behaviour will be fixed...?

Just played the first time with this tool - finally a so-so working SFTP plugin for TC (a lot better than the old PuTTY-based one)!

Nonetheless, found a few bugs:

First some background info: The reason why I was looking for a plugin replacement for the PuTTY based one at all was because I was not able to copy files with 8bit characters (Umlauts) from my local NTFS drive to a Debian server with UTF-8 encoding. This resulted in 'damaged' characters which are shown as "?" when I log in with PuTTY and am viewing them with Midnight Commander (mc).

As to the bugs of the new plugin: In one specific directory, I'm currently seeing the following directory and file in a PuTTY session using mc:

/altsch?ler
5.2 altschüler 1.BMP

The directory "altsch?ler" had been copied by the old SFTP plugin (and apparently not - or not correctly - encoded). The file "5.2 altschüler 1.BMP" had been copied by the old SFTP plugin as well, but afterwards manually renamed directly on the server to show the correct "ü" character.

The new TC plugin is showing instead:

/altsch�
5.2 altschüler 1.BMP

(Where the "/" stands for the folder icon, of course). So the folder name is truncated, and instead of a "?", it shows a block character.

Much more problematic is: I can't even go into the folder with the block character using the <Enter> key. But I can of course directly on the server with mc.

Now I'm going to copy a file with the name "5.2 altschüler 1.txt" from my local drive to the server. Look at the result in TC:

/altsch�tschüler 1.txt
5.2 altschüler 1.BMP
5.2 altschüler 1.txt

Although I just copied a file, the name of the directory seems to have changed!

But of course it has not changed, mc in a parallel PuTTY session is still showing:

/altsch?ler
5.2 altschüler 1.BMP
5.2 altschüler 1.txt

Even if I press <Ctrl-R> or go one folder back and forth again, the display is still wrong in the same way. And I still can't get access to the folder with the block character in TC (but still with mc).

Sorry for the lengthy bug description, but I think it's better to be elaborately rather than meaningless. And if this has been mentioned somewhere else already, my apologies.

[ghisler(Author)]

Try changing the encoding in the connection settings. Apparently the plugin cannot auto-detect the encoding correctly via environment variables (it tries several methods), so you have to set it yourself.

[myfreexp]

Try changing the encoding in the connection settings. Apparently the plugin cannot auto-detect the encoding correctly via environment variables (it tries several methods), so you have to set it yourself.

Thanks for the fast response. But I HAD set the encoding to UTF-8 before my first connect already...

Copying back and forth with correct encoded files/directories is not the problem. The problem is with this not (or incorrect) encoded directory name, created by the old PuTTY based plugin...

[myfreexp]

Try changing the encoding in the connection settings. Apparently the plugin cannot auto-detect the encoding correctly via environment variables (it tries several methods), so you have to set it yourself.

Thanks for the fast response. But I HAD set the encoding to UTF-8 before my first connect already...

Copying back and forth with correct encoded files/directories is not the problem. The problem is with this not (or incorrect) encoded directory name, created by the old PuTTY based plugin...

Let me try a quick analysis. As I fiddled around with UTF-8 decoding end encoding routines a while ago myself, the following is a bit more than just wild guessing:

I believe that the old PuTTY based (and apparently not UTF-8 aware) plugin stores an 8bit-caracter simply as such when it creates a directory (or file) on the server. But a single 8bit character can never be a valid UTF-8 sequence (there are clear rules what is a valid UTF-8 sequence, I have once written a TP routine for this purpose).

Now the decoding comes into place: While the server itself observes these rules and detects the "sequence" as invalid, it stops decoding, displays a "?" for the single 8bit character and leaves the following characters untouched. The directory remains usable for the system.

The decoding routine of the TC plugin (or the DLLs involved?) does apparently not check for a valid UTF-8 sequence, therefore continues decoding to no avail, and this is the reason for the block character as well as for the truncated string. When trying to enter the directory, it logically looks for an incorrect and non-existing directory name.

Please check if this might be true...

[Boofo]

2ghisler(Author),

Any chance on getting the SCP mode bug fixed with the new versions of the DDLs?

[ghisler(Author)]

The problem is with this not (or incorrect) encoded directory name, created by the old PuTTY based plugin...

If you created a folder with ANSI encoding and then list it with UTF-8 encoding, you will of course get garbage.

Solution: Connect with ANSI encoding, rename the folder to an English-only name, disconnect, reconnect with UTF-8, and rename the folder back to what it should be.

Any chance on getting the SCP mode bug fixed with the new versions of the DDLs?

Sorry, what do you mean?

[Boofo]

Any chance on getting the SCP mode bug fixed with the new versions of the DDLs?

Sorry, what do you mean?

With the latest DDLs, SCP mode enabled doesn't allow downloading of files. It just sits there and then gives an error. If I try to view a file in Lister or EditPlus, it doesn't download the file. Copying files doesn't work either in SCP mode with version 7.19.6 of the DLLs.

[myfreexp]

The problem is with this not (or incorrect) encoded directory name, created by the old PuTTY based plugin...

If you created a folder with ANSI encoding and then list it with UTF-8 encoding, you will of course get garbage.

Well, not necessarily. As I said in my bug report, everything is ok directly on the server in a PuTTY session: I have access to all incorrect (i.e. ANSI) encoded directories and files, and all file and directory names are displayed correctly - except for the 8bit character which is replaced by a "?" of course. I can even copy, delete and rename those files. Without having to switch between ANSI and UTF-8 encoding as you say:

Solution: Connect with ANSI encoding, rename the folder to an English-only name, disconnect, reconnect with UTF-8, and rename the folder back to what it should be.

Sounds more like a workaround rather than a solution... We are talking about thousands of files and directories, BTW.

From my point of view, the solution would be to fix the plugin and/or the DLLs, so that everything behaves exactly like the server console itself.

Wouldn't you agree? Or why should the plugin behave different from the server...?

[ghisler(Author)]

Mazbe Putty contains some workaround code to detect false encodings? There is no way to do that with the provided data alone.

[myfreexp]

Mazbe Putty contains some workaround code to detect false encodings? There is no way to do that with the provided data alone.

Well, in fact there is. I don't know if it is PuTTY or the server itself which is responsible for handling false encodings defensively, but yes, it may well be PuTTY. OTOH, I thought PuTTY just displays what the server delivers (so as if you would be sitting right in front of the machine).

As I said in my initial bug post, there are clearly defined rules what is a valid UTF-8 byte sequence and what not. Based on these rules, I once wrote such a function myself years ago, and I'm happy to contribute it - please advise. You would just need to translate it from Turbo Pascal to Delphi. But as PuTTY is open source, the code could probably be found there as well.

The logic then is: If you come across an invalid UTF-8 sequence (and a single 8bit character definitely IS an invalid UTF-8 byte sequence), abort decoding, display an "?" instead of the single 8bit character (or the entire invalid 8bit sequence), and display the remaining 7bit characters "as is". That's what PuTTY (or the server?) apparently is doing. Just test it...

So much to the display... I can't tell exactly what to do with commands such as cd, mv (ren), rm (del) etc., but at least in PuTTY even those will work (but that may well be the server's task). Let's assume you have an ANSI encoded file "Schüler.txt" on an UTF-8 server (which PuTTY will display as "Sch?ler.txt") - then you can invoke the following command on the PuTTY console:

mv Sch?ler.txt Schüler.txt

Afterwards you'll have a correctly and valid UTF-8 encoded file "Schüler.txt" on the server disk. Without having to do the switching from UTF-8 to ANSI and back to UTF-8 again (as you described earlier)!

Instead of using the command line, you can do the same with mc (Midnight Commander), of course.

And I'd really like to convince you to implement the same defensive (and extremely useful) behaviour in the SFTP plugin rather than showing and producing "garbage" (your words). I just don't know who is doing the decoding/encoding at all? Is it implemented in the DLLs or in the plugin itself?

WinSCP by the way crashes in such scenarios (I'm not using WinSCP, but another user told me so) - isn't that a perfect reason for being better than that...?

From my point of view, this would be a sort of "killer feature". Although "feature" is not exactly the right wording in this respect. It is simply how such a plugin should behave.