[TC8.0ß3 x64]: sFTP plugin where to put 64-bit SSH libs?

[karlchen]

Hello, Christian.

Could it be that the 64-bit DLLs which the sFTP plugin v1.1 needs and which you offer for download here, 64-bit plugins (download list), actually only holds the 32-bit DLLs, but not the 64-bit DLLs?

openssl+ssh64.zip content:

Code: Select all

libeay32.dll	1.767.936	26.04.2011 22:17	-a--
libssh2.dll	320.000	26.04.2011 22:36	-a--
ssleay32.dll	401.408	26.04.2011 22:17	-a--
zlibwapi.dll	182.784	26.04.2011 22:40	-a--

Evidence:
The sFTP plugin has been installed here: %commander_path%\plugins\sftp. If I put the files libeay32 and libssh2.dll taken from the download file here, only T.C. 8.0ß3 32-bit will be able to establish sFTP connections.
Trying to establish an sFTP connection using T.C. 8.0ß3 64-bit will always only yield the error message which tells me to install libeay32 and libssh2.dll to the plugin folder, %commander_path% or another folder listed in the %PATH% variable.

Side issue:
Putting the 32-bit files libeay32 and libssh2.dll anywhere else than in the sFTP plugin folder will make T.C. 8.0ß3 32-bit fail to find and load the DLLs. (DllLoad=0)

Kind regards,
Karl

[Sob]

No, they are really 64-bit. It has something to do with your anticipation (well, at least a little, because with only one TC you could always put dlls to TC dir, which is always in search path, while with two TCs and dlls with same names you can't do it).

More info:

http://ghisler.ch/board/viewtopic.php?p=227920#227920 (note the "For now, you need to put them in the Total Commander directory.")
http://ghisler.ch/board/viewtopic.php?p=228554#228554

[sqa_wizard]

2karlchen
You just have to put them into subdir "64" under the Totalcmd directory (as stated here)

[karlchen]

Hello, Sob. Hello, sqa_wizard.

Thank you for your helpful input and advice.

About %Commander_path%\64:

This definitely does not work on this system for whichever reason. Trying to establish a connection just gives me the well-known error message which tells me to install libeay32 and libssh2.dll to the plugin folder, %commander_path% or another folder listed in the %PATH% variable.

About %commander_path%:

Though I had tried before putting the 64-bit DLLs provided by Christian here, it had not worked. Now it does. And it is the only location which works for this system.
I assume I had not copied all 4 DLLs to %COMMANDER_PATH%, because the original 32-bit curl DLLs are only libeay32.dll and libssh2.dll, but the 64-bit DLL list seems to be all 4 files, libeay32.dll, libssh2.dll, ssleay32.dll and zlibwapi.dll.
Anyway, after moving all 4 files to %COMMANDER_PATH% and launching "Secure FTP" and selecting a connection, this connection worked. No more error message. Pooh.

About the 64-bit SFTP Plugin v1.1:
Seems to work fine now.

About the 32-bit SFTP Plugin v1.1:
Seems to work fine still.

As this is one of the plugins which are used every day, time will tell pretty quickly whether both plugin flavours will do their job equally well.

Cheers,
Karl
--
Total Commander 8.0ß3 64-bit
Server 2008 R2
SFTP 1.1

[Sob]

1) I don't think dlls in %Commander_path%\64 are supposed to work with SFTP plugin, they're just for internal FTP.

2) If you have 64-bit dlls in %commander_path%, where do you have 32-bit ones? In plugin directory? And if both 32 and 64-bit SFTP plugins work, what about internal FTP with SSL, both 32 and 64-bit? Do they also work at the same time?

[karlchen]

Hi, sob.

First things first:
My assumption - cf. the thread title - proved to be incorrect: [TC8.0ß3 x64]: sFTP plugin does not lack 64-bit SSH libs. T.C. may just fail to find them. Or one may have put them in a folder where it will never look for them.

1) I don't think dlls in %Commander_path%\64 are supposed to work with SFTP plugin, they're just for internal FTP.

Hm. Might be yet another misunderstanding on my side. Will have to go through the thread once again which sqa_wizard pointed me to.

2) If you have 64-bit dlls in %commander_path%, where do you have 32-bit ones? In plugin directory? And if both 32 and 64-bit SFTP plugins work, what about internal FTP with SSL, both 32 and 64-bit? Do they also work at the same time?

All 32-bit SFTP files including the curl DLLs are located in %COMMANDER_PATH%\plugins\sftp.
Have never really tested FTPS, because I have never had access to any FTP server requiring FTPS. So I cannot tell whether they worked or still work.

Kind regards,
Karl

[Sob]

You can test it with http://secureftp-test.com/. Expected result: 64-bit TC will work, while 32-bit will complain about missing OpenSSL dlls. Of course, because the only copy is in SFTP plugin directory and TC won't look there for internal FTP use.

So what to do about it? If we skip Windows and System directories, because we don't want to put anything there (or can't in case of portable TC), it only leaves us %commander_path% where 32-bit TC will look for OpenSSL dlls. 64-bit TC will look there too, but in addition also in 64 subdirectory. So it's clear where to put which set of dlls.

But then 64-bit SFTP won't work, because plugin uses standard library directory search order and it does not include %commander_path%\64. We can put another copy of 64-bit dlls into plugin directory, but it also won't help, because it won't find zlibwapi.dll there. Furtunately 32-bit version does not have zlibwapi.dll, so we can move it to %commander_path%. Then everything will work, but it will be fine mess by my standards. :)

Another option is to keep all 32-bit dlls in %commander_path%, 64-bit dlls in %commander_path%\64 and start 64-bit TC using batch (for regular use you'd of course want something that does not open cmd window):

Code: Select all

set PATH=C:\path\to\totalcmd\64;%PATH%
totalcmd64.exe

Then everything will work just fine without any duplicate dlls.

Unfortunately I don't think it's universal recipe for all plugins.

[karlchen]

<Note>

Changed the thread title
from "[TC8.0ß3 x64]: sFTP plugin lacks 64-bit SSH libs"
to "[TC8.0ß3 x64]: sFTP plugin where to put 64-bit SSH libs?"

Reason:
Christian's download file does hold the 64-bit DLLs. The issue really is where to put them on disk.

</Note>

[karlchen]

You can test it with http://secureftp-test.com/.

Thanks for this piece of advice. Will test it as soon as I have got access to my own Windows 7 64-bit machine. Most of the time it will be hijacked by my wife.

Karl

[karlchen]

Hi, Sob.

You are right.
The combination of
+ SFTP plugin 32-bit
+ SFTP plugin 64-bit
+ FTPS 32-bit
+ FTPS 64-bit
results in unsolvable conflicts, because the names of DLLs which are needed partially conflict which each other.
The next reason is:
If you only use SFTP 32-bit and 64-bit plugins, you can workaround the conflict by keeping the 32-bit DLLs in the plugin folder and put the 64-bit DLLs in the T.C. programme folder.
But in case you also wish to use FTPS, this workaround does not help any longer, because FTPS only looks for the needed DLLs in the T.C. programme folder.
The only functional workaround will be by separating the 32-bit and the 64-bit Total Commander programme folders completely.
As has been said before, at some point in the future the need to run 32-bit and 64-bit Total Commander simultaneously will cease to exist, once most of the 32-bit plugins have been compiled and linked as 64-bit versions as well.
But at the time being, most of the plugins that I use do not have a 64-bit equivalent. Hence, currently I am in a plight.

Kind regards,
Karl

[ghisler(Author)]

The plugin has not been updated yet to support "64" subdir, sorry. Only the internal FTPS client supports it.

[Sob]

@karlchen: It's not unsolvable, I gave you two solutions in previous post. :)

a) Put files like this (base directory being %commander_path%):

Code: Select all

libeay32.dll (32-bit)
libssh2.dll (32-bit)
ssleay32.dll (32-bit)
zlibwapi.dll (64-bit)
64\libeay32.dll (64-bit)
64\ssleay32.dll (64-bit)
plugins\wfx\sftpplug\libeay32.dll (64-bit)
plugins\wfx\sftpplug\libssh2.dll (64-bit)

It works fine even with portable TC. I just don't like duplicate 64-bit libeay32.dll and generally the files in too many locations.

b) If you could live with simple loader (just adding 64 subdir to PATH) for 64-bit TC, then you can have:

Code: Select all

libeay32.dll (32-bit)
libssh2.dll (32-bit)
ssleay32.dll (32-bit)
64\libeay32.dll (64-bit)
64\libssh2.dll (64-bit)
64\ssleay32.dll (64-bit)
64\zlibwapi.dll (64-bit)

Also works fine with portable TC.

c) If you don't care about portable TC and have several other apps also using same dlls (and want to avoid unnecessary duplicates), you can do what I have. All dlls are in c:\my\custom\lib32 and c:\my\custom\lib64 (both in system PATH). And then in %commander_path%:

Code: Select all

libeay32.dll -> softlink to c:\my\custom\lib32\libeay32.dll
ssleay32.dll -> softlink to c:\my\custom\lib32\ssleay32.dll
64\libeay32.dll -> softlink to c:\my\custom\lib64\libeay32.dll
64\ssleay32.dll -> softlink to c:\my\custom\lib64\ssleay32.dll

[karlchen]

Hello, Sob.

Yes, I understood your explanation on how to workaround the faulty design. Yet, I simply will not do it.

Total Commander including its (partially(*)) built-in FTPS support and the SFTP plugin have been created and are maintained by the same developer, Christian Ghisler, not by two independent developers.
When SFTP and FTPS were introduced into Total Commander a long time ago, this was the time to make sure / find a way that both looked for the partially identical DLLs in the same folder(s).
Instead of doing so, as a quick and dirty solution one (FTPS) expected to find the DLLs in %COMMANDER_PATH% and one (SFTP) expected to find them in its plugin folder. (The latter location is fine, in particular for portable installations.)

Now the same approach is used for the 64-bit T.C. The makers of the SSL and SSH libraries do not care about name conflicts between 32-bit and 64-bit DLLs. Why should they? They do not create their DLLs with T.C. 32-bit / 64-bit in mind.
This time, however, the 64-bit SFTP plugin forces us to put the external DLLs into %COMMANDER_PATH% where the 32-bit FTPS looks for its DLLs. Ouch!

As a result this leaves us users in a plight and the need to resort to dirty workarounds.
As we all know backward compatibility at all costs is a one of the holy cows of Total Commander. This has got a lot of advantages, yet, it in this case it is pretty likely to make sure that this plight will never be resolved, but preserved in order to be backward compatible with the T.C. version that first introduced the issue.

Therefore, this time simply working around the problem is a no-go for me. It simply will not happen on any machine that I am responsible for.

From my point of view it would make much more sense to remove the root cause of the DLL hell created by not synchronizing the locations where FTPS and SFTP look for their external DLLs from the very start.
One approach might be by introducing yet another 4 initialization parameters which if present are used:
+ sFTP-32bit-dir=<path to 32-bit curl DLLs>
+ sFTP-64bit-dir=<path to 64-bit curl DLLs>
+ FTPs-32bit-dir=<path to 32-bit curl DLLs>
+ FTPs-64bit-dir=<path to 64-bit curl DLLs>

Kind regards,
Karl
--
(*) partially built-in, because FTPS requires third party DLLs which are not installed by the T.C. installer.

[Sob]

When SFTP and FTPS were introduced into Total Commander a long time ago, this was the time to make sure / find a way that both looked for the partially identical DLLs in the same folder(s).

To be fair to author, SSL support for internal FTP was added sooner than SFTP plugin was created (at that point SFTP was just a distant dream, because there wasn't any library for it and author himself couldn't do much with cryptography because of local laws, as I understood it). And %COMMANDER_PATH% as location for dlls was obvious choice, because system looks for them in executable's directory by default. I don't think any of us would do it differently at that time. So the first step towards current problems was taken.

... and one (SFTP) expected to find them in its plugin folder.

Or also in %COMMANDER_PATH%, or better, in any standard search path. And internal FTPS did initially the same, but was later changed to not look into directories in PATH in the name of security (*1).

So far there was no problem. It came with TC64 and support for having it together with TC32 in the same directory. Which in theory is completely unnecessary, because as user you care only about shared settings. And I don't see why it wouldn't be possible even with separate directories. But - and that's the major problem - it would not just magically work, without user touching anything. And so here comes our friend, the backward compatibility. Don't get me wrong, I love it, but sometimes... ;)

One approach might be by introducing yet another 4 initialization parameters...

I'd rather stay with standard way how system searches for dlls. So I'd remove (*1), which would solve it for people who want to manage their PATHs, lets call them advanced users who know what they're doing and don't need TC to protect them.
And then, as the convenience for others, have e.g. %COMMANDER_PATH%\lib32 and %COMMANDER_PATH%\lib64 and automatically add them at the beginning of search path for whole TC process (lib32 for TC32 and lib64 for TC64). That way anything in there would be available to TC and any plugin. And it would of course be the recommended locations for dlls and therefore safe, because they would be first loaded from there and potentically malicious dlls somewhere in PATH would not be touched.