Page 1 of 1
SFTP - can't transfer files with spaces in the filename [hetzner Storage Box]
Posted: 2025-05-06, 17:06 UTC
by EATgahh3
Hello,
I'm using the SFTP plugin, but I can't transfer files with spaces in the filename, UL and DL affected.
If I remove the space chars in filename, the file can be transferred ok.
My sftp storage supports filenames with spaces.
Code: Select all
Error reading the file !
///SFTP/..../20250505 Monatlicher Kontoauszug.pdf
tested on Android, multiple devices, SFTP type: hetzner Storage Boxes
Is this a known issue ?
Thanks very much.
Re: SFTP - can't transfer files with spaces in the filename
Posted: 2025-05-07, 09:38 UTC
by ghisler(Author)
No, the problem isn't known. It must be a bug in their SFTP server. This forum is running on Hetzner, and I can upload and download files with spaces in the name just fine.
Please try enabling SCP for transfers, it sends names in a different way.
Re: SFTP - can't transfer files with spaces in the filename
Posted: 2025-05-07, 18:29 UTC
by EATgahh3
Thanks very much for the feedback.
ssh and scp access is enabled in my storage box.
I found the log function in TC, here it is:
Code: Select all
2025-05-07 20:14:57.724 HOST:SFTP:get dir: /
2025-05-07 20:15:04.649 SFTP:refresh list
2025-05-07 20:15:04.815 HOST:SFTP:get dir: /
2025-05-07 20:15:10.940 HOST:SFTP:get dir: /haa/x/
2025-05-07 20:15:11.001 SFTP:=====================
2025-05-07 20:15:11.060 SFTP:IP[0]: 2a01:4f8:xxxxxxx
2025-05-07 20:15:11.061 SFTP:IP[1]: 49.12.xxxxx
2025-05-07 20:15:11.062 SFTP:Connecting...: 49.12.xxxxx
2025-05-07 20:15:11.168 SFTP:Algorithm: RSA, Fingerprint: 3d:7b:6f:99:5f:68:53:21:73:15:f9:2e:6b:3a:9f:e3
2025-05-07 20:15:11.169 SFTP:Negotiated algorithms: [ kex=diffie-hellman-group-exchange-sha256; sig=rsa-sha2-512; c2sCipher=aes128-ctr; s2cCipher=aes128-ctr; c2sMAC=hmac-sha2-256; s2cMAC=hmac-sha2-256; c2sComp=none; s2cComp=none; ]
2025-05-07 20:15:11.170 SFTP:Logging in...
2025-05-07 20:15:11.226 SFTP:Password: ********
2025-05-07 20:15:11.228 SFTP:SFTP start...
2025-05-07 20:15:11.283 SFTP:Connect OK!
2025-05-07 20:15:11.284 SFTP:SFTP LIST /x/
2025-05-07 20:15:11.360 SFTP:CONNECT haa
2025-05-07 20:15:15.821 tc:Copy 1 files:///SFTP/haa/x/->/storage/emulated/0/Download/
2025-05-07 20:15:17.252 SFTP:GET /x/20250505 Monatlicher Kontoauszug.pdf
2025-05-07 20:15:17.312 SFTP:GET error: SCPRemoteException: Remote SCP command returned error: '/x/20250505: No such file or directory
Looks like a quoting issue ?
I found a workaround, when I disable "SCP" option in the SFTP plugin on Android then it works:
Code: Select all
2025-05-07 20:12:15.062 HOST:SFTP:checkConnectionAlive: 1
2025-05-07 20:12:15.063 HOST:SFTP:get dir: /haa/x/
2025-05-07 20:12:15.133 SFTP:=====================
2025-05-07 20:12:15.191 SFTP:IP[0]: 2a01:4f8:xxxx
2025-05-07 20:12:15.192 SFTP:IP[1]: 49.12.xxxxx
2025-05-07 20:12:15.193 SFTP:Connecting...: 49.12.xxxx
2025-05-07 20:12:15.312 SFTP:Algorithm: RSA, Fingerprint: 3d:7b:6f:99:5f:68:53:21:73:15:f9:2e:6b:3a:9f:e3
2025-05-07 20:12:15.315 SFTP:Negotiated algorithms: [ kex=diffie-hellman-group-exchange-sha256; sig=rsa-sha2-512; c2sCipher=aes128-ctr; s2cCipher=aes128-ctr; c2sMAC=hmac-sha2-256; s2cMAC=hmac-sha2-256; c2sComp=none; s2cComp=none; ]
2025-05-07 20:12:15.316 SFTP:Logging in...
2025-05-07 20:12:15.368 SFTP:Password: ********
2025-05-07 20:12:15.369 SFTP:SFTP start...
2025-05-07 20:12:15.422 SFTP:Connect OK!
2025-05-07 20:12:15.423 SFTP:SFTP LIST /x/
2025-05-07 20:12:15.490 SFTP:CONNECT haa
2025-05-07 20:12:19.615 tc:Copy 1 files:///SFTP/haa/x/->/storage/emulated/0/Download/
2025-05-07 20:12:21.043 SFTP:GET /x/20250505 Monatlicher Kontoauszug.pdf
2025-05-07 20:12:34.092 HOST:SFTP:disconnect all: haa
2025-05-07 20:12:34.114 SFTP:OFFLINE! xxxxxxxxxxxx.your-storagebox.de
I don't understand this. Does this makes sense to you ?
BTW: hetzner offers 2 ways to connect: port 22 (which I use) and port 23 (which did here not work for me), Maybe here is a difference ?
https://docs.hetzner.com/storage/storage-box/access/access-overview
Re: SFTP - can't transfer files with spaces in the filename
Posted: 2025-05-08, 08:40 UTC
by ghisler(Author)
It doesn't make sense, it works just fine with ghisler.ch.
Re: SFTP - can't transfer files with spaces in the filename
Posted: 2025-05-08, 16:41 UTC
by EATgahh3
Except the error I can't see a relevant delta in my two provided log files ?
What does the "SCP" option in the SFTP plugin do ? Default is on ?
Checking the hetzer storage box from my linux machine, scp vs sftp: both works fine.
Code: Select all
# x86_64 GNU/Linux 6.13
$ scp haa:x/20250505\ Monatlicher\ Kontoauszug.pdf .
20250505 Monatlicher Kontoauszug.pdf 100% 17KB 483.1KB/s 00:00
$ sftp haa
Connected to haa.
sftp> get x/20250505\ Monatlicher\ Kontoauszug.pdf
Fetching /x/20250505 Monatlicher Kontoauszug.pdf to 20250505 Monatlicher Kontoauszug.pdf
20250505 Monatlicher Kontoauszug.pdf 100% 17KB 479.9KB/s 00:00
sftp>
Of course, filenames must be escapes properly, either by a \ for the spaces, or within double quotes " or single quotes '
And this is what I conclude from post #3 error log : escaping seems not to be done:
Code: Select all
SCPRemoteException: Remote SCP command returned error: '/x/20250505: No such file or directory
Re: SFTP - can't transfer files with spaces in the filename
Posted: 2025-05-08, 17:26 UTC
by ghisler(Author)
The SCP option makes the plugin download the file via scp command instead of via sftp file transfer. Normally scp is faster.
According to the sshj library source file
ScpCommandLine.java, spaces are escaped by a backslash followed by a space.
Code: Select all
public enum EscapeMode {
NoEscape,
Space {
@Override
String escapedPath(String path) {
return path.replace(" ", "\\ ");
}
},
...
I don't understand why this works with all other ssh servers but not with Hetzner's.
Re: SFTP - can't transfer files with spaces in the filename
Posted: 2025-05-08, 19:37 UTC
by EATgahh3
I'm no java expert, but EscapeMode provides 4 modes: NoEscape, Space, DoubleQuote, SingleQuote
But in this java file, no one is choosen /selected ?
Where is decided, which mode is used ?
Or is it extern decided and the sub routine is called with the right mode ?
And you are right: I tested TC Android SFTP versus a fresh VPS server just booted with an Arch Linux iso - no issue here.
Thanks for you testing + hint, that there is no issue with your server.
So there is a rare bug in the Hetzern's storage box product ?
Or does the java code does not detect the correct escape mode with active SCP mode ?
Further test, Android SFTP vs hetzer storage box:
filename with DoubleQuote : not possible for me. Android does not allow filename with DoubleQuote ?
filename with SingleQuote : download direction working, upload not !
Strange...
Re: SFTP - can't transfer files with spaces in the filename [hetzner Storage Box]
Posted: 2025-05-09, 08:57 UTC
by ghisler(Author)
I just found out that my new plugin host is actually a Hetzner storage box, and I can indeed reproduce the issue.
I have further investigated the code: In SCPDownloadClient.java, the sshj implementation uses ScpCommandLine.EscapeMode.NoEscape, while TC uses ScpCommandLine.EscapeMode.SingleQuote, which means that it puts the parameter in single quotes and doesn't escape spaces. The actual command sent is:
scp -t -r -p '/public_html/test/a a.txt'
For test purposes, I have changed to ScpCommandLine.EscapeMode.Space, which sends the following command:
scp -t -r -p /public_html/test/a\ a.txt
However, this also returns an error.
Same problem with downloading, the command lines
scp -f -q -p -r '/public_html/test/a a.txt'
and
scp -f -q -p -r /public_html/test/a\ a.txt
both return an error.
A successful command line for uploading looks like this (no spaces):
scp -t -r -p /public_html/test/b.txt
Re: SFTP - can't transfer files with spaces in the filename [hetzner Storage Box]
Posted: 2025-05-20, 15:38 UTC
by EATgahh3
Sorry, I'm confused now:
is the root cause in the hetzner product, or in sshj or in TC ?
And how to deal with
Code: Select all
07 - Queen - It's A Hard Life.flac
or better
Code: Select all
07 - Queen - It's A Hard Life (12" single version).mp3
Re: SFTP - can't transfer files with spaces in the filename [hetzner Storage Box]
Posted: 2025-05-20, 16:28 UTC
by ghisler(Author)
It's the ssh server used by Hetzner's storage box which causes the problem. Normal Linux installations don't have the problem.
Re: SFTP - can't transfer files with spaces in the filename [hetzner Storage Box]
Posted: 2025-05-21, 16:33 UTC
by EATgahh3
Thanks!
I'm willing to contact hetzner on this bug, but I fear they will reject it.
Their answer will be: scp client with a vanilla Linux (I consider my up-to-date arch linux as one) works fine with their product:
Code: Select all
$ scp haa:x/20250505\ Monatlicher\ Kontoauszug.pdf .
20250505 Monatlicher Kontoauszug.pdf 100% 17KB 484.9KB/s 00:00
$ scp "haa:x/20250505 Monatlicher Kontoauszug.pdf" .
20250505 Monatlicher Kontoauszug.pdf 100% 17KB 484.5KB/s 00:00
$ scp 'haa:x/20250505 Monatlicher Kontoauszug.pdf' .
20250505 Monatlicher Kontoauszug.pdf 100% 17KB 432.8KB/s 00:00
$ pacman -Qi openssh
Name : openssh
Version : 10.0p1-3
Architecture : x86_64
Do you maybe have a hint for me, how we can present this bug to hetzner, so we can prove, it's a bug on their side ?
Re: SFTP - can't transfer files with spaces in the filename [hetzner Storage Box]
Posted: 2025-05-22, 07:00 UTC
by ghisler(Author)
When you use the scp client for Linux, the quotes are handled by the local shell. When using scp via ssh, the quotes are handled by the remote shell. On their box, only double quotes work, not single quotes (used by my plugin) and backslashes for escaping. I could try using double quotes, but this will cause other problems, like variable expansion and command substitution. This has to be escaped too, and I'm sure that this is broken too on their storage box.