(again) Slow FTP transfer

[hlloyge]

I can't believe this I've read all threads here regarding this subject, made changes to wcx_ftp.ini, entered 115200 as chunk size, and still I get slow FTP transfer. I am on gigabit LAN, and the max transfer speed I get to linux FTP server (vsftpd) is 25 MB/sec. Passive, non passive, default size, 115200... it doesn't matter. When transfering with FileZilla, I get 50-70 MB/sec, which is the max speed of software RAID5 on that server (tested). I know this isn't so high on mr. Ghisler list of priorities, because it works, just slower but please, can you check out what is the source of this? I don't transfer often, but when I do, it is rather large file sets, so speed is somewhat important
Is there some mode where this can be debugged? I'd like to help as much as I can, I am not programmer, but I am skillfull with computers, and I'd like to help somehow to solve this problem.

[hlloyge]

Bumping the thread.

[ghisler(Author)]

Sorry, I cannot say where the bottleneck is in your case.

[hlloyge]

Do you use some outside ftp library, or do you have your own written library?

[ghisler(Author)]

I'm not using any outside library, all direct access to wsock32.dll or ws2_32.dll.

[Mjolnir]

Hum, very interresting, I get same problem.

Here mine :

- Local system : XP SP2 // TC 7.50 RC2
--- wsock32.dll : 32088 oct (vers 5.1.2600.2180)
--- ws2_32.dll : 82944 oct (vers 5.1.2600.2180)
- Distant system : Solaris 10 OS

For a 20Mb file :
-> FTP using TC internal FTP : ~60 to 150 koctets/s
-> FTP using FileZilla : 2.8 MB/s
-> FTP using TC internal FTP (= try again) : ~60 to 150 koctets/s

(and this also occurs wuth same behavior on another colleage system running with XP SP2 & TC7.5PB5)

Any clue ?


Another question : is there away to use an external dll from TC (and continue to use the "ftp GUI popup window" from TC), rather the wsock32 or ws2_32 ?

[ghisler(Author)]

Sorry, no clue, I get > 20 MB/sec with a local server...

[Mjolnir]

Tried again this morning with XP's ftp.exe and works with same transfert rate that one I get with Filezilla.
I've also tried an old TC : unpack32_656.exe, always same bad transfert rate.

Here my TC log dump for a 2Mo file :

Code: Select all

221-Total traffic for this session was 765859 bytes in 3 transfers.
221-Thank you for using the FTP service on SolarisHost1.
221 Goodbye.
----------
Connect to: (09/02/2009 10:58:05)
hostname=SolarisHost1
username=lsav
startdir=
SolarisHost1=XXXXXXX
220 SolarisHost1 FTP server ready.
USER lsav
331 Password required for lsav.
PASS ***********
230-Please read the file README.txt
230-  it was last modified on Wed Jul 16 16:31:33 2008 - 412 days ago
230 User lsav logged in.
SYST
215 UNIX Type: L8 Version: SUNOS
FEAT
500 'FEAT': command not understood.
CLNT Total Commander (UTF-8)
500 'CLNT Total Commander (UTF-8)': command not understood.
OPTS UTF8 ON
500 'OPTS UTF8 ON': command not understood.
Connect ok!
PWD
257 "/export/home/lsav" is current directory.
Lire le répertoire
TYPE A
200 Type set to A.
PORT XXXXXXX,13,5
200 PORT command successful.
LIST
150 Opening ASCII mode data connection for /bin/ls.
Téléchargé¦s
En attente du serveur...
226 Transfer complete.
TYPE I
200 Type set to I.
PORT XXXXXXX,13,6
200 PORT command successful.
STOR 2Mo.gz
150 Opening BINARY mode data connection for 2Mo.gz.
Envoyés: 1,910,002 octets, 109 koctets/s
226 Transfer complete.
SIZE 2Mo.gz
213 1910002
Copied (09/02/2009 10:58:21): C:\Temp\2Mo.gz -> ftp://SolarisHost1/export/home/lsav/2Mo.gz 1,910,002 octets, 150 koctets/s
Lire le répertoire
TYPE A
200 Type set to A.
PORT XXXXXXX,13,7
200 PORT command successful.
LIST
150 Opening ASCII mode data connection for /bin/ls.
Téléchargé¦s
En attente du serveur...
226 Transfer complete.

And here log for FileZilla (always "faster") :

Code: Select all

Status:	Resolving address of SolarisHost1 
Status:	Connecting to SolarisHost1:21...
Status:	Connection established, waiting for welcome message...
Response:	220 crux.grenoble.ferma FTP server ready.
Command:	USER lsav
Response:	331 Password required for lsav.
Command:	PASS ****
Response:	230-Please read the file README.txt
Response:	230-  it was last modified on Wed Jul 16 16:31:33 2008 - 412 days ago
Response:	230 User lsav logged in.
Command:	SYST
Response:	215 UNIX Type: L8 Version: SUNOS
Command:	FEAT
Response:	500 'FEAT': command not understood.
Status:	Connected
Status:	Retrieving directory listing...
Command:	PWD
Response:	257 "/export/home/lsav" is current directory.
Command:	TYPE I
Response:	200 Type set to I.
Command:	PASV
Response:	227 Entering Passive Mode (XXXXXXX,235,86)
Command:	LIST
Response:	150 Opening BINARY mode data connection for /bin/ls.
Response:	226 Transfer complete.
Status:	Directory listing successful
Status:	Resolving address of SolarisHost1
Status:	Connecting to XXXXXXX:21...
Status:	Connection established, waiting for welcome message...
Response:	220 SolarisHost1 FTP server ready.
Command:	USER lsav
Response:	331 Password required for lsav.
Command:	PASS ****
Response:	230-Please read the file README.txt
Response:	230-  it was last modified on Wed Jul 16 16:31:33 2008 - 412 days ago
Response:	230 User lsav logged in.
Status:	Connected
Status:	Starting upload of C:\Temp\2Mo.gz
Command:	CWD /export/home/lsav
Response:	250 CWD command successful.
Command:	PWD
Response:	257 "/export/home/lsav" is current directory.
Command:	TYPE I
Response:	200 Type set to I.
Command:	PASV
Response:	227 Entering Passive Mode (XXXXXXX,209,3)
Command:	STOR 2Mo.gz
Response:	150 Opening BINARY mode data connection for 2Mo.gz.
Response:	226 Transfer complete.
Status:	File transfer successful
Status:	Retrieving directory listing...
Command:	PASV
Response:	227 Entering Passive Mode (XXXXXXX,96,175)
Command:	LIST
Response:	150 Opening BINARY mode data connection for /bin/ls.
Response:	226 Transfer complete.
Status:	Directory listing successful

Remarks :
- I've changed my IP & real Host Name in the traces for (privacy) by XXXXXXX or SolarisHost1
- from TC log, seems there are little error at "connection", but connection still occurs :

Code: Select all

FEAT
CLNT Total Commander (UTF-8)
OPTS UTF8 ON

Questions (if you know ?) :
- are those errors "normal" (or acceptable) ? // are those error "responsible" of our TC slowness ?
- is the internal TC ftp is "natif" or does TC use a XP dll ? (so that I could change it ?)
- is there a way to use another FTP plugin ? (I already got a sftp plugin, but no ftp plugin...)
- wanna make me perform other tests ? // get other logs ?

Thx !

[ghisler(Author)]

The only difference I see is that you use Passive mode in FileZilla - you may try that in Total Commander too. But it shouldn't have any effect on the transfer rate, except when using some kind of content filtering firewall...

[Mjolnir]

I did also use passive mode (in FTP generic Configuration, and in the FTP detail connection) without success (still slow)
I also activated the passive mode for FileZilla, same high rate.

[Hacker]

Mjolnir,
Then perhaps Wireshark could help.

Roman

[Mjolnir]

OK, I'm not a LAN expert, but here something interresting : I've used WireShark to check the FTP frames from my PC to the Solaris system I target :

- with FileZilla : frame length is 1460 bytes all over the time, with something like 0.5% of Retransmission

- win TC : here is the strange behavior : ~50% time, frame length is 1460, and the other ~50% time, Frame length is 1024 or 512 or even 76 bytes ?!? (and I got at least 3x to 10x times more "Retransmission" frames that with FileZilla)

[hlloyge]

Hm, that could explain, as it waits for each packet to get confirmation of arrival from server...

[Mjolnir]

Hm, that could explain, as it waits for each packet to get confirmation of arrival from server...

Yep... but question is : why do I got this pb with TC and not with FileZilla ?!?

[ghisler(Author)]

Is this for uploads or downloads? For uploads, you can play with the UploadBlockSize inc wcx_ftp.ini (see help). For downloads, TC shouldn't control the size as the recipient...