FTP bugs in TC 5.51 through 6.54a

[Garlin007]

I use the Synchronize function to backup data from a web server to my hard drive. I am experiencing a number of problems with the FTP functionality of Total Commander, which I will list here.

1. If the files change on the web server after you establish the FTP connection (say by uploading changes with FrontPage), TC does not recognize the changes, even if you press Ctrl-R to reread the folder. The only way to see the new files is to disconnect and then reconnect.

2. I have tried every setting to maintain the connection with the server, but I still lose the connection after about 10 minutes of idle time It is possible that whatever is causing my connection to be lost is also what is causing the other problems, since ultimately my other problems are related to TC not being able to recover from a port error, and this same error may be what's causing my connection to be lost.

3. I get frequent port errors. These errors occur when the Synchronize is trying to read all the folders to determine which files are different, and it also occurs during the copy process. My website is aproximately 3500 files, 300 folders, and 60MB. The failures are frequent enough that it is practially impossible to synchronize the entire site. If I synchronize just 10% of the site, then about one of three times I will get a failure. If I try to synchronize more than that, the faiures occur so frequently that I can never finish the synchronize. When the failure occurs, TC just hangs. From recollection, on one of the screens it says that there is a Port error. The only way to shut down TC is by shutting down the process with the Task Manager.

4. I have previously reported a bug in the date calculation that offsets dates, so the date on a server file in a different time zone can be compared to the date on a local file. The error occurs when a server file is on the first day of a month that follows a month that has fewer than 31 days, and the time zone offset is negative. The program tries to subtract the offset from the file on the server, but if the offset is large enough for the date to be the last day of the previous month, instead of using the correct number of days in the month, it uses 31. For example, if the offset is -6, and the date on the server is 1:00 am on March 1, 2003, instead of getting 19:00 on Februeary 28, you get 19:00 on February 31. When TC tries to put this date into the file, Windows XP doesn't allow the invalid date, so it sets the date to 1/1/85.

[Hacker]

Do you use Passive mode?

TIA
Roman

[Garlin007]

Hi Hacker,

If I could fix these by changing my FTP settings, that would be great!

Here are all the FTP settings that I could find:

Misc.
Number of times to retry if FTP server busy: 10
Delay between retries: 5
Default transfer mode: Binary

Within a typical FTP connection detail (I have left out the server, user, etc.):
Send Commands: (blank)
Server type: Auto detect
Use firewall proxy server: Unchecked
Use passive mode for transers (like a WWW browser): Unchecked
Sed command to keep connection alive: PWD
Send interval every: 90 s

By the way, I have tried NOOP and SYST for the keep alive command, and none of the options keep the connection alive.

[Hacker]

Use passive mode for transers (like a WWW browser): Unchecked

Try checking this options and let us know what happens.
Also try adding pasvmode=1 to the [Default] section of your wcx_ftp.ini .

HTH
Roman

[Garlin007]

Great news. Using the passive mode I was able to copy 267 files / 7.5 MB in one session without stalling. In a second test, I was able to compare 2000 files without stalling. (Normally I would have divided that compare into 5 passes to insure success.) Two other smaller synchronizations worked properly. Reading the directories also appeared to be working faster. So far so good!

[Garlin007]

Unfortunately, my web host has disallowed passive mode FTP at their end. They claim it was causing their customers problems. Ever since then I was forced to go back to non-passive mode and my previous problems have resumed. I cannot reliably synchronize large directories because I get a PORT error and TC disconnects from the server before the operation completes.

These newest problems were in version 6.53. I also tried to use FTP in version 6.54a, and the problem got worse! At least with version 6.53 when there was an error I got immidiate access to the user interface for TC, so I could close the synchronize window, reconnect, and start the synchronize again. With version 6.54a TC just hangs when there is a PORT command and I have two options. The first is to wait for TC to issue an error, which usually takes 60 seconds or so. When you get a PORT error every couple of minutes, those extra 60 seconds start to add up! However, at least I get the option to reconnect. The second option is to prese esc or cancel. 75% of the time I would get the interface again after a 10 second or so delay, and I could then do as I did in version 6.53, i.e. close the synchronize, reconnect, and restart the synchronize. Often I would have to close the connection and reopen it, thus also having to take the time to navigate back to the folders I was synchronizing. The other 25% of the time after pressing esc TC would just hang and be non-responsive except for a beep when you would click the interface. Then my only option is to close TC using the task manager (even the red X to close the window to force a shutdown or right clicking on the button on the task bar wouldn't let me shut the window).

In summary, TC does not handle PORT errors very well. If TC would just reconnect to the server on its own and continue where it left off then I could use TC to handle synchronization of my site. As it stands now it is a very laborious process since I have to break the site down into chunks of 10 folders at a time, since trying to do more than that will result in failure and having to wait for TC to rescan too many folders after the error. It means I have to synchronize the site manually, instead of just setting TC at the root and letting it do the 60 minutes or so of file transfers on its own.

----------------------------------------------------------------------------------

I left the original post alone, but in retrospect, the 25% of the time I had to end TC using the task manager was simply due to a lack of patience. If you wait for 60 seconds or so, TC eventually reports that it has lost its connection to the server and offers to reconnect. I just wasn't waiting long enough.

[Garlin007]

I did a few tests, and discovered that the port errors are most likely related to using TC through a router. My home network is set up with a cable modem connected to a Linksys wireless router that lets me share the network connection with two computers that are connected to the router using wireless network cards.

In the first test I tried to limit the range of ports and I set FTP to retry only once after 1 second if the server is busy. No improvement. I don't know why with these settings that TC is unresponsive for 30 to 60 seconds when there is a port error. I don't know if there is a setting to shorten the time that TC tries to recover from an error before it indicates that it has lost the connection to the server, but being able to shorten that time would make the port errors easier to live with. I left these retry settings for the remainder of the tests.

In the second test I tried to put my computer in the router's DMZ, with no effect.

In the third test I connected my computer directly to the cable modem, bypassing the wireless network and the router. I was able to transfer almost 2000 files before getting my first and only port error. I used this setup for over an hour, and experienced just one port error in all. Normally I would get a port error after 1 to 30 files, so this was a dramatic improvement.

I suspect that the source of the problem is the NAT aspect of the router. If necessary, I could test this supposition by connecting to the router using 100 baseT, and if the port problems occur again I could deduce that the problem is due to the NAT functionality of the wireless router, and not due to the wireless network aspect.

Given this new information, is there a setting on the wireless router that I could use to avoid the port errors?

[ghisler(Author)]

I'm also connecting to the Internet over a Router (with stateful packet inspection), and I never get any port errors.

What you can try is the following:
1. Redirect a range of incoming ports (at least 1000) from the router to your PC
2. Set TC to use these ports for port connections.
3. Make sure that TC doesn't get blocked by some desktop firewall

[Sheepdog]

2. Set TC to use these ports for port connections.

That could be done by the keys

Code: Select all

[General]
Portfrom=10000
PortTo=11000

in wxc_ftp.ini where the values ar of course only examples.


[Edit]

Or in Configuration->Options->FTP:Port mode:Only Ports from limited range.

[/edit]
sheepdog