TC - Too slow copy over the network !

[MonkY]

Well, as i've upgraded the networks lately from 100mbps to 1000mbps (gigabit network) i was shocked to see how slow Total Commander can copy files compared with MS internal Explorer.

Using Explorer, i get arround around 40-50MB/s on both Upload/Download from one PC to another.
Using TC 6.02 i only get 20MB Download and max 10MB Upload... This is so slooow.

All the computers in the network have Intel Gigabit netcards, CSA enabled, P4@2.4 minimum configuration.
Here are some benchmark results i got, so that i can prove there is something wrong with TC, nothing wrong with the configs...

NetCPS From Computer A to Computer B:
---> CPS 5751808.00 KPS: 5617.00 MPS: 5.49
Avrg CPS 5725856.00 KPS: 5591.66 MPS: 5.46
Peek CPS 5778432.00 KPS: 5643.00 MPS: 5.51
Client disconnected. 104857600 Kb transferred in 18.31 seconds.

NetCPS From Computer B to Computer A:
---> CPS 6070272.00 KPS: 5928.00 MPS: 5.79
Avrg CPS 6089645.00 KPS: 5946.92 MPS: 5.81
Peek CPS 6166528.00 KPS: 6022.00 MPS: 5.88
Done. 104857600 Kb transferred in 17.22 seconds.

NETIO v1.23

server on A
command used:
win32-i386.exe -s 172.16.3.3
client on B
command used:
win32-i386.exe -t 172.16.3.3
results:
TCP connection established.
Packet size 1k bytes: 4016 KByte/s Tx, 4602 KByte/s Rx.
Packet size 2k bytes: 6805 KByte/s Tx, 8714 KByte/s Rx.
Packet size 4k bytes: 12259 KByte/s Tx, 13626 KByte/s Rx.
Packet size 8k bytes: 23881 KByte/s Tx, 20677 KByte/s Rx.
Packet size 16k bytes: 42067 KByte/s Tx, 28132 KByte/s Rx.
Packet size 32k bytes: 63881 KByte/s Tx, 35911 KByte/s Rx.
Done.

server on B
command used:
win32-i386.exe -s 172.16.3.2
client on A
command used:
win32-i386.exe -t 172.16.3.2
results:
TCP connection established.
Packet size 1k bytes: 4910 KByte/s Tx, 3765 KByte/s Rx.
Packet size 2k bytes: 8566 KByte/s Tx, 6761 KByte/s Rx.
Packet size 4k bytes: 14263 KByte/s Tx, 12143 KByte/s Rx.
Packet size 8k bytes: 21171 KByte/s Tx, 23415 KByte/s Rx.
Packet size 16k bytes: 28513 KByte/s Tx, 40648 KByte/s Rx.
Packet size 32k bytes: 35924 KByte/s Tx, 65651 KByte/s Rx.
Done.

So, what can i do ?!? Can this be solved ?

[MonkY]

Well, i've been playing with TC/Configuration/Copy-Delete Options... and setting *Also use big file copy method with 4092kb i am able now to copy through network with 20MB/upload and 20MB/download... What is going on ? This is better... but now the normal copy (using 10.000 RPM WD 74GB with 8MB Cache) is about half the original copy speed... What settings do u recommend ?

[icfu]

Recommended is 10240kb Minimum for Source and Target on same disk. The value below, Source and target on different disks depends on the disks. I have set it to 128 only because my cdrom starts to slow down when i set it too high, recommended is 32k.

Have you tried the setting
SamePartitions=?

With this setting you can specify which partitions are on the same hard disk and therefore for which partitions the 10240k is used.

For example
SamePartitions=CDE\ means that partitions C, D, E and Network is always considered being on the same hard disk.

No warranty that this works, I don't even know if that setting makes sense regarding the network but because your network is fast as hell I suppose it will.

Icfu

[MonkY]

Recommended is 10240kb Minimum for Source and Target on same disk. The value below, Source and target on different disks depends on the disks. I have set it to 128 only because my cdrom starts to slow down when i set it too high, recommended is 32k.

Have you tried the setting
SamePartitions=?

With this setting you can specify which partitions are on the same hard disk and therefore for which partitions the 10240k is used.

For example
SamePartitions=CDE\ means that partitions C, D, E and Network is always considered being on the same hard disk.

No warranty that this works, I don't even know if that setting makes sense regarding the network but because your network is fast as hell I suppose it will.

Icfu

Well, i've setup TC to use "Also use big file copy mode (not using cache)"
and Source+Target on the same disk = 64 K
and Source+Target on different disk = 4032 K

And then i've setup something like this: CPQRS\,DPQRS\,EPQRS\,FPQRS\,GPQRS\,HPQRS\,IPQRS\,JPQRS\,KPQRS\,LPQRS\,MPQRS\,NPQRS\,OPQRS\ as the same physical drive (as you may wonder, PRQRS are mapped drive from network, and the rest is every letter put together with the mapped drives).

Well, using these settings now i have 24-25MB/s for download and 18-20MB/s for upload... but reduced my "not network" copy to about 30MB/s from 50MB/s

Setting it to cache copy mode.. helped me with nothing...

It's a mess in here... and i don't understand why the author can't make an autodetect of what means SAME DRIVE.. or if a drive is mapped from network... and make an advanced copy mode, where setup a different cache for bigger files and a differente one for smaller files that changes in function of what drive is used. I mean, i am talking in here about two computers, but at work i am using it on over 40 computers... and i can't stay and decide what drive is mapped, who's not.. benchmark them all... set a different cache and so on...

Please Ghisler, help me out (once more )

Thank you in advance for your help.

[icfu]

and Source+Target on the same disk = 64 K
and Source+Target on different disk = 4032 K

Vice versa...

Like I said, recommended minimum values:
Source+Target on the same disk = 32 K
Source+Target on different disk = 10240 K

SamePartitions=CDEFGHIJKLMNOPQRS\ is enough btw. or do you have 13 separate hard disks from C to O?

Icfu

[MonkY]

and Source+Target on the same disk = 64 K
and Source+Target on different disk = 4032 K

Vice versa...

Like I said, recommended minimum values:
Source+Target on the same disk = 32 K
Source+Target on different disk = 10240 K

SamePartitions=CDEFGHIJKLMNOPQRS\ is enough btw. or do you have 13 separate hard disks from C to O?

Icfu

OK, i've done as u said... but, using secondary method (not using cache) YOU CAN NOT EVEN SET 32K !!! The minimum is 64K. You must be reffering to the first method.. the one using cache (default method)...
OK, let's see the results with your recomandation (although when i set 32K it's automatically set as 0 value).

Download from network: 28-30MB/s
Upload to network: 9MB/s (lowest i can get)
Copy from one WD 10.000 RPM to another (40MB/s)

I must say i am also checking the transfer rates with perfmon.msc and tc so i am sure the rates are not too underrated.

[icfu]

No problems with setting to 32k, using TC 6.02 here:
http://icfu.i-networx.de/tc_config.jpg

You entered the values in the wrong places, that's why I said vice versa...

Edit:
Sorry, I had copy&pasted your lines and have forgotten to correct them...

The default recommended minimum values like shown in the screenshot are:
Source+Target on the same disk = 10240 K
Source+Target on different disk = 32 K

I hope it's working for you now...

Icfu

[MonkY]

No problems with setting to 32k, using TC 6.02 here:
http://icfu.i-networx.de/tc_config.jpg

You entered the values in the wrong places, that's why I said vice versa...

Edit:
Sorry, I had copy&pasted your lines and have forgotten to correct them...

The default recommended minimum values like shown in the screenshot are:
Source+Target on the same disk = 10240 K
Source+Target on different disk = 32 K

I hope it's working for you now...

Icfu

OK. Thank you for willing to help. I am setting up the way you told me.. and let's see the results with the settings you provided me:

Total Commander 6.02
Download from network: 23MB/s
Upload to network: 18MB/s
Copy from one WD 10.000 RPM to another: 28MB/s

Microsoft Explorer
Download from network: 33MB/s
Upload to network: 33MB/s
Copy from one WD 10.000 RPM to another: 46MB/s

Well, i wished that TC should win over this... but i see about 50% the speed of old Explorer... i know the functions TC has are incomparable with the Explorer... but 90% of the time i use TC for File Exploring, copy, move, manage files (the new tabs are lovely)... and in this category, TC FAILS BIG TIME.. I hope the new version will improve the file transfer (move/copy/delete).. specially now when gigabit network is all over the place...

[icfu]

Increase the value 10240, this is just the recommended minimum.

If that doesn't help:
Virus scanners can slow down TC copy method, that's what the "use copy&paste via explorer" setting is for".

So if you are running an AV monitor disable it or exclude TC in AV soft, maybe that helps to solve the prob.

Running out of ideas, sorry, so please, ghisler or some other magician, it's your turn.

Icfu

[MonkY]

Increase the value 10240, this is just the recommended minimum.

If that doesn't help:
Virus scanners can slow down TC copy method, that's what the "use copy&paste via explorer" setting is for".

So if you are running an AV monitor disable it or exclude TC in AV soft, maybe that helps to solve the prob.

Running out of ideas, sorry, so please, ghisler or some other magician, it's your turn.

Icfu

Thank you again... increasing the value from 10240 to 20480 or even 40960 only resulted in slower results... And yes, i disabled firewall and antivirus while performing these tests...
Now, maybe Ghisler can help... i can't imagine how can be so fast MS Explorer next to TC...

[Hacker]

What happens when you use the "Use copy+paste via Explorer" option?

TIA
Roman

[MonkY]

What happens when you use the "Use copy+paste via Explorer" option?

TIA
Roman

This give me about the same results as i would use TC. In fact, i think IT IS Explorer that's copying the files. I think TC just transfer the file list to be copied/moved... Damn, if i could get that transfer but inside TC... that would be something

[Hacker]

Hmm, and how do SlowCopy or FileCopy (or what those names were) perform? You can find them on http://clubtotal.free.fr/modules.php?name=Downloads&d_op=viewdownload&cid=2 .

TIA
Roman

[SanskritFritz]

What happens when you use the "Use copy+paste via Explorer" option?
This give me about the same results as i would use TC. In fact, i think IT IS Explorer that's copying the files

I see a contradiction here. In fact, you're right, windows does the copy, and this way you should get the same result as in explorer.

[MonkY]

Hmm, and how do SlowCopy or FileCopy (or what those names were) perform? You can find them on http://clubtotal.free.fr/modules.php?name=Downloads&d_op=viewdownload&cid=2 .

TIA
Roman

I've downloaded the utilitiest.. testing:

Slowcopy with 3MB cache (fastest for me)
Download from network: 20MB/s
Upload to network: 18MB/s
Copy from one WD 10.000 RPM to another: 25MB/s
Plus: IT CRASHED A LOT OF TIMES. VERY UNRELIABLE (at least with TC v6.02)
All the results i've read with perfmon.msc, as the internal transfer of slowcopy was showing very strange results (like 70MB/s although perfmon was showing me only 25MB/s.. and then i used a clock and measured for big files.. and it was really 25MB/s not 70MB/s.. plus it was fluctuating a lot like from 70MB/s to 5MB/s.. even the graph bar would somtime stop... sto Slowcopy didn't helped)

Filecopy
Download from network: 3-4MB/s
Upload to network: 3-4MB/s
Copy from one WD 10.000 RPM to another: 3-4MB/s
Plus: Filecopy created temporary files if the file copy wasn't completed... undeletable ones (at least not until filecopy killed from Task Manager... )

So untill now, Explorer is the best ! Damn... this is bad.. really bad.