[REQ] Tool to determine best copy method settings

Post by *Hacker » 2006-03-20, 16:25 UTC

I wonder - wouldn't it be great if Christian made a small utility to test various values for various copy methods so we wouldn't have to guess the best values for our drives? The tool would simply try to copy eg 50 MB, each time with different settings. These would be customizable (so we can test our own values) but there would also be an automatic mode, where eg the 50 most promising value combinations would be tested and after 10-20 minutes the best would be shown.

What do you think?

TIA
Roman

Post by *ghisler(Author) » 2006-03-20, 16:37 UTC

If only it would be so simple! The Windows disk cache and the cache of the harddisk itself have big influence too! Currently the default configuration is maximum safety, but not maximum speed.

StickyNomad · Post by *StickyNomad » 2006-03-20, 16:37 UTC

Indeed such a tool would be very useful, I support that! Manual testing is neither comfortable nor accurate and I suppose the effort to create such an autotest-tool should not be too much.

[EDIT]
2Ghisler(Author)

Hm, but if the user would run the test-tool on his machine, he could at least find an optimal setting for his actual windows-conifguration, or am I wrong?

Of course security is an important aspect, but if the deafault settings are kept as set actually, the user could be pointed to possible issues when tuning the settings with the tool.
[/EDIT]

Post by *Hacker » 2006-03-20, 16:42 UTC

Christian,

The Windows disk cache and the cache of the harddisk itself have big influence too!

That's exactly the reason I suggested the test tool.
The cache could be circumvented by first writing a few sets of random data to the source disk and then copying them.

Roman

Lefteous · Post by *Lefteous » 2006-03-20, 16:45 UTC

2ghisler(Author)

Currently the default configuration is maximum safety, but not maximum speed.

What does "safety" means here? Will copied data be corrupted when I use other settings?

Post by *ghisler(Author) » 2006-03-20, 16:59 UTC

What does "safety" means here? Will copied data be corrupted when I use other settings?

If you remember, we had a problem with a specific SCSI controller from Adaptec where the big file copy mode was causing corrupted files. The problem was caused by unclearly formulated documentation of the Windows API. The function was subsequently changed by us to make it work also with this specific Adaptec controller. This problem has prevented me so far from making this copy method the default. There have been no other problems since then, though.

Sheepdog · Post by *Sheepdog » 2006-03-20, 22:55 UTC

A tool for testing would really be great.

Such a tool should allow the following:

Select a file/folder to be copied.
Select one or more Drives/paths to copy to.
Select a range of block sizes to test.

Do the test automatically and log the results.
Optionally delete testfile after operation.

Thus I could find the best values for my Machine easily by running the tool overnight.

Now I would have to copy a file over and over again, probably to many different Harddisks and all the time watch this copying with my stopwatch.
And never forget to adjust the blocksize. Today I do not even know which blocksizes are making sense (10240; 10260;10280? or rather 10240; 11262 etc.)

If only it would be so simple! The Windows disk cache and the cache of the harddisk itself have big influence too!

The average user is not very interested in what influences the copy speed but wants to find out the best setting for copying on his machine, I think.

sheepdog

WatchUer · Post by *WatchUer » 2006-03-21, 01:11 UTC

Support!

majkinetor ! · Post by *majkinetor ! » 2006-03-21, 12:25 UTC

If only it would be so simple!

raeubi · Post by *raeubi » 2006-03-21, 20:41 UTC

A tool for testing would really be great

Support ++

A good solution is a button in the TC copy/delete configuration dialog to start this test tool.

Postkutscher · Post by *Postkutscher » 2006-04-10, 04:52 UTC

As far as I understand,our dear author is too busy to do this.I mean anybody else can do this. This was a great idea.
It can be not a part of TC. Special utility would be a good solution too.

Post by *Hacker » 2006-04-10, 07:44 UTC

Postkutscher,

anybody else can do this

I don't think so. Only Christian knows how exactly TC uses the values entered in the Copy/Delete settings.

Roman

levicki · Post by *levicki » 2006-10-17, 14:39 UTC

Crucial problem here is this -- once you create a source file for copying it is cached and it stays cached until you reboot. That would influence the results of every test but the first one. So you would have to reboot before each test pass.

Anyway, why bother when everybody knows that the fastest way to copy files in Windows is to use asynchronous file I/O?

Post by *Hacker » 2006-10-17, 16:52 UTC

once you create a source file for copying it is cached and it stays cached until you reboot.

This has been discussed above. There is no reason to reuse the same file.

Roman

Sticky · Post by *Sticky » 2006-10-17, 20:22 UTC

Indeed, it's not necessary to reuse the file. Using slightly different files would have some effect on the results, perhaps, but that's "fine" tuning. What we're talking about is a test that gives you "coarse" tuning, because the difference between the different values can be quite significant, I think.

It's currently too time consuming, annoying, and unreliable to test these things manually. The effort might be worth it if you'll be using the same drives indefinitely (for several months/years), but in my work we're often using a particular drive for just one or two days, a week tops.

A tool like this would allow me to find the best copy parameters for that drive on the first day in just a few minutes (e.g. a coffee break), and thus speed up operations for the rest of the week.

Support +++