Synch Dirs/Compare by Content: Bug in "compare by conte

[handil]

Hello ghisler,

thank you for your suggestion, i have made a little test drive with the above quoted image files. Here is the result http://www.handil.de/pub/text/TC_synchronize_dirs-testdrive.LOG. For the capturing i have disabled success messages, so the log can be shorter.

As you can see, at the end of the first file, there comes an failure message from tc. In the picture there ist the graphical result Image: http://www.handil.de/pub/Bilder/TC_synchronize_dirs-testdrive.jpg

[ghisler(Author)]

That's a very bad sign, it's actually a read error at the driver level. Either the disk is bad, or there are connection problems (cable too long or broken or bent to a very small radius).

[handil]

Hello ghisler, thank you for your annotation, but why does no other program (comp.exe delivered with windows os, NexusFile directory synchonisation) produce problems.

Tests with tools for check of integrity of harddisk, low level check of harddisks and controller communication, does not report any problem.

Only one program, totalcommander, produce problems while compare files.

- first try: Check 7 files, the same 7 files on two different harddisks, with NexusFile, in result the files are identical
- second try: Check the same 7 files with tc, in result the files are unequal
- third try: Check the same 7 files with NexusFile again, all files are identical, and with comp.exe at the same time (with a short time difference), also all files are identical
- fourth try: check again the same 7 files with tc, the files are unequal

I have made all tests without an reboot or restart of any service or program

[ghisler(Author)]

Sorry, I have no idea why it happens in TC only. Maybe you only copy that many files in TC, so the drive becomes too hot...

[handil]

This isn't an good message.

I'm monitoring my harddisks with crystaldiskinfo, and the temperature is lower than 30 degree celsius (also while running the tests), so the temperature can't be a reason.

Who have an suggestion of the cause.

[Mish]

First of all, hello to everybody in this topic and very special thanks to Christian for the excellent software!

I want to confirm this strange behavior of TC on multiple machines with x32 Windows XP SP3 or Windows 2003 server SP2 with 4Gb RAM installed.

I am copying data (~900Gb with huge number of subfolders and big and small files within them) from one HDD (non-system, data only) to another (with "copy date/time of directories" option).

Then I'm comparing directories with "Synchronize dirs" - "Compare" with "by content", "subdirs", "ignore date" and TC 7.56a (some older versions as well) reports me some files being different.

However, when I start another copy of TC and compare those problematic files one by one - TC reports me that they are identical. Sometimes multiple big (500Mb-20Gb) files in one folder are reported being different, but comparison of just those folders on 2 drives either reports no difference or reports another set of files being different!!!

I've faced this problem multiple times within many years when copying data between HDDs and finally decided to register on the forum and look for the solution here. To make short line-up of my experience with this problem:

- this happens mostly on really huge comparisons with multiple big files
- generally results are not repeatable, next comparison attempt will likely report different set of files
- file-by-file comparison will likely report that "problematic" files are identical (however, sometimes "not enough memory - files are different" was observed as well, then system shutdown and restart heals the "file-by-file" comparison, but not the "synchronize dirs" comparison)
- hard drives are ok and there are actually no problems with copied files within next years.

The problem is really annoying since it makes data copy-and-verification operation very time consuming and nervous due to false negative comparison results.

Maybe this info will help to find the solution...

[handil]

Hello mish,

happy new year for you and all readers.

It's reassuring for me to see that there are others, that have the same problems.

My solution was:

1. Set "/MAXMEM=4095" switch in the boot.ini file for windows server 2003 (windows home server)
2. Change the memory to the exact type, that are recommended by the board manufacturer (including the timing rates)
3. Install all windows updates, included the optional updates and hardware driver updates

Since this time there are no more problems while synchronize folders with the option compare by content, for now.

handil

[ghisler(Author)]

Sounds like a damaged RAM module to me. Do you also get CRC errors when trying to create large ZIP archives?

[handil]

Yes, i'm think so, but i'm not absolutely sure. The memory i have used, was definitly not certified by the board manufacturer. It is too long ago, a couple of weeks, sorry

handil

[Mish]

Hi All,

Christian, after hard tests (~8-10hours per pass) - many thanks to handil's prompt reply! - I can confirm your suggestion:

(1) checked Microsoft's site with their suggestion to use /MAXMEM in case of problems with RAM (kb 108393): "This switch may be valuable in troubleshooting parity errors, mismatched SIMM speeds, or other memory related problems."

(2) tried /MAXMEM - depending on exact parameter (4000, 3500, 3000) number of different files was visibly reducing together with parameter decrease. NO extra patches or driver updates where installed. RAM timings where at their default values for sure.

(3) Replaced 2x2Gb RAM modules with another set 2x1Gb and - voila! - no more errors! Most ironically - the only hardware that was shifted from one system to another, from XP to Srv2003 - was this faulty 2x2Gb RAM set...

I'll also try to get the new 2x2Gb RAM set and make some more tests with it...

Right now I would assume the problem is solved.

Thanks again to handil for suggestion - as it helped to solve the puzzle!

Wish everybody good luck in the New Year!

[Mish]

Hi All,

As promised, some more tests made, with some fresh (but bad ) news...

Unexpectedly strange situation on the fresh new system with fresh new 2x2Gb RAM modules (no /MAXMEM, default timings, fresh Srv2003 with all updates on the separate system drive, 2 data drives from the previous tests to compare ~500000 files/1Tb):

- 2x2Gb RAM installed - errors

then the idea was to find the faulty RAM module, however:

- 1st 2Gb RAM module only - OK
- 2nd 2Gb RAM module only - OK

This means that each 2Gb module separately works fine, but just having 4Gb pushes TC back to the problem with comparison by content...

Any ideas?

[ghisler(Author)]

Are these indentical modules from the same manufacturer? Mixing modules can cause such problems.

[Mish]

Hmmm... below is the lineup of my efforts:

1st set (2x2GB DDR2 DIMMs) - same modules from Kingston (not paired)
- 2x2GB tested on GigaByte motherboard, errors (initially reported)
- 2x2GB tested on Zotac MB, errors (initially reported)
2nd set (2x1Gb DDR2 DIMMs) - identical (paired) set from Corsair
- 2x1Gb tested on Zotac MB, OK
3rd set (2x2Gb DDR3 SODIMMs) - same modules from Kingston (not paired)
- 2x2Gb tested on SuperMicro MB, errors
- 1st 1x2Gb tested on SuperMicro MB, OK
- 2nd 1x2Gb tested on SuperMicro MB, OK
4th set (2x2Gb DDR3 SODIMMs) - same modules from Samsung (looks like not paired - do not remember)
- 2x2Gb tested on SuperMicro MB, errors

I've also made tests on different controllers (built-in SATA AHCI and ATTO PCIe 16-port Host SAS ), but there was no difference with them

In a few days I'll replace the 3rd set with identical (paired) one - as seller kindly offers easy moneyback - and will make some more tests...

[ghisler(Author)]

You should check whether there is a BIOS update for your board. Also try to check the timings of the modules when just one is connected, they may be different.

[Mish]

5th set (2x2Gb DDR3 SODIMMs) - identical (paired) set from Corsair
- 2x2Gb tested on SuperMicro MB, errors

Latest BIOS, same timings...