Compare by content shows false positives with ascii files

MMx · Post by *MMx » 2013-05-06, 13:20 UTC

I have two files, called inner and outer (inside hxxp://ksp.sk/~mmx/pack.zip). When I compare them by content in text mode, there's a lot of differences (hxxp://people.ksp.sk/~mmx/differences.jpg), while those lines appear to be the same. In binary mode the only difference is on lines 160 and 161 (diff inner outer --binary confirms this).

I'm using Total Commander 8.01 x64 on Windows 7 SP1.

EDIT: Replace hxxp in URLs with the real thing (apparently I'm not allowed to post links to external sites).

Dalai · Post by *Dalai » 2013-05-06, 13:58 UTC

I can confirm this. WinMerge shows only one difference with the files you provided. It seems that TC has a line length limit of some kind because the horizontal scrollbars show a weird behaviour when you pull them to the right.

Regards
Dalai

Post by *ghisler(Author) » 2013-05-06, 14:12 UTC

The compare tool has a line length limit in text mode - 4kByte. If the line is longer, it will not be compared. This was necessary because lines are compared "on the fly" while drawn to the screen.

MMx · Post by *MMx » 2013-05-06, 15:28 UTC

In retrospect it sounds kind of obvious. Although knowing that in advance by way of some warning message would have saved me some headache

AntonyD · Post by *AntonyD » 2013-05-28, 05:14 UTC

OR - and that should be much more correct decision - Ghisler should to think about an introducing in TС - third party lib(or standalone exe) for comparing files - which will not have such stupid limitations. Because right now we should always to have on HDD/SSD at least one specialized comparing tool - exactly for such files. But obviously that such approach is not good and non suitable for everyday work.
Come on - we live at 21th centuries - but we are creating a limitations for files comparison...

Dalai · Post by *Dalai » 2013-05-28, 09:25 UTC

AntonDudarenko wrote:OR - and that should be much more correct decision - Ghisler should to think about an introducing in TС - third party lib(or standalone exe) for comparing files - which will not have such stupid limitations.

You're too late with that suggestion:

TC help file, section 4.b) wrote:Comparetool= Set external compare tool for "Compare by contents", e.g. WinMerge

MfG Dalai

AntonyD · Post by *AntonyD » 2013-05-30, 08:08 UTC

this is not a good suggestion - coz as I said - to have special tool for comparing - is a nice but not a perfect idea. IF inside the TC it was decided by author to implement the file comparison mechanism - then it should be enhanced and improved the same way as is done for the TC now.
As an example of the field for fixing we have now current limitation on 4k of text length - which is a very annoying "feature" which should be removed/fixed ASAP.
And IF author do not have enough time for improving this subtool - when definitely - he should to think about an introducing on a permanent basis at installation setup file along with TC's mains files and some 3tdparty comparison tool - which will be much more mature and useful.

MarcinW · Post by *MarcinW » 2013-05-30, 08:18 UTC

ghisler(Author) wrote:This was necessary because lines are compared "on the fly" while drawn to the screen.

As I suppose, the speed of drawing is a reason. So the maximum line length could be configurable - on modern machines comparing lines longer than 4k may still not cause flickering.

siealex · Post by *siealex » 2013-06-20, 22:22 UTC

As I suppose, the speed of drawing is a reason.

Why must we draw the text to compare it??? Isn't it simpler to compare it in RAM and draw only after gathering all differences?
PS. Frequent redrawing on slow systems can physically damage the video card.

MarcinW · Post by *MarcinW » 2013-06-25, 11:13 UTC

This was simplification. Strings are of course compared in RAM, and then the results are drawn. But longer lines cause longer comparison time, so there is a longer time between drawing consecutive lines on the screen.

Horst.Epp · Post by *Horst.Epp » 2013-06-25, 11:30 UTC

siealex wrote:
As I suppose, the speed of drawing is a reason.
...
PS. Frequent redrawing on slow systems can physically damage the video card.

What a joke, normal configured video cards will never be destroyed whatever you are drawing.

siealex · Post by *siealex » 2013-06-28, 12:20 UTC

This was simplification. Strings are of course compared in RAM, and then the results are drawn. But longer lines cause longer comparison time, so there is a longer time between drawing consecutive lines on the screen.

It will be simpler (and will consume less RAM) to write the comparison results to a temporary file on the HDD.

MarcinW · Post by *MarcinW » 2013-06-28, 22:46 UTC

siealex wrote: ... write the comparison results to a temporary file on the HDD.

HDD is very slow when compared to RAM. Lack of RAM is not a problem - operating system moves some data from RAM to swap file in case of excessive RAM usage. So you don't have to worry about RAM usage and about writing data to HDD - operating system does this for you.

Post by *ghisler(Author) » 2013-07-01, 09:00 UTC

TC makes the comparison within a line only when it is displayed. This is much faster than during the initial comparison, because only a few lines (as many as fit on the screen) have to be compared, not the entire file.

The initial comparison of two files just looks for the line breaks, and then calculates a checksum (hash value) for each line. Then in a second step, only these hash values are compared to find identical lines.