Compare file contents: mojibake

The behaviour described in the bug report is either by design, or would be far too complex/time-consuming to be changed

Moderators: white, Hacker, petermad, Stefan2

ChristianGruen
Junior Member
Junior Member
Posts: 28
Joined: 2008-09-14, 15:45 UTC

Compare file contents: mojibake

Post by *ChristianGruen »

When comparing two larger text files (around 900 kb; I’m sorry cannot share here), something goes wrong when rendering the diff:

Image: https://files.basex.org/.tc/compare.png

Observations:
• The line in question has around 26,000 characters.
• The problem eventually disappears when I reduce the file size.
• It doesn’t matter which font I choose.
JOUBE
Power Member
Power Member
Posts: 1478
Joined: 2004-07-08, 08:58 UTC

Re: Compare file contents: mojibake

Post by *JOUBE »

Should be fixed in Tc10.50 (see history.txt):

Code: Select all

26.06.22 Fixed: Compare by content: Very long lines over 10000 characters could result in overlapping text (32/64)
Which Tc version do you use?
ChristianGruen
Junior Member
Junior Member
Posts: 28
Joined: 2008-09-14, 15:45 UTC

Re: Compare file contents: mojibake

Post by *ChristianGruen »

Which Tc version do you use?
11.01
JOUBE
Power Member
Power Member
Posts: 1478
Joined: 2004-07-08, 08:58 UTC

Re: Compare file contents: mojibake

Post by *JOUBE »

Please test it: Up to what line length does the problem not occur in your case? That would help. Thanks.
ChristianGruen
Junior Member
Junior Member
Posts: 28
Joined: 2008-09-14, 15:45 UTC

Re: Compare file contents: mojibake

Post by *ChristianGruen »

Please test it: Up to what line length does the problem not occur in your case? That would help. Thanks.
Thank you. It seems to be a combination of the length of a line and the total file size that matters. I’ve uploaded encoded and truncated versions of the two input files:

https://files.basex.org/.tc/1.txt
https://files.basex.org/.tc/2.txt

You should encounter the phenomenon after scrolling appr. 550 characters to the right.
User avatar
white
Power Member
Power Member
Posts: 4623
Joined: 2003-11-19, 08:16 UTC
Location: Netherlands

Re: Compare file contents: mojibake

Post by *white »

ChristianGruen wrote: 2023-09-19, 06:27 UTC I’ve uploaded encoded and truncated versions of the two input files:

https://files.basex.org/.tc/1.txt
https://files.basex.org/.tc/2.txt

You should encounter the phenomenon after scrolling appr. 550 characters to the right.
I could not reproduce using these files. Can you provide a screenshot when comparing these 2 files?
Does it occur with both 32-bit and 64-bit TC?
User avatar
petermad
Power Member
Power Member
Posts: 14809
Joined: 2003-02-05, 20:24 UTC
Location: Denmark
Contact:

Re: Compare file contents: mojibake

Post by *petermad »

ChristianGruen wrote: 2023-09-19, 06:27 UTC
https://files.basex.org/.tc/1.txt
https://files.basex.org/.tc/2.txt

You should encounter the phenomenon after scrolling appr. 550 characters to the right.
I cannot reproduce it with those two files either. What is your font setting for the Compare tool?

I also tried with a clean .ini file. Is it at the first difference in the line that it can be seen? Could you maybe give a sttring to search for, to find the spot?
License #524 (1994)
Danish Total Commander Translator
TC 11.03 32+64bit on Win XP 32bit & Win 7, 8.1 & 10 (22H2) 64bit, 'Everything' 1.5.0.1371a
TC 3.50 on Android 6 & 13
Try: TC Extended Menus | TC Languagebar | TC Dark Help | PHSM-Calendar
ChristianGruen
Junior Member
Junior Member
Posts: 28
Joined: 2008-09-14, 15:45 UTC

Re: Compare file contents: mojibake

Post by *ChristianGruen »

Thanks again. Here’s an updated screenshot:

Image: https://files.basex.org/.tc/compare2.png

My font is Consolas (it happened with other fonts as well). Here’s a (admittedly cryptic) string you could search for:

Code: Select all

,c~:2s;..1.skn~ ,;-,s
If it works for some of you, it seems to be a configuration problem. Maybe I should reset my wincmd.ini file and try different settings (?).
JOUBE
Power Member
Power Member
Posts: 1478
Joined: 2004-07-08, 08:58 UTC

Re: Compare file contents: mojibake

Post by *JOUBE »

ChristianGruen wrote: 2023-09-19, 10:25 UTCIf it works for some of you, it seems to be a configuration problem. Maybe I should reset my wincmd.ini file and try different settings (?).
Thank you for additonal informations. There is no error here either. Shure you are using 11.01 in this case (Menu -> Help -> About Total Commander)? No need to reset wincmd.ini at the moment. Start Tc with a fresh ini: "TOTALCMD.EXE" /i=C:\YourPath\fresh.ini
User avatar
petermad
Power Member
Power Member
Posts: 14809
Joined: 2003-02-05, 20:24 UTC
Location: Denmark
Contact:

Re: Compare file contents: mojibake

Post by *petermad »

My font is Consolas
I tried with Consolas in size 8, 9 and 10, but can still not reproduce it. I have tested with TC 11.01 32bit and 64bit on both Windows 7 and 10.
License #524 (1994)
Danish Total Commander Translator
TC 11.03 32+64bit on Win XP 32bit & Win 7, 8.1 & 10 (22H2) 64bit, 'Everything' 1.5.0.1371a
TC 3.50 on Android 6 & 13
Try: TC Extended Menus | TC Languagebar | TC Dark Help | PHSM-Calendar
User avatar
white
Power Member
Power Member
Posts: 4623
Joined: 2003-11-19, 08:16 UTC
Location: Netherlands

Re: Compare file contents: mojibake

Post by *white »

But, I found something goes wrong.
  1. Press F6 (Edit mode), End (go to end of the line)
    The cursor doesn't seem to be at the end of the line.
  2. Press Shift+Home (mark from cursor to beginning of line)
  3. Press Ctr+C
  4. Paste in an editor
    Now the whole line is copied. It seems the the cursor position in step 1 was right, but the displaying of the line was wrong.
JOUBE
Power Member
Power Member
Posts: 1478
Joined: 2004-07-08, 08:58 UTC

Re: Compare file contents: mojibake

Post by *JOUBE »

white wrote: 2023-09-19, 11:39 UTC But, I found something goes wrong.
It is simple: It is not an editor but a compare tool. What goes wrong: The expectation at the tool is wrong.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48088
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Compare file contents: mojibake

Post by *ghisler(Author) »

It could be a display driver problem. In general, the tool isn't suited for lines that long, sorry.
Author of Total Commander
https://www.ghisler.com
ChristianGruen
Junior Member
Junior Member
Posts: 28
Joined: 2008-09-14, 15:45 UTC

Re: Compare file contents: mojibake

Post by *ChristianGruen »

ghisler(Author) wrote: 2023-09-20, 10:53 UTC It could be a display driver problem. In general, the tool isn't suited for lines that long, sorry.
Thank you. It could be my high screen resolution that causes the problem. I could make it work by removing the "Compfontsize" option from the corresponding section…

Code: Select all

[3840x2160 (10x20)]
Compfontsize=13
…and I’ve learned that it works when I use a smaller font size.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48088
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Compare file contents: mojibake

Post by *ghisler(Author) »

Indeed the problem occurs when the text reaches a certain size in pixels.
Author of Total Commander
https://www.ghisler.com
Post Reply