[FULLY FIXED!] Bugs with rendering HEX view of a file

The behaviour described in the bug report is either by design, or would be far too complex/time-consuming to be changed

Moderators: Stefan2, Hacker, petermad

Post Reply
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 53139
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Bugs with rendering HEX view of a file

Post by *ghisler(Author) »

For now I'm trying to get the character display to work before making any major changes. Also I think that readability of the text is more important than alignment.
Author of Total Commander
https://www.ghisler.com
User avatar
AntonyD
Power Member
Power Member
Posts: 2182
Joined: 2006-11-04, 15:30 UTC
Location: Russian Federation

Re: Bugs with rendering HEX view of a file

Post by *AntonyD »

It seems to me that you deal with HEX output too little. Of course, “direct,” so to speak, reading of values from the screen is important—but it’s not even comparable when you literally need to compare byte by byte and find a difference or indicate the exact place where some changes need to be made, for which it was enough to view this file specifically in your viewer rather than immediately in a professional HEx-editor. And here what is important and fundamental is precisely the literal binding of characters to their bytes/pos. Because text as readable text—we can get it just by copying it to the clipboard. And then you can read it calmly. But the output regarding the position of characters in bytes—it’s important to see it exactly in relation to the bytes’ positions))) That’s why the viewing is called HEX and not text.

But of course major changes could be postponed a little bit.
#146217 personal license
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 53139
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Bugs with rendering HEX view of a file

Post by *ghisler(Author) »

Indeed this isn't the purpose of the hex view here. Btw, you can jut select text to compare the hex and text part.
Author of Total Commander
https://www.ghisler.com
User avatar
AntonyD
Power Member
Power Member
Posts: 2182
Joined: 2006-11-04, 15:30 UTC
Location: Russian Federation

Re: Bugs with rendering HEX view of a file

Post by *AntonyD »

Can HEX viewing really have very different or even too simple reasons for being used?
A person might suddenly just want to know — how beautiful the symbols of different countries look in codes? Or how, what?

No, viewing in codes is always some kind of far-reaching goal, based fundamentally on the literalness of displaying codes and readable characters.
Now, if we are engaged in deep file editing, particularly resources – then yes, there’s no need to see the symbols hidden behind the bytes themselves –
because we are editing the essence of the binary stream, the hidden entities within it, and sometimes in such editors the viewing is limited to ANSI/ASCII(OEM) variants. And for these purposes, a different editor is needed. Viewing there is only a partial goal in it.
And in order not to run that editor in vain, we can/want to use your viewer – but for that, it should work for good)))
Not just be able to show a couple of pretty symbols. :roll:
#146217 personal license
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 53139
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Bugs with rendering HEX view of a file

Post by *ghisler(Author) »

well, for me it's more important o to be able to read the text than to see a relation between code and text, but everyone has different needs.
Author of Total Commander
https://www.ghisler.com
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 53139
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Bugs with rendering HEX view of a file

Post by *ghisler(Author) »

Please check whether the display error with the "ж" character has been fixed in beta 4.
Author of Total Commander
https://www.ghisler.com
User avatar
AntonyD
Power Member
Power Member
Posts: 2182
Joined: 2006-11-04, 15:30 UTC
Location: Russian Federation

Re: Bugs with rendering HEX view of a file

Post by *AntonyD »

Yep, this "ж"-problem has gone))
#146217 personal license
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 53139
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Bugs with rendering HEX view of a file

Post by *ghisler(Author) »

That's great to hear! Now about the character alignment, I'm currently not converting to UTF-16 when drawing the text with Uniscribe disabled, so it could be difficult to show the text the way you do.
Author of Total Commander
https://www.ghisler.com
User avatar
AntonyD
Power Member
Power Member
Posts: 2182
Joined: 2006-11-04, 15:30 UTC
Location: Russian Federation

Re: Bugs with rendering HEX view of a file

Post by *AntonyD »

But we realized for sure that you can not disable the Uniscribe-option in any case - even in my simple example,
when some very special eastern fonts, the ligatures are not used become completely impossible to read anything.
Therefore, it is not even worth solving this problem together with disabling the option.

After all, the essence of the proposed NEW display mode is not to COMPLETELY REPLACE something existing, but to have the possibility to be enabled at the user's discretion, if they need it. This means that we can allow ourselves in the code to combine ONLY the working methods of data rendering with the new positions of symbols.
#146217 personal license
User avatar
AntonyD
Power Member
Power Member
Posts: 2182
Joined: 2006-11-04, 15:30 UTC
Location: Russian Federation

Re: Bugs with rendering HEX view of a file

Post by *AntonyD »

NB!!!

If I understand correctly, something broke in mode 2 - Binary View. Logically, it should be exactly what we see in the HEX view on the right side - that is, text. And yes, in this mode 2, the text is present, but it does not completely match what is visible in the HEX view. In this mode 2, the problem 'ж', as we called it here, manifests in exactly the same way. It’s just that in this mode it has DIFFERENT boundaries - depending on what length of binary text I set as acceptable for myself in the settings.
Image: https://i.ibb.co/7THL4hd/2026-03-19-151956.png
Here is our quite adequate representation of the file in HEX mode. We solved the problem with "ж" here.

And now let's look at the same file but in mode 2. And I highlighted those places on the new boundary.
Image: https://i.ibb.co/B2NgmZ42/2026-03-19-152017.png
And yes - boundary looks floating, imprecise, because we are displaying characters simply consecutively - not according to
a template of cells of a kind of virtual table, strictly separating the positions of each character. And here again
the same problem with "ж" appears, and even more than one...

P.S.
And what, of course, is most surprising is that the problem is not characteristic of all the readable RU words that have to be broken when outputting onto different lines — which would be quite logical if this problem could even be called logical at all.
#146217 personal license
User avatar
AntonyD
Power Member
Power Member
Posts: 2182
Joined: 2006-11-04, 15:30 UTC
Location: Russian Federation

Re: Bugs with rendering HEX view of a file

Post by *AntonyD »

Image: https://i.ibb.co/wFNTfqZf/2026-03-19-153421.png
Wow! I checked the current release 11.56 - and the problem is in exactly the same places! That is, the problem existed and still exists!
#146217 personal license
User avatar
AntonyD
Power Member
Power Member
Posts: 2182
Joined: 2006-11-04, 15:30 UTC
Location: Russian Federation

Re: Bugs with rendering HEX view of a file

Post by *AntonyD »

Also, it looks like the text copying is broken. When viewing in HEX mode. However, it is still difficult to understand - in which case exactly....
Kind of like with long dedicated sets of UTF-8 originally encoded UTF-8 characters within a binary file which of course
are in a file mixed in with other binary sequences...
#146217 personal license
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 53139
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Bugs with rendering HEX view of a file

Post by *ghisler(Author) »

something broke in mode 2 - Binary View
Please use Options - 7 UTF-8 in binary mode. ANSI mode with codepage 65001 is very problematic because it doesn't act like a typical double byte character codepage. Currently there are no plans to support codepage 65001 in binary mode, sorry.
Also, it looks like the text copying is broken. When viewing in HEX mode.
No, it's not, it will copy exactly the bytes (NOT charavcters) selected. When selecting from left to right, Lister will select entire characters, but when selecting/unselecting from right to left, it will not at line ends, so you may end with half characters.
Author of Total Commander
https://www.ghisler.com
User avatar
AntonyD
Power Member
Power Member
Posts: 2182
Joined: 2006-11-04, 15:30 UTC
Location: Russian Federation

Re: Bugs with rendering HEX view of a file

Post by *AntonyD »

Please use Options - 7 UTF-8 in binary mode. ANSI mode with codepage 65001 is very problematic because it doesn't act like a typical double byte character codepage. Currently there are no plans to support codepage 65001 in binary mode, sorry.
Did you write separate code for each rendering mode? Right now, the correct text is displayed in HEX mode. Already!
IT SHOULD also be displayed in binary mode! That is, you already have everything! Why is there something that needs to be finished?
It's just that in HEX mode, the text is displayed in a very narrow column – roughly on the right side of the screen.
In binary mode, you just need to DISPLAY THE SAME text that has already been obtained, just starting from the left side of the screen
and taking into account a different 'output column' width. That seems to be all.
The reverse is also true – if you have prepared the text for binary viewing, IT SHOULD be displayed in HEX viewing mode as well,
just considering a different 'output column' width.

Actually, why I even decided to switch to this mode and check the presence of some words I needed in the file?
And precisely and exactly because I FIRST checked and made sure that in the HEX code display mode you now output the necessary characters
in its right part – while specifically using the Lister setting so that it takes the 65001 encoding for rendering characters. Otherwise, in HEX mode,
there will just be a bunch of unreadable characters on the right. And after I made sure that everything was okay, everything was displayed as it
should be, I decided that now it is definitely possible to switch to binary viewing – because for searching for TEXT I need exactly & only textual
information, not information about codes, which is what is in HEX mode. And in binary it should be just that. Just without HEX codes.
No, it's not, it will copy exactly the bytes (NOT characters) selected. When selecting from left to right, Lister will select entire characters, but when selecting/unselecting from right to left, it will not at line ends, so you may end with half characters.
I didn't quite understand the essence of the answer. I'm selecting the "Copy as text" option from the context menu.
Accordingly, I get the text in the clipboard – just as it should be.
It's just that some part of it gets lost. And it's not about the beginning or the end – it's about the middle!

So far, I can't clearly formulate reproducible steps for a test, unfortunately. I'll let you know how it goes.
#146217 personal license
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 53139
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Bugs with rendering HEX view of a file

Post by *ghisler(Author) »

Did you write separate code for each rendering mode?
Some code is shared, some is not. Have you tried using UTF-8 mode instead of code page 65001?
It's just that some part of it gets lost.
I don't have this problem. Can you give an example with the file exif_ru.lng? What do I need to select, and in which mode, to get wrong copied text?
Author of Total Commander
https://www.ghisler.com
Post Reply