Encoding of custom column language files and pluginst.inf - Unicode?
Moderators: white, Hacker, petermad, Stefan2
Re: Encoding of custom column language files and pluginst.inf - Unicode?
Ah, now it makes sense.
Regards
Dalai
Regards
Dalai
#101164 Personal licence
Ryzen 5 2600, 16 GiB RAM, ASUS Prime X370-A, Win7 x64
Plugins: Services2, Startups, CertificateInfo, SignatureInfo, LineBreakInfo - Download-Mirror
Ryzen 5 2600, 16 GiB RAM, ASUS Prime X370-A, Win7 x64
Plugins: Services2, Startups, CertificateInfo, SignatureInfo, LineBreakInfo - Download-Mirror
Re: Encoding of custom column language files and pluginst.inf - Unicode?
2ghisler(Author)
I'm afraid it's a total mess with Unicode support:
I'm afraid it's a total mess with Unicode support:
- LNG file may be in ANSI or UTF-8 with BOM, UTF-16 is not supported;
- MNU files may be in ANSI or UTF-8 with NO BOM only, UTF-16 is not supported;
- INI files may be in ANSI or UTF-16, UTF-8 support is broken…
- What about INF, INC, BAR and other files?
Andrzej P. Wozniak
Polish subforum moderator
Polish subforum moderator
- ghisler(Author)
- Site Admin
- Posts: 48154
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Re: Encoding of custom column language files and pluginst.inf - Unicode?
mnu and plugin ini files need to use the same encoding as the main lng file.
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
Re: Encoding of custom column language files and pluginst.inf - Unicode?
It would still be much better if TC could respect pure-Unicode INI files (which are 100% valid in Windows) and read them using pure-Unicode API...
Re: Encoding of custom column language files and pluginst.inf - Unicode?
2ghisler(Author)
Does that apply to pluginst.inf too ?
Does that apply to pluginst.inf too ?
License #524 (1994)
Danish Total Commander Translator
TC 11.03 32+64bit on Win XP 32bit & Win 7, 8.1 & 10 (22H2) 64bit, 'Everything' 1.5.0.1371a
TC 3.50 on Android 6 & 13
Try: TC Extended Menus | TC Languagebar | TC Dark Help | PHSM-Calendar
Danish Total Commander Translator
TC 11.03 32+64bit on Win XP 32bit & Win 7, 8.1 & 10 (22H2) 64bit, 'Everything' 1.5.0.1371a
TC 3.50 on Android 6 & 13
Try: TC Extended Menus | TC Languagebar | TC Dark Help | PHSM-Calendar
- ghisler(Author)
- Site Admin
- Posts: 48154
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Re: Encoding of custom column language files and pluginst.inf - Unicode?
While I could add this as a feature, the problem is that thousands of people will continue to use older versions for many years. Such language files wouldn't work for those users.It would still be much better if TC could respect pure-Unicode INI files (which are 100% valid in Windows) and read them using pure-Unicode API...
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
Re: Encoding of custom column language files and pluginst.inf - Unicode?
Well, wincmd.ini encoding may be safely changed, but for plugin LNG files you're right, this will require newest TC, so perhaps you could use ulng extension for UTF-16 files, or just leave this on users (update TC or edit LNG files). But this is the feature, so we need it...
Following parameters may be used for indication:
+ codepage=1200 for TC LNG files (UTF-16 codepage number)
+ codepage=1200 for wincmd.ini in [Configuration] section
+ some kind of similar parameter for plugin LNG files (e.g. codepage=1200 before all sections for manual parsing or in some section like [encoding] or [lng] for reading via API)
So all old LNG files would work as expected but new ones could use UTF-16.
Following parameters may be used for indication:
+ codepage=1200 for TC LNG files (UTF-16 codepage number)
+ codepage=1200 for wincmd.ini in [Configuration] section
+ some kind of similar parameter for plugin LNG files (e.g. codepage=1200 before all sections for manual parsing or in some section like [encoding] or [lng] for reading via API)
So all old LNG files would work as expected but new ones could use UTF-16.
Re: Encoding of custom column language files and pluginst.inf - Unicode?
2MVV
Codepage declaration is needed mostly for 8-bit charsets, including UTF-8, as BOM is not required in this case.
You don't need codepage declaration in UTF-16 files. They all must use BOM. The problem is that TC in some situations doesn't recognize BOM and ignores files with embedded 0x00 bytes. They are unreadable so codepage declaration inside such files is also ignored.
IMHO it's highest time to start working on fully Unicode Total Commander X, where X stands for number 10 and XP+ compatibility.
Codepage declaration is needed mostly for 8-bit charsets, including UTF-8, as BOM is not required in this case.
You don't need codepage declaration in UTF-16 files. They all must use BOM. The problem is that TC in some situations doesn't recognize BOM and ignores files with embedded 0x00 bytes. They are unreadable so codepage declaration inside such files is also ignored.
IMHO it's highest time to start working on fully Unicode Total Commander X, where X stands for number 10 and XP+ compatibility.
Andrzej P. Wozniak
Polish subforum moderator
Polish subforum moderator
Re: Encoding of custom column language files and pluginst.inf - Unicode?
Usher,
Direct UTF-16 indication is needed for TC to know that this INI (or LNG) must be read using Unicode API and not ANSI API. Windows allows transparently reading both UTF-16 and ANSI INI files with both Unicode and ANSI API (with automatic conversation according to system codepage if needed), and TC currently reads all INI files as ANSI with manual codepage conversation, but in case of UTF-16 INI manual conversation doesn't allow mixing languages/codepages in the same UTF-16 INI file.
Direct UTF-16 indication is needed for TC to know that this INI (or LNG) must be read using Unicode API and not ANSI API. Windows allows transparently reading both UTF-16 and ANSI INI files with both Unicode and ANSI API (with automatic conversation according to system codepage if needed), and TC currently reads all INI files as ANSI with manual codepage conversation, but in case of UTF-16 INI manual conversation doesn't allow mixing languages/codepages in the same UTF-16 INI file.
Re: Encoding of custom column language files and pluginst.inf - Unicode?
Well, in case of the suggestions to use a second file (*W.inf and *W.lng, or *.ulng) it's already indicated to TC that it should use Unicode API to read it. Don't you think such indicator is sufficient?
Regards
Dalai
Regards
Dalai
#101164 Personal licence
Ryzen 5 2600, 16 GiB RAM, ASUS Prime X370-A, Win7 x64
Plugins: Services2, Startups, CertificateInfo, SignatureInfo, LineBreakInfo - Download-Mirror
Ryzen 5 2600, 16 GiB RAM, ASUS Prime X370-A, Win7 x64
Plugins: Services2, Startups, CertificateInfo, SignatureInfo, LineBreakInfo - Download-Mirror
Re: Encoding of custom column language files and pluginst.inf - Unicode?
Yes, suffixed name (or prefixed/suffixed extension) is sufficient to indicate that file must be read as pure Unicode, of course.