Multi-Rename Tool - Unicode support issue

[dyhan81]

Multi-Rename Tool is Unicode support tool but, in some function on Multi-Rename Tool, not support Unicode.

At Context Menu (on the right of Upper/Lowercase), if names are edited with Unicode, it is possible to some Unicode characters can get flaw because the temporary file that be used for filenames editing is not Unicode format but ANSI format.

For example, One of the Japanese animation titles,

やっぱりふたりはプリッキュア～！伝説は続くよどこまでも！ (Unicode)

was saved in ANSI encoding that string got lose some characters like this:

やっぱりふたりはプリッキュア～！??は?くよどこまでも！ (ANSI)

As you can see some characters on this string, 伝, 説, and 続, substituted with ? (question mark). This phenomenon caused by Unicode character mapping on some locale (on this case, Korean) cannot map perpectly...

Anyway, I think, if we solve this problem, renaming temp file will be edited and read in Unicode, and also apply it to multirename tool's new name.

[ghisler(Author)]

I guess that you used the option "Edit names" of the rename tool? I cannot confirm this error. I checked my code, TC converts all names to ANSI and then back to Unicode. If the name is different, TC will create a Unicode text file, otherwise an ANSI text file.

Here is what I did:
1. Go to Control Panel - Regional and Language options
2. Set the language to Korean for numeric formats, and also the location to Korea
3. Also set the language for non-Unicode programs to Korean
4. Had to restart Windows to apply this change
5. Create file named やっぱりふたりはプリッキュア～！伝説は続くよどこまでも！ with Shift+F4
6. Select the file
7. Press Ctrl+M
8. Click on menu button, choose "Edit names"
-> notepad opens with that name.
9. Switched font to "Microsoft Sans Serif" because some characters appeared as squares
10. Edit the name (putting a '2' at the end)
11. Looked in "Save as" dialog to be sure the file would be saved as Unicode (it was preselected)
12. Saved the file via "Save"
13. Clicked button to take new names -> name was used correctly.

What did you do differently?

[dyhan81]

Thanks to your effort for the bug reproduction.

I guess that you used the option "Edit names" of the rename tool?

Yes, you're right.

I cannot confirm this error. I checked my code, TC converts all names to ANSI and then back to Unicode. If the name is different, TC will create a Unicode text file, otherwise an ANSI text file.

Oh, I've got the problem in the method of creating the text file. In the case that renaming texts are containing some Unicode characters, that will
make the Unicode text file what we want, but if texts are not containing any Unicode characters, then file will be make as ANSI file.

In this case, because the text file is made as ANSI file, we cannot change file name into Unicode name.

Here is what I did:
1. Go to Control Panel - Regional and Language options
2. Set the language to Korean for numeric formats, and also the location to Korea
3. Also set the language for non-Unicode programs to Korean
4. Had to restart Windows to apply this change
5. Create file named やっぱりふたりはプリッキュア～！伝説は続くよどこまでも！ with Shift+F4
6. Select the file
7. Press Ctrl+M
8. Click on menu button, choose "Edit names"
-> notepad opens with that name.
9. Switched font to "Microsoft Sans Serif" because some characters appeared as squares
10. Edit the name (putting a '2' at the end)
11. Looked in "Save as" dialog to be sure the file would be saved as Unicode (it was preselected)
12. Saved the file via "Save"
13. Clicked button to take new names -> name was used correctly.

What did you do differently?

OK. Then do like this instead:
5. Create file named "a" with Shift+F4
10. Edit the name to "やっぱりふたりはプリッキュア～！伝説は続くよどこまでも！"
14. Press 'Start!' button on Multi-Rename Tool Main window to start renaming. (probably you will get an error message)

[ghisler(Author)]

Well, it depends on the editor used. With notepad, I get a warning that the text contains Unicode characters which would be lost if saved like this. So when I save the file as Unicode, TC can pick up the names.

Unfortunately I cannot just use Unicode all the time, because many many editors out there are simply not Unicode-capable.

[dyhan81]

Unfortunately I cannot just use Unicode all the time, because many many editors out there are simply not Unicode-capable.

You're right... Actually, that's what I've expected and real problem... backwards compatibility.

Um.... oh, then how about using UTF-8 encoding? That can be a solution.

Common ASCII characters are also UTF-8 characters. if files are named as just ASCII characters, although that names are encoded to UTF-8, that are still ASCII that can be read in common text editors.

[ghisler(Author)]

The problem is that if the names contains characters from the current codepage, they would be scrambled when encoded as UTF-8 and the editor doesn't support it.

The only solution I see would be just another configuration option...

[ghisler(Author)]

You can now tell Total Commander to always pass Unicode files to the file name editor:

Add to wincmd.ini section [Configuration] the following:
RenameEditUnicode=1