The file name contains characters typical for a virus/worm. (Unicode RTL Spiegelschrift Bidi-Marker)

Bug reports will be moved here when the described bug has been fixed

Moderators: white, Hacker, petermad, Stefan2

User avatar
ts4242
Power Member
Power Member
Posts: 2081
Joined: 2004-02-02, 20:08 UTC
Contact:

The file name contains characters typical for a virus/worm. (Unicode RTL Spiegelschrift Bidi-Marker)

Post by *ts4242 »

The following error message appear when trying to open text file

Code: Select all

---------------------------
اسم الملف.txt
---------------------------
WARNING: The file name contains characters typical for a virus/worm. Function aborted.
---------------------------
OK   
---------------------------
This message occurs only with TC 9.0 but no problem with older versions

I didn't create this file but downloaded from the internet
Last edited by ts4242 on 2016-11-19, 22:18 UTC, edited 1 time in total.
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 »

Interesting.
The only suspicious character is
U+202B
(RIGHT-TO-LEFT EMBEDDING)

Which can be pretty normal on some texts. Copying such text to a filename shouldn't be considered harmful, so seems a bit overcautious IMO.
TC plugins: PCREsearch and RegXtract
User avatar
ts4242
Power Member
Power Member
Posts: 2081
Joined: 2004-02-02, 20:08 UTC
Contact:

Post by *ts4242 »

2milo1012

I found this in History.txt under Release Total Commander 8.52 beta 1
History.txt wrote:05.05.15 Fixed: Remove right to left markers from file names (used almost exclusively by viruses/worms) before displaying them in file lists (32/64)
I also wonder how file name can be harmful of any kind, specially it is contain just plain text no executable code at all!
User avatar
Stefan2
Power Member
Power Member
Posts: 4133
Joined: 2007-09-13, 22:20 UTC
Location: Europa

Post by *Stefan2 »

I also wonder how file name can be harmful of any kind,
specially it is contain just plain text no executable code at all!

See, "TXT.exe" could be displayed as
"Banking house ann<RIGHT-TO-LEFT>exe.TXT"

But is in real
"Banking house annTXT.exe"      ...and BOOM   :shock:



 
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 »

ts4242 wrote:2milo1012I found this in History.txt under Release Total Commander 8.52 beta 1
History.txt wrote:05.05.15 Fixed: Remove right to left markers from file names (used almost exclusively by viruses/worms) before displaying them in file lists (32/64)
Well, this just means that the filename will look different in TC than in Explorer, due to removed Bidi markers.
(on my Windows set to Western Europe locale, "txt" appears on the right in TC, but on the left in Explorer!)
I'm not sure if the warning is related to this or was actually added in an earlier or later TC version.
ts4242 wrote:I also wonder how file name can be harmful of any kind, specially it is contain just plain text no executable code at all!
Well, Stefan2 explained one example. More details:
https://blog.malwarebytes.com/cybercrime/2014/01/the-rtlo-method/
and
http://superuser.com/questions/408792/what-are-ways-to-prevent-files-with-the-right-to-left-override-unicode-character
Each tells us that especially U+202E (RIGHT-TO-LEFT OVERRIDE) is the main security culprit. (not the mentioned U+202B, or the simple U+200E or U+200F.)


Anyway, I don't think it's a file managers task to make arbitrary decisions about what is a harmful filename and what not. Especially blocking the complete range of RTL markers is just too much, at least when used without a basic analysis, like if the file extension is even affected at all. And like I said above, copying any normal piece of text, that by chance contains an Bidi(RTL) marker, to name a file with it, would probably trigger this behavior. So at least an Ini option would be advisable for this (both, removing the Bidi markers in the file list, and the warning).
TC plugins: PCREsearch and RegXtract
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

This is intentional and will not be changed.

It's done to fight against viruses which store names like this:
"Photo01 By<right to left marker> gpj.SCR"
they appear as
Photo01 By RCS.jpg

If you see a way to distinguish this malicious naming scheme from legitimate use, I will gladly change it, but for now it's better to block it.
Author of Total Commander
https://www.ghisler.com
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

I think I found a reasonable compromize now:
1. Remove RTL markers when displaying names, but only when the extension is displayed attached to the file name (not aligned extensions).

2. Allow to open files containing RTL markers, but only if the name contains Unicode characters from the right to left group: Unicode 0590-08FF, e.g. Arabic, Hebrew, Syric etc. The viruses/worms using this scheme always used Western characters so far, and RTL markers have no reasonable use in this case. And (1) ensures that the user does see the real file extension.
Author of Total Commander
https://www.ghisler.com
User avatar
Dalai
Power Member
Power Member
Posts: 9364
Joined: 2005-01-28, 22:17 UTC
Location: Meiningen (Südthüringen)

Post by *Dalai »

2ghisler(Author)
What does (1) mean for aligned extensions? Will TC show the real extension in such cases, too?

Regards
Dalai
#101164 Personal licence
Ryzen 5 2600, 16 GiB RAM, ASUS Prime X370-A, Win7 x64

Plugins: Services2, Startups, CertificateInfo, SignatureInfo, LineBreakInfo - Download-Mirror
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

Yes, because when using aligned extensions, the filename is split into name and extension parts at the last dot. Since the right to left marker is before the last dot, the extension isn't affected by it when using this dirty trick to modify the name. If the right to left marker were after the last dot, the extension wouldn't work anyway.
Author of Total Commander
https://www.ghisler.com
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 »

2ghisler
Well, I have to see how this works in practice for the next version, and I'm not really sure what you mean by "aligned extension".

I think the easiest pre-filter would be:
Check if there is only ONE Bidi marker in the name string and if this one is NOT U+202E. If both is the case, the filename should be considered harmless, since all other markers can not really fake the extension, but only make it switch sides altogether at the most, e.g.
[U+202B][RLE characters][.jpg] would read as [jpg.][RLE characters]

In case of multiple Bidi marker characters, the Unicode Bidirectional Algorithm can be quite complex, meaning that multiple levels (up to 125 IIRC) can validate and invalidate the reading direction. I'm not sure if a simple filter for the RLO character (like explained in my linked articles) would be enough, or if you'd even be able to analyze the file extension isolated on your own w/o implementing the whole Bidi algorithm by yourself.


Anyway, what I meant in my last post is that the filter should still be optional to some degree (Ini option), or if the file behind it even has the "MZ" magic number for executable files. Yes, I know that fake extension might be even harmful for other file types, like a doc file with macros instead of a mere pdf file, but I think the executable type is still the main security issue out there. So a simple file name filter might be a good idea, but becomes useless if it's too overcautious, like the well-known virus scanners false alarms we all love so dearly ;)


Edit: typo
Last edited by milo1012 on 2016-11-22, 15:24 UTC, edited 2 times in total.
TC plugins: PCREsearch and RegXtract
User avatar
MVV
Power Member
Power Member
Posts: 8702
Joined: 2008-08-03, 12:51 UTC
Location: Russian Federation

Post by *MVV »

I'm not really sure what you mean by "aligned extension"
It is about alignment of file extensions on Tabstops configuration page: true extension is shown separately from the rest of file name so it will be shown properly always.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

2ts4242
I have sent you a test version of Total Commander to your forum e-mail account. Could you try it please?
Author of Total Commander
https://www.ghisler.com
User avatar
ts4242
Power Member
Power Member
Posts: 2081
Joined: 2004-02-02, 20:08 UTC
Contact:

Post by *ts4242 »

2ghisler(Author)

I can open the file now, there is no warning message, but according to your described solution if i set the option to show file extension directly after file name, there should be a warning, but this doesn't happen!

Here is the file name cause the problem

Code: Select all

MIME-Version: 1.0
Content-Type: application/octet-stream; name="rtl_test_file.rar"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="rtl_test_file.rar"

UmFyIRoHAM+QcwAADQAAAAAAAADwiXQgkkgAGwAAADcAAAACER7/ODW7dkkdMyMAIAAAAKXpP+3p
IKCp6qTvoS50eHQABlstRCsggFs1IICIRSxcSikCALBrdUQIwMvsz3QeHe5qCDnb/5eUu8ynroFZ
veLqHNDEPXsAQAcA

User avatar
MVV
Power Member
Power Member
Posts: 8702
Joined: 2008-08-03, 12:51 UTC
Location: Russian Federation

Post by *MVV »

I've created a double-extension file and tested it with different TC versions and in Explorer: Explorer and old TC show "exe" at the beginning of name while TC 8.52 and 9.0+ show "exe" at the end of name (or in extension column if extensions are aligned). And test TC version doesn't show warnings anymore.

Code: Select all

MIME-Version: 1.0
Content-Type: application/octet-stream; name="rtl_test.7z"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="rtl_test.7z"

N3q8ryccAAPZMhxmywAAAAAAAACMAAAAAAAAADXaSv8AJpaOcAAX9+wFu+r0/5QBL0TvfOb1ZGdf
BpzE7zSCTWOFHYXpPuzSdg3nHODbbsyd+5i8Mo6/QjZOMYrPqet7vXh5NCnLnohK3fw3K6ORFmAW
DTN+axoobBqucxR7p6c7H6sMPbr7lyzuO3Qf3gLr30cLjf6iAy8AqXhxPpijJ/aGKVpAom7lr+7s
bcbGTsZyyKxl+uhWUHXxRdNP/mFOwCJlrnC9C2QQ4D7FQvDYqNjV4lg7SDDTrnBeAAAAAAAAAAAA
AAAAAAAAAAEEBgAECYC8BQUFAAcLAQAEIwMBAQVdAAABACMDAQEFXQAAAQAjAwEBBV0AAAEAFAMD
ARsEAQUABAEDAgIGAQAMAACEAIQAAAgKAUHCs30AAAUBESEAKyBoAGUAbABsAG8ALgB0AHgAdAAx
Bi4AZQB4AGUAAAAUCgEAPpmQMqMLzgEVBgEAIAgAAAAA
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 »

MVV wrote:I've created a double-extension file and tested it with different TC versions and in Explorer: Explorer and old TC show "exe" at the beginning of name while TC 8.52 and 9.0+ show "exe" at the end of name (or in extension column if extensions are aligned).
I already showed this above.
But this a different thing compared to RLO (U+202E), where all characters mirror, like:

Code: Select all

HP_SCAN_FORM_N90952011___Coll.[U+202E]fdp.exe
The user would see in explorer "foo.pdf" when file extensions are masked, or "foo.exe.pdf" when not or when in TC < 8.52.
In opposite to that, the U+202B character will swap whole filename parts after possibly following RLE chars (Arabian, etc.), but only IF such char(s) will follow. The user would see "exe" at the name's beginning, which might be confusing as a name start, but might still be better than RLO. But it's up to Christian if he wants to block such a case of mixed/non-RLE/RLE chars as well.


ts4242 wrote:but according to your described solution if i set the option to show file extension directly after file name, there should be a warning, but this doesn't happen!
I don't have the test version, but I think it's intended to be the reversed way:
ghisler(Author) wrote:2. Allow to open files containing RTL markers, but only if the name contains Unicode characters from the right to left group: Unicode 0590-08FF
Since your test filename contains such chars, it works as described, issuing no warning any more.
Last edited by milo1012 on 2016-11-23, 00:07 UTC, edited 1 time in total.
TC plugins: PCREsearch and RegXtract
Post Reply