[OT] AV software and scanning

[m^2]

Check the INI setting "ThreadFindFirst=1" for network probs.

Space calculation probs are highly related to realtime virus scanning. Exclude totalcmd.exe from AV, or try this UnUPXed totalcmd.exe:
https://plugins.ghisler.com/img/unpack32_701.exe

Icfu

The only advantage unUPXed totalcmd.exe has over regular one is faster startup with poor antivirus running. So it won't help here.

[mod]Thread split from Move out function executions from UI thread.

Hacker (Moderator)[/mod]

[icfu]

Well, the OP has even mentioned that his problems are related to "in the presence of an antivirus software". So, why are you sure that it doesn't help?

If you exclude totalcmd.exe, AV will not examine each and every file and directory totalcmd.exe wants to process. Well, this is what cool AV apps should do at least, otherwise you should dump the one you are using.

The DeUPXed totalcmd.exe can work too if the AV app sucks. Some of them may scan the Exe again and again and again and just not be able to look inside.

Icfu

[m^2]

Some of them may scan the Exe again and again and again and just not be able to look inside.

It doesn't make sense..they'd do better if they scanned memory image, not the file, this is not only faster, but also works better with self-modifying code.
I doubt that released antiviruses do this, but I don't know it.

[gigaman]

If you exclude totalcmd.exe, AV will not examine each and every file and directory totalcmd.exe wants to process.

In some AVs, you might be able to exclude the process (so that its file access is not being scanned). Normally, however, I think that "excluding totalcmd.exe" means that AV won't scan totalcmd.exe executable, not the executables totalcmd is trying to open.
Some AVs, on the other hand, are able to distinguish between opening and executing - so they won't bother scanning files when they are only read.

Actually, thinking about it, I think it's all nonsence - I suppose TotalCmd is just browsing the directories (when you press Space), not touching the particular files (why would it?). So antivirus doesn't have any reason to scan the files, slowing down the operation.

It doesn't make sense..they'd do better if they scanned memory image, not the file, this is not only faster, but also works better with self-modifying code.

Why's that?
First, I don't know why it should be faster (it may cause slightly higher memory usage, because it would cause bigger parts of the files to page-in at start), second - how could it help against self-modyfing code? If the file is being scanned right after it's mapped, it contains exactly the same as the data on disk; if it's scanned after the self-modyfing code has been executed, it's too late - the malware has already been activated.

[icfu]

I think that "excluding totalcmd.exe" means that AV won't scan totalcmd.exe executable, not the executables totalcmd is trying to open.

I wasn't talking about executing but processing directories, the topic of this thread. Of course, if TC starts a document or executable, the AV will scan it, no matter if totalcmd.exe is excluded or not.

I am in severe doubt that there are ANY intelligent AV apps out there. Not even Kaspersky has been able to understand my wish that I want to ONLY use the on demand scanning feature and to have it OFF during normal operation. What does it instead? It STILL updates it's fuckin' database files to grow bigger and bigger and bigger for no reason. Filemon proves that Kaspersky is active PERMANENTLY, even if you disable realtime protection.

My conclusion, I had already planned it years ago:
I don't use local AV crap anymore. When I need to scan a dubious file, I send it by mail to virustotal.com and wait for the results. This is more effective than just one single installation with no chance to decide if you have a false positive or not.

Icfu

[m^2]

Some AVs, on the other hand, are able to distinguish between opening and executing - so they won't bother scanning files when they are only read.

When you'll find AV which does it, uninstall it. It greatly speeds things up, but reading is not secure! A virus could put the malicious code in a dll and name it ringtone.mp3.
Then put the file in own memory space, find an entry point and just jump there. A slightly smarter AV would look for executable header, it's easy to recognize. But it's also easy to overcome beacuse you don't need the header, you need the code.

It doesn't make sense..they'd do better if they scanned memory image, not the file, this is not only faster, but also works better with self-modifying code.

Why's that?
First, I don't know why it should be faster (it may cause slightly higher memory usage, because it would cause bigger parts of the files to page-in at start), second - how could it help against self-modifying code? If the file is being scanned right after it's mapped, it contains exactly the same as the data on disk

It would be faster because memory is a lot faster than hard drives. If the program doesn't modify own code, there's no difference, just as you said. But upxed executables have to be read from hard drive. AV could use memory mapped files, so only the first scan would be slow, but I don't expect it from AV that's so stupid that scans one file multiple times.

if it's scanned after the self-modifying code has been executed, it's too late - the malware has already been activated.

In general it is. But there's always hope that you'll capture it's trial to harm computer and then - kill it.
BTW I'd like to know how would such AV kill the app. It should somehow prevent OS from calling DllMain in unloaded dlls

[gigaman]

Some AVs, on the other hand, are able to distinguish between opening and executing - so they won't bother scanning files when they are only read.

When you'll find AV which does it, uninstall it. It greatly speeds things up, but reading is not secure! A virus could put the malicious code in a dll and name it ringtone.mp3.
Then put the file in own memory space, find an entry point and just jump there. A slightly smarter AV would look for executable header, it's easy to recognize. But it's also easy to overcome beacuse you don't need the header, you need the code.

Erm, no
OK, I wasn't very precise - by "execution" I meant mapping into memory. Of course, loading of a DLL must be checked as well, just as execution.

But still - opening of a DLL file for simple reading doesn't have to. You said a virus could load something into its memory and execute it - sure, but then it's the virus executable that should be detected; if it can't activate, it cannot load its datafile into memory.

It doesn't make sense..they'd do better if they scanned memory image, not the file, this is not only faster, but also works better with self-modifying code.

Why's that?
First, I don't know why it should be faster (it may cause slightly higher memory usage, because it would cause bigger parts of the files to page-in at start), second - how could it help against self-modifying code? If the file is being scanned right after it's mapped, it contains exactly the same as the data on disk

It would be faster because memory is a lot faster than hard drives. If the program doesn't modify own code, there's no difference, just as you said. But upxed executables have to be read from hard drive. AV could use memory mapped files, so only the first scan would be slow, but I don't expect it from AV that's so stupid that scans one file multiple times.

If the file is mapped into memory by AV and scanned, it has to be read from the hard disk anyway. So, it doesn't really matter. Besides, if it's read in an "ordinary way" (i.e. not mapped), the cache manager will most likely cache the data, so the subsequent program execution will be just a memory copy... no speed difference.
Also, I don't see any difference about packed/unpacked files here; it's always the disk file that's mapped/read.

if it's scanned after the self-modifying code has been executed, it's too late - the malware has already been activated.

In general it is. But there's always hope that you'll capture it's trial to harm computer and then - kill it.
BTW I'd like to know how would such AV kill the app. It should somehow prevent OS from calling DllMain in unloaded dlls

I doubt (though I am not completely sure about this) that any reasonable AV does that; allowing an executable to run and hoping it's gonna be killed later if it's found malicious... that's basically a failure of the AV, it let the malicious file be executed.

[m^2]

Some AVs, on the other hand, are able to distinguish between opening and executing - so they won't bother scanning files when they are only read.

When you'll find AV which does it, uninstall it. It greatly speeds things up, but reading is not secure! A virus could put the malicious code in a dll and name it ringtone.mp3.
Then put the file in own memory space, find an entry point and just jump there. A slightly smarter AV would look for executable header, it's easy to recognize. But it's also easy to overcome beacuse you don't need the header, you need the code.

Erm, no
OK, I wasn't very precise - by "execution" I meant mapping into memory. Of course, loading of a DLL must be checked as well, just as execution.


But still - opening of a DLL file for simple reading doesn't have to. You said a virus could load something into its memory and execute it - sure, but then it's the virus executable that should be detected; if it can't activate, it cannot load its datafile into memory.

I think understood you correctly, if not, your further explanation didn't help
You don't need kernel32.dll::LoadLibrary to load libraries, you can implement your own, if you need only to execute it's main function and don't need HMODULE, it's a piece of cake-in the previous post I wrote everything you need to do. If you have several pages with PAGE_EXECUTE_READWRITE, you can do it.

Why's that?
First, I don't know why it should be faster (it may cause slightly higher memory usage, because it would cause bigger parts of the files to page-in at start), second - how could it help against self-modifying code? If the file is being scanned right after it's mapped, it contains exactly the same as the data on disk

It would be faster because memory is a lot faster than hard drives. If the program doesn't modify own code, there's no difference, just as you said. But upxed executables have to be read from hard drive. AV could use memory mapped files, so only the first scan would be slow, but I don't expect it from AV that's so stupid that scans one file multiple times.

If the file is mapped into memory by AV and scanned, it has to be read from the hard disk anyway. So, it doesn't really matter. Besides, if it's read in an "ordinary way" (i.e. not mapped), the cache manager will most likely cache the data, so the subsequent program execution will be just a memory copy... no speed difference.

Cache manager can decide that there's something more important and it doesn't have enough space for this file. It can also decide to unload it too quickly. As long as cache manager knows what will be needed later, it's as fast as MMF. But it usually has to guess...right here situation seems clear, at least after several reads. But I wouldn't trust it.

Also, I don't see any difference about packed/unpacked files here; it's always the disk file that's mapped/read.

I was refering to hypothetical situation when AV can scan memory image of a file. And maybe I went too far...

if it's scanned after the self-modifying code has been executed, it's too late - the malware has already been activated.

In general it is. But there's always hope that you'll capture it's trial to harm computer and then - kill it.
BTW I'd like to know how would such AV kill the app. It should somehow prevent OS from calling DllMain in unloaded dlls

I doubt (though I am not completely sure about this) that any reasonable AV does that; allowing an executable to run and hoping it's gonna be killed later if it's found malicious... that's basically a failure of the AV, it let the malicious file be executed.

As I said before, I also doubt that any AV scans executable during it's run (if is not clearly said to do it, is not just starting etc.).

[gigaman]

You don't need kernel32.dll::LoadLibrary to load libraries, you can implement your own, if you need only to execute it's main function and don't need HMODULE, it's a piece of cake-in the previous post I wrote everything you need to do. If you have several pages with PAGE_EXECUTE_READWRITE, you can do it.

Yes, I understand that - the question is: where do you put this CustomLoadLibrary function (what file, I mean)? How do you call it (where from)? (and maybe also: why would you do that?)

[m^2]

You don't need kernel32.dll::LoadLibrary to load libraries, you can implement your own, if you need only to execute it's main function and don't need HMODULE, it's a piece of cake-in the previous post I wrote everything you need to do. If you have several pages with PAGE_EXECUTE_READWRITE, you can do it.

Yes, I understand that - the question is: where do you put this CustomLoadLibrary function (what file, I mean)? How do you call it (where from)? (and maybe also: why would you do that?)

I'd put it in the main virus executable. This function isn't malicious, so it can be there. Where to call it from? Actually it can be the main virus function. Why would I do that? Because with such implementation, "smart" AV which scans only executables will miss malicious code that's placed in a data file.

[gigaman]

I'd put it in the main virus executable. This function isn't malicious, so it can be there. Where to call it from? Actually it can be the main virus function. Why would I do that? Because with such implementation, "smart" AV which scans only executables will miss malicious code that's placed in a data file.

Antiviruses would simply detect the executable, no doubt about it. This function may not be malicious itself (even though it's questionable if its purpose is to avoid AV detection), but the same can be said about many other pieces of code in viruses (library functions, ...). But still, it's a malicious executable - so it would be detected.
If it's reading additional parts of its code from another file, or of it's downloading them from Internet, or if it simply has them crypted in its own (exe) body, it doesn't matter - there's no need to bother with these additional parts if you can simply detect the main exe.

[m^2]

I'd put it in the main virus executable. This function isn't malicious, so it can be there. Where to call it from? Actually it can be the main virus function. Why would I do that? Because with such implementation, "smart" AV which scans only executables will miss malicious code that's placed in a data file.

Antiviruses would simply detect the executable, no doubt about it. This function may not be malicious itself (even though it's questionable if its purpose is to avoid AV detection), but the same can be said about many other pieces of code in viruses (library functions, ...). But still, it's a malicious executable - so it would be detected.
If it's reading additional parts of its code from another file, or of it's downloading them from Internet, or if it simply has them crypted in its own (exe) body, it doesn't matter - there's no need to bother with these additional parts if you can simply detect the main exe.

You can detect everything if you have a signature. The goal of virus writers is to pass heuristic analysis and (usually) barricade (or hide) in a system to last there as long as possible.

[gigaman]

Ah, heuristics... I don't think scanning of ordinary reads will bring any significant increase of security here either. It doesn't make much sense (for the virus writer) to read an ordinary DLL file from disk and then make its own loader. The DLL file would get detected by on-demand scanners at least... so if they make their own loader, why not read a completely encrypted file instead (i.e. no MZ/PE header, no plaintext code)? Such a file won't get detected by heuristics anyway, no matter if the file access is hooked or not, and the change in the loader is trivial.

[m^2]

Ah, heuristics... I don't think scanning of ordinary reads will bring any significant increase of security here either. It doesn't make much sense (for the virus writer) to read an ordinary DLL file from disk and then make its own loader. The DLL file would get detected by on-demand scanners at least... so if they make their own loader, why not read a completely encrypted file instead (i.e. no MZ/PE header, no plaintext code)? Such a file won't get detected by heuristics anyway, no matter if the file access is hooked or not, and the change in the loader is trivial.

Yes, you're right. But c.a. doubles effort.