Poor performance of ALT+F7 search

[digitaldog4]

I'm also using FindFirstFileExW directly. It's much faster when you repeat a search because it takes data from cache.
If you need extremely fast search, use Everything - there is no way to search an entire disk (with Windows on it) in just 6 seconds if no indexing is used.

You have the evidence in the screencast above. It's not an entire disk. In the first case it was my profile directory, in the second case it was C:\Windows.

Why the status bar needs to be updated > 1000 times per second?

[digitaldog4]

Search query in TC using tool 'Everything'
602 423 items (485 049 files, 117 374 folders)

Search for:

Code: Select all

ev:files:asdfasdfasdfasdf

Search results:

Code: Select all

[No files found]

Time of scan:
<1 second

---

---

Search directly in Everything 1.5.0.1350a (x64)

Search for:

Code: Select all

files:asdfasdfasdfasdf

Search result:

Code: Select all

0 items (0 files, 0 folders)

Time of scan:
<1 second

I'm hesitant to use yet another software to do thing that Total Commander can easily do by itself by just not updating UI so often.

[Dalai]

Or maybe TC's UI status bar updates are causing slowdown. It's as simple as that.

This surely will add to the total time (compared to less UI updates and Double Commander). But as I said: my search times are not even in the ballpark of the one minute mark you have. It can't just be UI updates responsible for this HUGE difference. Just try it without anti-virus software, or even better: in Windows Safe Mode.

Regards
Dalai

[Galizza]

 
I've just tested Tc 10.52 & Dc 1.0.11 both 32bits, searching through the Windows directory

Tested on Windows 10 SAFE MODE :

I made 16 consecutive searches, discarding first four so the directory contents are served from the system cache.

Tc searches, 5th to 16th = Same time on all searches 36 seconds

Dc searches, 5th to 16th = Same time on all searches 9 seconds

 

[Dalai]

I ran some tests on a different computer (old Athlon II X2 250 with Win7 installed) running Windows PE. This eliminates all factors of the OS accessing the same disk and background software. Here are the results:

• The maximum amount of time to search through the Windows directory (of the installed Windows) was about 30 seconds for the first search. This was when running WinPE 10.0 (1809) x64.
• The fastest search with about 13 seconds was measured on WinPE 3.1 (based on Win7) x86.
• TC 64-bit is slower than TC 32-bit. In some cases it took about double the amount of time.
• TC 8.52a is a little bit faster than TC 10.52, up to around 20%.
• Repeated searches take between 8 and roughly 24 seconds, depending on the environment, i.e. TC version and bitness and OS version and bitness - and of course run to run variance.
• The newer the WinPE environment the slower it is, i.e. WinPE 3.1 is faster than 5.1 and that's faster than Win10 (1809)

I can't make any tests with a Windows directory of a Win10 installation because I don't have a Win10 system installed on bare metal, only a couple VMs. Timing a VM would be neither fair nor realistic or comparable because all these VMs are located on an HDD while all physical systems are on an SSD.

BTW, another way to check if and how much UI updates add to the total search time: Run repeated searches so the directory contents are served from the system cache. That way the most relevant factors are UI updates and CPU and cache/memory performance.

Regards
Dalai

[funkymonk]

I also have the impression (based on using other programs) that the search in TC is not as fast as it could be.

The GUI update could be at least a partial reason for this.
But to put an end to the speculation:

@Ghisler Couldn't you "just" try how fast TC would search completely without GUI updates?

If it makes a big difference, surely a trade-off can be found.

Note: Of course one can use Everything (or maybe other tricks/additions) to speed up the search. However, I would expect a file manager to support the basic file handling functions (e.g. searching for files) with **high performance** **on its own**.

[AntonyD]

And at least it would be possible to evaluate the code from another application from a professional point of view in order to unambiguously compare the current implementation and the other. Perhaps there is still some clever trick hiding there.

[ghisler(Author)]

It could indeed be the GUI update. When this code was written, harddisks were slow and reading the directories was the limiting factor. Now with modern SSDs reading multiple directories is much faster (no head positioning), and processing power has increased a lot too. I will try limiting the refresh rate of the status bar to check if it makes any difference. I did the same previously for the copy operation and it had a big effect.

[tuska]

I'm hesitant to use yet another software to do thing that Total Commander can easily do by itself by just not updating UI so often.

I would not hesitate.

If you get indexed(!) search results (including file contents) in Total Commander when using Everything
almost in real time, then you will hardly want to do without it.

A further development in TC can shorten the time for the search result preparation in TC even more.

[digitaldog4]

It could indeed be the GUI update. When this code was written, harddisks were slow and reading the directories was the limiting factor. Now with modern SSDs reading multiple directories is much faster (no head positioning), and processing power has increased a lot too. I will try limiting the refresh rate of the status bar to check if it makes any difference. I did the same previously for the copy operation and it had a big effect.

That's exactly my thinking. I remember floppy disks, when reading directories and files was audible - you could hear the drive head moving and it could take as much as 1 second.

Thank you for your positive attitude towards this issue.

[funkymonk]

It could indeed be the GUI update. When this code was written, harddisks were slow and reading the directories was the limiting factor. Now with modern SSDs reading multiple directories is much faster (no head positioning), and processing power has increased a lot too. I will try limiting the refresh rate of the status bar to check if it makes any difference. I did the same previously for the copy operation and it had a big effect.

Thank you for giving it a try!

Looking forward to the result...

[ghisler(Author)]

It does make a difference, although not as big as in your case - it goes from about 40 to 7 seconds, which is already rather significant. But things like ignore list and complex name matching also takes time.

[funkymonk]

Huh! That's indeed significant!
Looking forward to TC11.00b7 with a great speed up!

[digitaldog4]

It does make a difference, although not as big as in your case - it goes from about 40 to 7 seconds, which is already rather significant. But things like ignore list and complex name matching also takes time.

Wonderful! Can't wait to try it

[Galizza]

2ghisler(Author)

I will try limiting the refresh rate of the status bar to check if it makes any difference. I did the same previously for the copy operation and it had a big effect.

Is it possible that limiting refresh rate on Verify Checksums window could increase verify speed ?, may you please try it ?. Thanks.