"Compare by Content" takes a few seconds when 100% CPU usage

English support forum

Moderators: white, Hacker, petermad, Stefan2

Post Reply
menaquinone
Junior Member
Junior Member
Posts: 6
Joined: 2020-06-08, 05:08 UTC

"Compare by Content" takes a few seconds when 100% CPU usage

Post by *menaquinone »

When system is using 100% CPU (ie. Prime95), then comparing(Files->Compare By Content...) two text files of about 100KiB each, the comparison slows down at 60% and takes a few seconds to complete. Only when complete you can see the first diff. Happens on both Intel and AMD processors.

Total Commander 9.51 64bit

Why does this happen? Will it be fixed ?


I have a 2.3MiB .wmv video showcasing this but have no idea where I could upload it for you to see.
User avatar
Horst.Epp
Power Member
Power Member
Posts: 6450
Joined: 2003-02-06, 17:36 UTC
Location: Germany

Re: "Compare by Content" takes a few seconds when 100% CPU usage

Post by *Horst.Epp »

I guess it will not be fixed, as there is no such common problem in TC.
You may have problems with a Anti-Virus software or other add-ons.
Windows 11 Home x64 Version 23H2 (OS Build 22631.3374)
TC 11.03 x64 / x86
Everything 1.5.0.1371a (x64), Everything Toolbar 1.3.2, Listary Pro 6.3.0.69
QAP 11.6.3.2 x64
menaquinone
Junior Member
Junior Member
Posts: 6
Joined: 2020-06-08, 05:08 UTC

Re: "Compare by Content" takes a few seconds when 100% CPU usage

Post by *menaquinone »

Horst.Epp wrote: 2020-06-08, 07:07 UTC I guess it will not be fixed, as there is no such common problem in TC.
You may have problems with a Anti-Virus software or other add-ons.
Maybe I can provide the steps necessary to replicate the issue... I could upload the two .txt files on github's gist perhaps? and you'd only need a copy of prime95 then. Still no idea where I could upload the video though. (I don't have gmail/youtube)

I don't use Antivirus(but maybe windows' defender is still active?), no plugins in Total Commander, I'm on Windows 7 64bit. "other add-ons" are practically nil. I was guessing that maybe that diff code in TC is riddled with give_CPU_time_slice (so to speak) and that's why it got slower only during load. Who knows...
menaquinone
Junior Member
Junior Member
Posts: 6
Joined: 2020-06-08, 05:08 UTC

Re: "Compare by Content" takes a few seconds when 100% CPU usage

Post by *menaquinone »

ok, here's the video, I uploaded it on mozilla's bugzilla :)
https://bugzilla.mozilla.org/attachment.cgi?id=9154928

...it downloads it instantly if you visit that link(a 2.3MiB .wmv, please use VLC otherwise it might freeze the video when playing with MPC-HC for example).

Otherwise you can visit the bug[1] that I uploaded it to, then click Show Obsolete then click the howto.wmv one to get it. That bug is unrelated to this video, but had nowhere else to upload it, so there.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1644008

Here's virustotal report(clean) for that .wvm if anyone's worried: https://www.virustotal.com/gui/file/9fa85b020fae38b7bcaedf13bec40e9dabd9f1176889dfa1cc4f3de0f9a866a5/detection
User avatar
Dalai
Power Member
Power Member
Posts: 9364
Joined: 2005-01-28, 22:17 UTC
Location: Meiningen (Südthüringen)

Re: "Compare by Content" takes a few seconds when 100% CPU usage

Post by *Dalai »

Can't reproduce this. Just tested Prime95 29.8 on my Ryzen 2600 with Small FFTs on all 12 threads, then comparing a ~400 KB binary file (mail program index DB) with a copy on my server. I also compared a ~500 KB log file (with a copy on the server). No issues whatsoever, not in any TC version (8.52, 9.51, 9.51 x64).

I suggest to check for other background programs, especially monitoring programs that access sensors and the like.

Regards
Dalai
#101164 Personal licence
Ryzen 5 2600, 16 GiB RAM, ASUS Prime X370-A, Win7 x64

Plugins: Services2, Startups, CertificateInfo, SignatureInfo, LineBreakInfo - Download-Mirror
menaquinone
Junior Member
Junior Member
Posts: 6
Joined: 2020-06-08, 05:08 UTC

Re: "Compare by Content" takes a few seconds when 100% CPU usage

Post by *menaquinone »

Thank you for testing Dalai. I cannot reproduce this by trying other files. Perhaps there's something specific* about my two files which I've tested to reproduce this? For this reason I'm uploading them on github's gist. Could you please try with these two files?

* maybe the number of things that are different and how far apart they are? wild guess.

Save this file as a.txt in a subdir named "left": https://gist.githubusercontent.com/menaquinoneK2/b6bd79293800b2a7ed8045860b6505ef/raw/dee15b8f5636bd3abfa26f535446631642c364cc/a.txt

Save this file as a.txt in a subdir named "right": https://gist.githubusercontent.com/menaquinoneK2/cdbb9c42b2501aa250ea1965805c7db1/raw/50e15753b1cd6e117c8f81608ee737107951b71d/a.txt

To be sure this issue still works, I've retried to replicate it as above, with Prime95 v29.8,build 6 with Smallest FFTs(because Small FFTs is not available on this AMD CPU, doesn't have L3 cache?). I used "Save page as" from Firefox to save the above two files.

On this AMD CPU it takes about 5 seconds to complete the comparison while Prime95 is using 100% CPU (all 4 cores). Note: I even have foobar2000 playing music in the background without any slowdown issues.

Since this happens on Intel too, I don't think it's a CPU specific thing. But since it only happens, thus far, with these two files from above... I don't know what to think. Thoughts?
menaquinone
Junior Member
Junior Member
Posts: 6
Joined: 2020-06-08, 05:08 UTC

Re: "Compare by Content" takes a few seconds when 100% CPU usage

Post by *menaquinone »

hmm. If I deselect "Ignore repeated spaces" then the delay is gone and the Comparison is instant ..
User avatar
Dalai
Power Member
Power Member
Posts: 9364
Joined: 2005-01-28, 22:17 UTC
Location: Meiningen (Südthüringen)

Re: "Compare by Content" takes a few seconds when 100% CPU usage

Post by *Dalai »

Can confirm your findings. It depends on the files compared, and repeated spaces, and maybe even the character set. Your files are automatically detected as UTF-8. When switching one side to UTF-8 and the other to ANSI, it's slow again even when "Ignore repeated spaces" is disabled.

Not sure what to make of this. In reality it shouldn't happen that often. And if it only occurs in conjunction with Prime95, I wouldn't worry about it at all.

Regards
Dalai
#101164 Personal licence
Ryzen 5 2600, 16 GiB RAM, ASUS Prime X370-A, Win7 x64

Plugins: Services2, Startups, CertificateInfo, SignatureInfo, LineBreakInfo - Download-Mirror
User avatar
Horst.Epp
Power Member
Power Member
Posts: 6450
Joined: 2003-02-06, 17:36 UTC
Location: Germany

Re: "Compare by Content" takes a few seconds when 100% CPU usage

Post by *Horst.Epp »

menaquinone wrote: 2020-06-08, 20:06 UTC hmm. If I deselect "Ignore repeated spaces" then the delay is gone and the Comparison is instant ..
I tested it with your files and there is no delay at all whatever I set for "Ignore repeated spaces"
Windows 11 Home x64 Version 23H2 (OS Build 22631.3374)
TC 11.03 x64 / x86
Everything 1.5.0.1371a (x64), Everything Toolbar 1.3.2, Listary Pro 6.3.0.69
QAP 11.6.3.2 x64
menaquinone
Junior Member
Junior Member
Posts: 6
Joined: 2020-06-08, 05:08 UTC

Re: "Compare by Content" takes a few seconds when 100% CPU usage

Post by *menaquinone »

How interesting! My guess is that Dalai and I being on Win 7 64bit experience this issue, but Horst.Epp being on Windows 10 x64 does not.

Thank you both for your time!

I looked at one of the reported stacktraces during the long delay, via Process Explorer(of Sysinternals), and I can see what looks like what I suspected was the culprit in the first place: that give_CPU_time_slice in the form of ntdll.dll!NtDelayExecution+0xa via KERNELBASE.dll!SleepEx+0xb3 (see: https://forum.powerbasic.com/forum/user-to-user-discussions/source-code/56824-ntdelayexecution-sleep-sleepex-with-higher-resolution-absolute-delay#post56824 for explanations)

0 ntoskrnl.exe!KiCpuId+0xaa
1 ntoskrnl.exe!CcGetFlushedValidData+0x301
2 ntoskrnl.exe!KiCpuId+0x43b1
3 ntoskrnl.exe!KeInsertQueueApc+0x5cb
4 ntoskrnl.exe!KiCpuId+0x2549
5 ntoskrnl.exe!longjmp+0x5a78
6 ntdll.dll!NtDelayExecution+0xa
7 KERNELBASE.dll!SleepEx+0xb3
8 TOTALCMD64.EXE+0x74b93
9 0x0000000000000000

Dalai wrote: 2020-06-08, 20:24 UTC When switching one side to UTF-8 and the other to ANSI, it's slow again even when "Ignore repeated spaces" is disabled.
that now takes 19 seconds to complete the comparison with those settings, and the stacktraces are thus easily seen.

0 ntoskrnl.exe!KiCpuId+0xaa
1 ntoskrnl.exe!CcGetFlushedValidData+0x301
2 ntoskrnl.exe!KiCpuId+0x43b1
3 ntoskrnl.exe!KeInsertQueueApc+0x5cb
4 ntoskrnl.exe!KiCpuId+0x2549
5 ntoskrnl.exe!RtlAreBitsClear+0x1545
6 ntoskrnl.exe!KeDelayExecutionThread+0x1cf
7 ntoskrnl.exe!MmLockPagableSectionByHandle+0x9c1
8 ntoskrnl.exe!longjmp+0x5bf3
9 ntdll.dll!NtDelayExecution+0xa
10 KERNELBASE.dll!SleepEx+0xb3
11 TOTALCMD64.EXE+0x74b93
12 0x0000000000000000

Image: https://user-images.githubusercontent.com/66539662/84083255-cb75ad00-a9d0-11ea-874c-125616e92b41.jpg

Oh well, I hope it gets fixed somehow, sometime...
Post Reply