?[8.50B4] Background search goes unresponsive

[Sagamal]

"Search in separate Process" through a few hundred "large" (25Mbyte) HTML files using "Find Text" + "challenging" regex e.g. .*text.*
Background search goes unresponsive every time.
"Simple" searches without regex or very simple regex appear to work OK.
Same challenging search works OK as foreground search.
Win7-64

[ghisler(Author)]

Unfortunately I do not have such large html files, so I tried with a few videos. The search dialog never became unresponsive.

1. Can you try the same with the internal search function Alt+F7? It uses the exact same functions as the background search.

2. It may be a virus scanner which blocks the program for several seconds to scan the file.

[Sagamal]

1) I tried again with Alt+F7 and the foreground search also went non-responsive - this does not look like a background process issue.

2) Process explorer shows no obvious scanner activity. totalcmd.64 is red-lined, everything else looks normal.

[Sagamal]

BTW: SlickEdit also did not handle the test case very well.
EditPlus was not shaken a bit by it.

[ghisler(Author)]

Can you check with "process monitor" from www.sysinternals.com to see what TC is doing while you think that it hangs?

[gdpr deleted 6]

@Sagamal, on what kind of drive do you have the files stored on (local/network/DVD)?

I did run a test according to your description on my local HDD, using 800 specifically created test files with a total size of approx. 20 GB. Although the search took ages to complete, the search window remained responsive and i could click the Stop or Cancel buttons at any time.

[ghisler(Author)]

Can anyone else reproduce this problem?

[gdpr deleted 6]

I don't know what other search options Sagamal used beside RegEx. By coincidence, i found a way to observe the described unresponsiveness, alas it is not the procedure as described by the OP.
I want to report it here; since it might perhaps be related to Sagamal's problem.

Having two very large identical files (approx. 1GB file size and identical on binary level) in the same folder and doing a search with "Find duplicate files", "Same size" and "Same contents" enabled (all other search options as defined by a fresh/empty wincmd.ini -- that means, no regex search), TC and its search window become unresponsive for the time TC doing the actual binary comparison of the two files. Interestingly, switching on RegEx text search additionally to the aforementioned search options prevents the search window from becoming unresponsive (probably because using different I/O routines). Tested with TC8.50b7 x64.

@Sagamal:
Aside from the RegEx option, did you have any other options or plug-ins active for your search?

[ghisler(Author)]

The search you describe is done in a foreground process and doesn't handle user input, so it may indeed cause short periods where TC is unresponsive. But I haven't seen this in the normal search function so far...

[Sagamal]

BTW: Problem persists in B7.

"Aside from the RegEx option, did you have any other options or plug-ins active for your search?" - No

"what kind of drive do you have the files stored on (local/network/DVD)? "
Manufacturer Western Digital
Model WDC WD15EARS-00Z5B1
Size 1500.3 GB
Firmware Version 80.00A80
Serial Number WD-WMAVU1594560
Rotational Speed N/A
Form Factor
Interface Serial ATA
Standard ATA8-ACS | ----
Advanced Format Supported N/A
Transfer Mode (Current / Max) SATA-300 / SATA-300
Features S.M.A.R.T., 48bit LBA, NCQ, AAM (Disabled)
Power Cycle Count 2144
Temperature 30 C (86 F)
Drive Letter(s) D: E: W:
Queue Depth 32
Removable No
Cache Enabled (Read / Write) Yes / Yes
SMART Support Yes

"Can you check with "process monitor" from www.sysinternals.com to see what TC is doing while you think that it hangs" - I'm not sure how to best do that for you.
ntoskrnl.exe!KeWaitForMultipleObjects+0xc0a
ntoskrnl.exe!KeAcquireSpinLockAtDpcLevel+0x732
ntoskrnl.exe!KeWaitForSingleObject+0x19f
ntoskrnl.exe!_misaligned_access+0xba4
ntoskrnl.exe!_misaligned_access+0x1821
TOTALCMD64.EXE+0xa04b
TOTALCMD64.EXE+0x4e7a1d
I have a dump, but how to upload?

[ghisler(Author)]

I'm sorry but I still can't reproduce it. Does anyone have a sample file which causes this problem?

[ghisler(Author)]

I have added some message loop calls to the regex search function now. Maybe it helps, maybe not...

[zaffi]

these are the settings I used;
Image: http://i57.tinypic.com/e7i3qg.jpg

total commander main window was not blocked (alt+shift+F7);
but the search window was blocked.

i'm using tc8.50 x64 (NOT the beta!)
on win7 x64


BTW:
to get hundreds of HTML files, either

1) run wget.exe on a big website, e.g.

Code: Select all

wget -mc -e robots=off --no-check-certificate  --save-cookies=temp-cookies.txt --user-age
nt="Mozilla/5.0 (Windows NT 5.2; rv:2.0.1) Gecko/20100101 Firefox/24.0" www.government.se

2) or use my python script:

Code: Select all

import os, random
def word(start=65):
	return ''.join([chr(random.randint(start,122)) for x in range(random.randint(5,15))])
def text():
	return ' '.join([word() for x in range(random.randint(500,5000))])
def create_file(num):
	d=str(int(num*0.001))
	if not os.path.isdir(d):
		os.makedirs(d)
	open(os.path.join(d, word(97)),'w').write(text())
def create_random_files():
	[create_file(x) for x in range(4000)]
os.makedirs('lots-of-files')
os.chdir('lots-of-files')
create_random_files()

it WILL block your UI

[ghisler(Author)]

.*text.* calls for trouble because lines in html can be VERY long (entire paragraphs). I assume that it's the RegEx libary which is simply busy for several seconds in some paragraphs.

Why do you use regex here anyway? A simple search for
text
without regex would be much better here.

[zaffi]

[quote="ghisler(Author)"].*text.* calls for trouble.[/quote]

you're totally correct of course,
i'm just trying to follow the original poster's steps;