solved [TC 10.50] Unexpected search result for list search with wildcards

[petermad]

Since we can't come to an agreement here, I will move this thread to "will not be changed" soon.

I will mark this topic as Solved, then - since the original problem wiyh wildcards IS solved.

[white]

The idea was that you could load a list of files and folders into the program via the search function, so they only get added when they exist. I will probably break a lot of saved searches if I changed that now. In the case of c:\path\*.ext it was logical to change it because the result was definitely wrong.

But now search depth is shifted for some entries in a search list and not for others. This is wrong too.

Consider searching using a search list in a file (@c:\test\list.txt).
Note that the option "search in subdirectories" does not apply to files in the list, but only to folders and to entries that contain a wildcard expression or regular expression.
Now in TC version 10.51, when the expressions are searched with search depth X, the folders are searched with search depth X-1.

An illustration:

Create the following files (with any content) and folder structure:

c:\test\Folder A\file.txt
c:\test\Folder A\Level 1\file.txt
c:\test\Folder A\Level 1\Level 2\file.txt
c:\test\Folder B\file.txt
c:\test\Folder B\Level 1\file.txt
c:\test\Folder B\Level 1\Level 2\file.txt

Also create the file "c:\test\list.txt" with contents:

c:\test\Folder A\
c:\test\Folder B\*.txt

Open Search (Alt+F7) and set the following options:

Code: Select all

Search for:    *.txt
Search in:     @c:\test\list.txt

Now examen the results of the search with different search depths:

Searching with "Search in subdirectories" set to "current dir only" (search depth 0)

• The "Search for" field is ignored and Folder A is not searched, instead the result is simply the folder, if it exists. (This is not obvious.)
• Folder B is searched at base level (search depth 0)
• The results are:

  [1 files and 1 directories found]
  [c:\test\Folder A]
  c:\test\Folder B\file.txt

Searching with "Search in subdirectories" set to "1 level"

• Folder A is searched with search depth 0
• Folder B is searched with search depth 1
• The results are:

  [3 files and 0 directories found]
  c:\test\Folder A\file.txt
  c:\test\Folder B\file.txt
  c:\test\Folder B\Level 1\file.txt

Searching with "Search in subdirectories" set to "2 levels"

• Folder A is searched with search depth 1
• Folder B is searched with search depth 2
• The results are:

  [5 files and 0 directories found]
  c:\test\Folder A\file.txt
  c:\test\Folder A\Level 1\file.txt
  c:\test\Folder B\file.txt
  c:\test\Folder B\Level 1\file.txt
  c:\test\Folder B\Level 1\Level 2\file.txt

---

But in the other cases it's really debatable which one is right and which one is wrong.

You say this is really debatable. I gave you my arguments why this is wrong (the misuse of "current dir only" option which isn't obvious, search depth is shifted for some entries and not for others).
What are your arguments why this is right?

Therefore I will not change it at this time, sorry.

You should at least explain the behavior in the help. Which is quite hard and complex.

[ghisler(Author)]

But now search depth is shifted for some entries in a search list and not for others. This is wrong too.

Well, the previous result for c:\path\wildcards was clearly wrong, so no harm should be done by changing that.

What are your arguments why this is right?

There is a clear difference between
c:\path
and
c:\path\*.extension

In the first case, finding just c:\path would be the lowest possible level.
In the second case, it clearly tells us that we want to find the files *.extension inside c:\path, so that would be the lowest possible level.

Also there would be no way to find just the files and folders themselves in the list file if I changed it.

[white]

But now search depth is shifted for some entries in a search list and not for others. This is wrong too.

Well, the previous result for c:\path\wildcards was clearly wrong, so no harm should be done by changing that.

It wasn't that clearly wrong if people used for example level 1 or 2. Any saved searches like that are now broken.
And as I explained search depth is now out of sync with other entries.

What are your arguments why this is right?

There is a clear difference between
c:\path
and
c:\path\*.extension

In the first case, finding just c:\path would be the lowest possible level.
In the second case, it clearly tells us that we want to find the files *.extension inside c:\path, so that would be the lowest possible level.

That's not true. If finding just c:\path is the lowest possible level then searching with higher search depths would find additional results. Searching with level 1 would result in:

c:\path
c:\path\file1.txt
c:\path\file2.txt
c:\path\program.exe
...

That is not the case. Searching at level 1 currently searches for whatever is in the Search for field and searches for that inside c:\path at the lowest possible level. And searching with level 2 and up finds additional results.

Finding just c:\path is not the lowest possible level. It's an entirely different search.

Also there would be no way to find just the files and folders themselves in the list file if I changed it.

A possible solution to this is to add a new "file/folder list only" item to the "Search in subdirectories" drop-down list for exactly this purpose.

The downside of this is:
Incompatibility with saved searches which search using a file list and a limited search level.

The upside of this is:
It is easier to understand and possible to explain well in the help. Currently the functionality is there, but quite incomprehensible.

[ghisler(Author)]

It wasn't that clearly wrong if people used for example level 1 or 2. Any saved searches like that are now broken.

The only way to prevent that would be to revert to the behaviour in TC 10.50 and older, with one important change:
Searching for c:\path\*.wildcards with "current dir only" would then find nothing at all.

This way the stored searches with level 1 or 2 would remain unchanged, and level 0 would return nothing for this specific search. Of course this would have to be explained in the help.

[white]

It wasn't that clearly wrong if people used for example level 1 or 2. Any saved searches like that are now broken.

The only way to prevent that would be to revert to the behaviour in TC 10.50 and older, with one important change:
Searching for c:\path\*.wildcards with "current dir only" would then find nothing at all.

This way the stored searches with level 1 or 2 would remain unchanged, and level 0 would return nothing for this specific search. Of course this would have to be explained in the help.

Correct, I had thought about that solution too and doubted to mention it. It solves the compatibility issue and becomes easier to explain in help.

The downside remains that it remains hard to understand and confusing for the user that when searching in search results or in a list in a file, the search depths options are shifted and searching at the lowest level means something else.

But for now this may be the best solution since you are now working on a minor TC update.

[ghisler(Author)]

OK, I will change it like that in RC3.

[Usher]

Searching for c:\path\*.wildcards with "current dir only" would then find nothing at all.

Could you issue warning message for such cases? For example "You haven't allowed search in subdirs".

[ghisler(Author)]

I have also thought about this. I will show the following as the first result:
[No files found] WARNING! "Current dir only" + "*.dll"
with *.dll the first wildcards found in the list file.

Here is the description I will put in the help:

Important: If the search depth is limited to "Current directory only", only the files and directories in the list file will be found, no files inside of the listed directories and no wildcards. With a depth of 1 level, the specified directories will be searched, with or without wildcards. With 2 levels, sub-directories found within the specified directories will be searched, etc.

What do you think?

[petermad]

Here is the description I will put in the help:

Important: If the search depth is limited to "Current directory only", only the files and directories in the list file will be found, no files inside of the listed directories and no wildcards. With a depth of 1 level, the specified directories will be searched, with or without wildcards. With 2 levels, sub-directories found within the specified directories will be searched, etc.

What do you think?

That sound good to me.

[white]

Here is the description I will put in the help:

Important: If the search depth is limited to "Current directory only", only the files and directories in the list file will be found, no files inside of the listed directories and no wildcards. With a depth of 1 level, the specified directories will be searched, with or without wildcards. With 2 levels, sub-directories found within the specified directories will be searched, etc.

I assume you want to add that to the Help about "Search in"?
How about:

Search in:

Here you can enter one or more starting directories, separated by ";". You can also start the search inside an archive.
Put '>' character in this field before saving the search to keep the "Search in" field unchanged when loading the saved search.
You can enter the name of a text file (ANSI, UTF-8 or UTF-16, the latter two with BOM) containing a list of files and directories to be searched, in the form @c:\path\filelist.txt. The file must contain one file or directory name per line, either an absolute path, or relative to the location of the list file. Wildcards and regular expressions are also supported, but not in paths. Use the character '<' to indicate the start of a regular expression.
Important: If the search depth is limited to "Current directory only", only the files and directories in the list file will be found. The "Search for" field is ignored and lines containing wildcards or regular expressions will not be searched. With a search depth of 1 or higher, the "Search for" field is not ignored and inside the specified directories will be the base level for the search.

Sample list:	Description
c:\testdir\filename.txt	Searches just for "filename.txt" in "c:\testdir" (search depth has no effect)
c:\testdir\	Searches inside "c:\testdir" for search string defined in "Search for" (search depth 1 or higher) Only searches for "c:\testdir" itself, ignores search string defined in "Search for" (search current dir only)
c:\testdir\*.txt *.doc	Searches "c:\testdir" for txt and doc files (search depth 1 or higher)
c:\testdir\<regular expression	Searches "c:\testdir" using regular expression (search depth 1 or higher)

F2 Search in found files/folders

If checked, the next search will be performed only within the displayed search results.
Important: If the search depth is limited to "Current directory only", only the list itself is searched, so the result is a subset of the current search results. With a search depth of 1 or higher, directories listed in the search results will be searched within, so these directories themselves will not be in the results.

[petermad]

22.08.22 Fixed: Search, use list file in "Search in", e.g. @c:\totalcmd\list.txt: Ignore lines containing wildcards like *.dll when search depth is set to "current dir only" (32/64)

10.08.22 Fixed: NOTE: THIS WAS CHANGED BACK IN RC3: Search, use list file in "Search in", e.g. @c:\totalcmd\list.txt containing wildcards like *.dll: "Search in subdirectories" was off by 1, e.g. "1 levels" only searched the base directory (32/64)

Based on the history this should be moved to "TC Behaviour which will not be changed"

[white]

2ghisler(Author)

Did you consider my suggestions for the help?

Based on the history this should be moved to "TC Behaviour which will not be changed"

Well, the behavior did change. And an explanation was added to the help. Don't you consider it fixed?

[ghisler(Author)]

2white
I used some of your suggestions for the help.

[petermad]

2white

Well, the behavior did change. And an explanation was added to the help. Don't you consider it fixed?

You are right - the weird [C:\totalcmd\} results from my first post is no longer there, and the help explains the behaviour - so: confirmed fixed