Bad search term lists every scanned object
Moderators: petermad, Stefan2, white, Hacker
Bad search term lists every scanned object
Hi,
I just came across this issue with v11.51RC3, which hopefully was not reported before, and I believe it should be fixed.
This issue may be present in earlier versions as well, but currently I do not have have to time to perform extensive testing in regard.
If a bad search term is submitted as per the example, the search result will display every object that was scanned despite none of them being relevant.
https://ibb.co/svPCJRR5
If the URL is not accessible for any reason, try to search for example.dll" (including the closing quotes without any opening quotes) in any non-empty folder and observe that every object of that folder is going to be listed in the search result.
A search performed like this can be a result of improper copy-pasting - a user mistake.
If another search is submitted without containing a dot character, no results will be displayed (as expected):
https://ibb.co/Bpr8TyK
This time the search was performed against exampledll"
I believe in both cases the acceptable behavior is to have no results reported.
I just came across this issue with v11.51RC3, which hopefully was not reported before, and I believe it should be fixed.
This issue may be present in earlier versions as well, but currently I do not have have to time to perform extensive testing in regard.
If a bad search term is submitted as per the example, the search result will display every object that was scanned despite none of them being relevant.
https://ibb.co/svPCJRR5
If the URL is not accessible for any reason, try to search for example.dll" (including the closing quotes without any opening quotes) in any non-empty folder and observe that every object of that folder is going to be listed in the search result.
A search performed like this can be a result of improper copy-pasting - a user mistake.
If another search is submitted without containing a dot character, no results will be displayed (as expected):
https://ibb.co/Bpr8TyK
This time the search was performed against exampledll"
I believe in both cases the acceptable behavior is to have no results reported.
- ghisler(Author)
- Site Admin
- Posts: 50105
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Re: Bad search term lists every scanned object
Currently wildcards are broken into parts like this:
1. If there are spaces, break at the next space if there isn't a quote character before it
2. If there is a quote character, search for the second quote and use the string within
So in your case, TC sees the quote and ignores the text before it. Since there is no second, closing quote, it uses the string until the end, which is an empty string. Therefore you get all files because an empty search string matches everything.
For now, I will return just the quote character if the text is empty, so nothing will be found.
I could try to interpret this quote as a closing quote instead, searching for example.dll. However, what should then happen when the quote character is in the middle of the string, e.g. example"dll ?
1. If there are spaces, break at the next space if there isn't a quote character before it
2. If there is a quote character, search for the second quote and use the string within
So in your case, TC sees the quote and ignores the text before it. Since there is no second, closing quote, it uses the string until the end, which is an empty string. Therefore you get all files because an empty search string matches everything.
For now, I will return just the quote character if the text is empty, so nothing will be found.
I could try to interpret this quote as a closing quote instead, searching for example.dll. However, what should then happen when the quote character is in the middle of the string, e.g. example"dll ?
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
Re: Bad search term lists every scanned object
The search syntax is rather complex and the search mode changes when the search term contains dot. See the explanation here:tempdrive wrote: 2025-02-04, 04:17 UTC If another search is submitted without containing a dot character, no results will be displayed (as expected):
https://www.ghisler.ch/wiki/index.php?title=Search
Re: Bad search term lists every scanned object
Thank you for the responses.
The syntax is fine in general, it was more about not accepting invalid ones.
No result is definitely more desired than all the returned values, however, if nothing is returned silently, the user may not understand the reason.
Since it already has been established that quotes are expected in pairs, how about a small pop up stating "Syntax error: missing closing quote" in a style when you get a pop up when clicking on Connect in "Connect to ftp server" when there is no entry specified?
Or if there is a message that you can show on-the-fly hovering over the search input, that would be even better, but I am not sure if I have seen anything like that in this program.
Given that on Windows systems it is not allowed to have quotes as part of the filename anyway, having only one in the middle of the user-entered syntax still would not make sense as per example"dll either if there is no other (opening or closing) quote present anywhere else.
I think the search should rather not be executed to begin with instead getting unexpected results, even if it is a clear mistake on user's end.
Some users may even have difficulties interpreting the syntax rules associated with it in general.
The syntax is fine in general, it was more about not accepting invalid ones.
No result is definitely more desired than all the returned values, however, if nothing is returned silently, the user may not understand the reason.
Since it already has been established that quotes are expected in pairs, how about a small pop up stating "Syntax error: missing closing quote" in a style when you get a pop up when clicking on Connect in "Connect to ftp server" when there is no entry specified?
Or if there is a message that you can show on-the-fly hovering over the search input, that would be even better, but I am not sure if I have seen anything like that in this program.
Given that on Windows systems it is not allowed to have quotes as part of the filename anyway, having only one in the middle of the user-entered syntax still would not make sense as per example"dll either if there is no other (opening or closing) quote present anywhere else.
I think the search should rather not be executed to begin with instead getting unexpected results, even if it is a clear mistake on user's end.
Some users may even have difficulties interpreting the syntax rules associated with it in general.
- ghisler(Author)
- Site Admin
- Posts: 50105
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Re: Bad search term lists every scanned object
I would prefer not to show an error, there may be lot of stored searches where the user forgot the closing double quote which work perfectly fine, e.g. searching for
"Program files
"Program files
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
Re: Bad search term lists every scanned object
It's fine.
I see now it mimics the behavior of Explorer.
Generally speaking I was more concerned that unpaired quotes can result in various issues for different types of applications and how they are being handled by the user.
Fixing the original issue as reported for example.dll" by displaying no results is good enough for this scenario.
Thank you.
I see now it mimics the behavior of Explorer.
Generally speaking I was more concerned that unpaired quotes can result in various issues for different types of applications and how they are being handled by the user.
Fixing the original issue as reported for example.dll" by displaying no results is good enough for this scenario.
Thank you.
Re: Bad search term lists every scanned object
If I use the examples given by tempdrive, I get the same results. But when I use Everything checkbox, checked, then I get no results in both examples.
Greetings,
Rob
Rob
- ghisler(Author)
- Site Admin
- Posts: 50105
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Re: Bad search term lists every scanned object
This should be fixed in RC4, please test it!
Please note that when the users enters e.g. file"name.ext Total Commander will continue to ignore the text before the quote character.04.02.25 Fixed: Find files: Find nothing when user sets invalid search string filename.ext" (32/64)
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
Re: Bad search term lists every scanned object
Something doesn't match the usage example and the logic.will continue to ignore the text before the quote character.
If you write that you simply discard EVERYTHING that was before the first quote - then you are working with what REMAINS after it? Right?
Then it turns out that the search string /file"name/ should find any object containing in its f.name /name/ substring. Isn't that so?
But the search practice shows that a more accurate response is returned - nothing found.
P.S.
And yet, the more incomprehensible fact is that we seem to know for sure that this symbol is not allowed to be part of a file system object name on Windows - but we still allow this symbol to be entered? Only because there was initially no check for the validity of the presence of such symbols?
#146217 personal license
Re: Bad search term lists every scanned object
2white
in case the user mistakenly entered a "somehow allowed" character 'double quote' only once.
And it seems that the inadmissibility of entering such a character is not clarified on the help page and no example of behavior is givenThe search syntax is rather complex and the search mode changes when the search term contains dot. See the explanation here:
https://www.ghisler.ch/wiki/index.php?title=Search
in case the user mistakenly entered a "somehow allowed" character 'double quote' only once.
#146217 personal license
Re: Bad search term lists every scanned object
ghisler(Author) wrote: 2025-02-05, 11:31 UTCPlease note that when the users enters e.g. file"name Total Commander will continue to ignore the text before the quote character.04.02.25 Fixed: Find files: Find nothing when user sets invalid search string filename" (32/64)
Both the description of the fix and Ghisler's statement contain bad examples, because those examples concern a different search mode. The examples should include a dot. So for example this would have been better:
04.02.25 Fixed: Find files: Find nothing when user sets invalid search string filename.ext" (32/64)
I assume that the behavior last described by Ghisler is for compatibility reasons.Please note that when the users enters e.g. filename.ext"text Total Commander will continue to ignore the text before the quote character.
- ghisler(Author)
- Site Admin
- Posts: 50105
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Re: Bad search term lists every scanned object
Sorry, with filename I implied a full name with extension. I have updated the online history1151.txt and my post above, please check.
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
Re: Bad search term lists every scanned object
Code: Select all
File name Size
------------------------------ -
D:\--only files--\
D:\--only files--\name 0
D:\--only files--\filename 0
D:\--only files--\file 0
D:\--only files--\name.ext 0
D:\==only folders==\
D:\==only folders==\file\
D:\==only folders==\filename\
D:\==only folders==\name\
D:\==only folders==\name.ext\
total files 4 total size 0
total folders 4
Now open File Search dialog and start search for: file"name
Result: NULL
Now let's search for: file"name.ext
Result: found 1 file and 1 folder:
[D:\==only folders==\name.ext]
D:\--only files--\name.ext
And I don't understand the essence of the logic hardcoded into the code, which doesn't find anything for the first search,
BUT for the second one, it suddenly finds "something": both the file and the folder...
IF it is stated that in the presence of the first oncoming character - "double quotes" there is a discarding of everything BEFORE them -
then EVERYTHING that reaches EITHER the second character of the "double quotes" OR the end of the line is searched AND used.
This means that in the first search for this process, a piece of /name/ should have been taken - which is reflected in the objects
of the file structure. That is, we HAD TO find in exactly the same way: 1 file and 1 folder:
[D:\==only folders==\name]
D:\--only files--\name
The presence of the DOT symbol should not change this logic, but only expand the boundaries of the search and matches...
#146217 personal license
- ghisler(Author)
- Site Admin
- Posts: 50105
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Re: Bad search term lists every scanned object
Without the dot, TC looks whether the text is present anywhere in the name, while with the dot it splits the search string into tokens and then tries to match them. Tokens are split at spaces, except when there are double quotes, which are needed to find names with spaces. I want tho change as little as possible in this logic, because there are millions of saved searches, where some of them would be broken by such a change.
So the only thing changed in RC4 is the handling of a single " at the end of the search string containing a dot.
So the only thing changed in RC4 is the handling of a single " at the end of the search string containing a dot.
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
Re: Bad search term lists every scanned object
Confirmed.ghisler(Author) wrote: 2025-02-06, 08:24 UTC Sorry, with filename I implied a full name with extension. I have updated the online history1151.txt and my post above, please check.
Since the dot is included now in the descriptions by Ghisler, it no longer suggests to apply to all search modes.AntonyD wrote: 2025-02-06, 09:11 UTC IF it is stated that in the presence of the first oncoming character - "double quotes" there is a discarding of everything BEFORE them -
then EVERYTHING that reaches EITHER the second character of the "double quotes" OR the end of the line is searched AND used.