Indexed search using WDX plugins

Here you can propose new features, make suggestions etc.

Moderators: white, Hacker, petermad, Stefan2

User avatar
Hacker
Moderator
Moderator
Posts: 13052
Joined: 2003-02-06, 14:56 UTC
Location: Bratislava, Slovakia

Indexed search using WDX plugins

Post by *Hacker »

Hi Christian and all,
I would like to suggest for TC to create its own indexed search using WDX plugins. This way it would be possible to instantly search anything that would take quite a lot of time now when using plugins. Imagine you could instantly find that long exposure nightshot of the sky from the summer of 2010 made by your old Fuji F30 which you know you have somewhere but can't wait a few hours until all your drives are scanned.
Of course, the question would be how this database would be updated (on demand? in the background? while copying files?).

Ideas and suggestions are welcome, just please stick to the original idea and don't change it too much. :)

Roman
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.
User avatar
Lefteous
Power Member
Power Member
Posts: 9535
Joined: 2003-02-09, 01:18 UTC
Location: Germany
Contact:

Post by *Lefteous »

I support the idea.
Of course, the question would be how this database would be updated (on demand? in the background? while copying files?).
Having a background service in addition to Windows search could be a performance problem. Disabling Windows search index is problematic in my experience.
I would probably stick with the background update as most PCs might be fast enough.
User avatar
Hacker
Moderator
Moderator
Posts: 13052
Joined: 2003-02-06, 14:56 UTC
Location: Bratislava, Slovakia

Post by *Hacker »

Also, the data extracted by WDX plugins such as movie codec info, MP3 tags, EXIF data, etc. would be cached this way so it could be displayed much faster when reading a folder.

Roman
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.
User avatar
MVV
Power Member
Power Member
Posts: 8702
Joined: 2008-08-03, 12:51 UTC
Location: Russian Federation

Post by *MVV »

BTW I think such thing may be done via additional proxy WDX plugin with cache function. So any user would chose some fields to show directly and other ones to show via caching proxy.

But indexed search would be useful for people who often search by WDX fields.
User avatar
Hacker
Moderator
Moderator
Posts: 13052
Joined: 2003-02-06, 14:56 UTC
Location: Bratislava, Slovakia

Post by *Hacker »

MVV,
But indexed search would be useful for people who often search by WDX fields.
General filesystem data should be indexed, too, like file names, time stamps, etc. And including a NTFS MFT search in TC would allow to combine the best features of both Locate32 and Everything.

Roman
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.
User avatar
MVV
Power Member
Power Member
Posts: 8702
Joined: 2008-08-03, 12:51 UTC
Location: Russian Federation

Post by *MVV »

I think MFT search should be included just as separate module so it would be easy to call Locate32/Everything from TC, I don't think we need to write this code from scratch.
User avatar
Lefteous
Power Member
Power Member
Posts: 9535
Joined: 2003-02-09, 01:18 UTC
Location: Germany
Contact:

Post by *Lefteous »

2MVV
Sources for Locate32 are available on the authors homepage.
User avatar
MVV
Power Member
Power Member
Posts: 8702
Joined: 2008-08-03, 12:51 UTC
Location: Russian Federation

Post by *MVV »

Anyway we need service part and user part, and someone should support it. It is much easier to create/support simple proxy plugin instead.
User avatar
Hacker
Moderator
Moderator
Posts: 13052
Joined: 2003-02-06, 14:56 UTC
Location: Bratislava, Slovakia

Post by *Hacker »

Also, right now I would have made use of instant search through all comments for all files on my hard drive. Caching / indexing descript.ion files would have made this a matter of seconds instead of minutes.

Roman
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.
User avatar
Hacker
Moderator
Moderator
Posts: 13052
Joined: 2003-02-06, 14:56 UTC
Location: Bratislava, Slovakia

Post by *Hacker »

I'd like to bring this one up again.

I was looking for a program that could index and search all types of files, e.g. I'd like it to be able to search for text in documents, EXIF metadata values of JPG files, MP3 tags, etc.
For documents I found DocFetcher, Agent Ransack / File Locator Pro and Lookeen, but these only work on documents (PDF files, Office files, emails, etc.) and even though DocFetcher indexes EXIF values, it is not able to search for them (and development is limited to bugfixes only). For images I would have to use some Digital Asset Management program like Adobe Lightroom or ACD Systems ACDSee. For MP3 / FLAC files yet another program.

Which brings me back to TC. TC can already extract all the necessary information using WDX plugins. All we'd need is a database to store the indexed values and a way to update the index. And we already have the search interface as well. All it needs is a checkbox "Use database".

Please consider this feature request.

Thanks
Roman
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.
User avatar
Horst.Epp
Power Member
Power Member
Posts: 6450
Joined: 2003-02-06, 17:36 UTC
Location: Germany

Post by *Horst.Epp »

Hacker wrote:I'd like to bring this one up again.

I was looking for a program that could index and search all types of files, e.g. I'd like it to be able to search for text in documents, EXIF metadata values of JPG files, MP3 tags, etc.
For documents I found DocFetcher, Agent Ransack / File Locator Pro and Lookeen, but these only work on documents (PDF files, Office files, emails, etc.) and even though DocFetcher indexes EXIF values, it is not able to search for them (and development is limited to bugfixes only). For images I would have to use some Digital Asset Management program like Adobe Lightroom or ACD Systems ACDSee. For MP3 / FLAC files yet another program.

Which brings me back to TC. TC can already extract all the necessary information using WDX plugins. All we'd need is a database to store the indexed values and a way to update the index. And we already have the search interface as well. All it needs is a checkbox "Use database".

Please consider this feature request.

Thanks
Roman
But the Windows index already has all of this information
and you can search for it.
Also there is a way to use the index from an application as can be seen
in the Ultimate version of xplorer².
I think this would be a much better way because IFilters exists for many formats and there is no need to invent that all again in TC with Wdx plugins.
Windows 11 Home x64 Version 23H2 (OS Build 22631.3374)
TC 11.03 x64 / x86
Everything 1.5.0.1371a (x64), Everything Toolbar 1.3.2, Listary Pro 6.3.0.69
QAP 11.6.3.2 x64
User avatar
Hacker
Moderator
Moderator
Posts: 13052
Joined: 2003-02-06, 14:56 UTC
Location: Bratislava, Slovakia

Post by *Hacker »

Horst.Epp,
But the Windows index already has all of this information
and you can search for it.
OK, I have now spent 20 minutes googling for how to do that but came up empty. How can I find files that fulfill all of these criteria:
- *.jpg
- ISO >= 400
- Modified date between 1.7.2010 and 31.8.2010
- Descript.ion for the file contains "*****"

Or is this only possible with xplorer²? Or do I need to install any IFilters? What file types are there IFilters for? Is there a list? Are there IFilters for more file types than there are WDX plugins for?
I'd appreciate any pointers.

TIA
Roman
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.
User avatar
tuska
Power Member
Power Member
Posts: 3741
Joined: 2007-05-21, 12:17 UTC

Post by *tuska »

Hacker wrote:... How can I find files that fulfill all of these criteria:
- *.jpg
- ISO >= 400
- Modified date between 1.7.2010 and 31.8.2010
- Descript.ion for the file contains "*****"
I think this is possible with a combination of TC and Everything.

Search for: ed: dm:01-07-2010-31-08-2010 jpg -> Everything ticked
Plugins
tc | comment | contains | *****
shelldetails | ISO-Filmempfindlichkeit | >= | 400
User avatar
Hacker
Moderator
Moderator
Posts: 13052
Joined: 2003-02-06, 14:56 UTC
Location: Bratislava, Slovakia

Post by *Hacker »

tuska,
Thank you but in your suggestion the EXIF values are not indexed so if I have 1 TB of *.jpg from those two months TC has to look through all of their EXIF's. If they were indexed it would only take a second.

Roman
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.
User avatar
Horst.Epp
Power Member
Power Member
Posts: 6450
Joined: 2003-02-06, 17:36 UTC
Location: Germany

Post by *Horst.Epp »

Hacker wrote:Horst.Epp,
But the Windows index already has all of this information
and you can search for it.
OK, I have now spent 20 minutes googling for how to do that but came up empty. How can I find files that fulfill all of these criteria:
- *.jpg
- ISO >= 400
- Modified date between 1.7.2010 and 31.8.2010
- Descript.ion for the file contains "*****"

Or is this only possible with xplorer²? Or do I need to install any IFilters? What file types are there IFilters for? Is there a list? Are there IFilters for more file types than there are WDX plugins for?
I'd appreciate any pointers.

TIA
Roman
The Wiki for IFilters contains some external links for several formats.
https://en.wikipedia.org/wiki/IFilter
MSDN explanation for search syntax
https://msdn.microsoft.com/en-us/library/windows/desktop/aa965711%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396
A list of installed search filters with a Nirsoft tool
http://www.nirsoft.net/utils/search_filter_view.html
Windows 11 Home x64 Version 23H2 (OS Build 22631.3374)
TC 11.03 x64 / x86
Everything 1.5.0.1371a (x64), Everything Toolbar 1.3.2, Listary Pro 6.3.0.69
QAP 11.6.3.2 x64
Post Reply