[BUG] Help. What are limits of RegEx?

[MarkFilipak]

Probably a bit OffTopic, but why don't you use more or less "dedicated" tools for this? E.g. VobEdit can show/interpret the sequence extension quite clearly. Just open the first vob file of the (main DVD) video stream and navigate to the first video pack / I frame pack.

I use VobEdit. It's even on my Windows Start Menu I use it that much. However, it provides just a fraction of the MPEG PES and misses some of the most important metadata. I'm writing a full parser.
I use IfoEdit, too, and have written javascript that extracts accurate # of sectors for IFOs that contain more than one sequence.

AFAIR dgindex can show the basic stream information (level@profile)as well, like probably a lot of other tools.

That site provides just a sketch, and it's full of blank spots and outright errors.

And back to topic: TC's limitations when it comes to splitting file content on newlines were the main reason why I started PCREsearch plugin. Your RegEx will probably work with it.

Thanks! I'll check it out.

[MarkFilipak]

Hi Karl,

I don't know what Everything is.

As of Total Commander 9.0, the tool 'Everything' can be integrated into a search query with its own search parameters.

In my signature above there are links to this tool and a documentation to help.

I looked here: viewtopic.php?f=3&t=56800.
I have no idea what I was looking at.

[NotNull]

Everything is a search utility for file- and foldernames. It is also able to search in the content of files.
At first sight, it looks like a simple, straightforward "type a part of the filename and I show you the matching files" utility, but under the hood it has a lot of features to help out power users.
Total Commander can make use this Everything search engine, if Everything is installed on this machine.

I have no idea what I was looking at.

Description of How to integrate and use Everything in Total Commander, including a walkthrough of some of the power-features of Everything.


On topic:
A regex query searches for characters, not bytes. What specific character a byte (/multiple bytes) represents, will depend on the encoding of the file, the codepage of the OS and the regex engine.

So, if it wasn't clear already: Don't use regex to search through binary files.


Example why not, using the aforementioned Everything:
Searching for "c:\some folder" ext:vob regex:ansicontent:\x00\x00\x01\xB5\x14 will find vob files, whereas
"c:\some folder" ext:vob regex:ansicontent:\x00\x00\x01\xB5\x14\x82 will not.
That is because \x82 is high-ascii (0x80-0xff) and depends on encoding (note that \xB5 slips through as it accidentally represents the same character)


I don't think TC can help you here, unless there is some plug-in to do the heavy lifting.
Personally, I would script this, using SFK (Swiss File Knife) although this has no regex syntax support.
(mainly because I have this already 'installed'; I bet there are lots of similar tools out there)

Otherwise I would search for a utility that can do a hex-dump of your vob-file (causing them to be dispalyed as ASCII) and use regex on that hex-dump.

[tuska]

Ahh, NotNull has already summarised it - thanks!

2MarkFilipak
I try to explain my view of things:

• Based on your first message, I had the impression that you received unexpected results with RegEx queries - performed with TC
  F1 - Help: Total Commander uses the free Delphi library TRegExpr by Andrey V. Sorokin.

  [BUG] Help. What are limits of RegEx?
  It looks like RegEx is totally broken. | How can I get this regex to work? | What am I doing wrong?

• Thereupon I have tried to give the following advice:
  In Total Commander you can also make RegEx queries by using another RegEx library (PCRE) with the tool 'Everything'.

  Please see: Search queries in TC using 'Everything' - point 3 RegEx - Regular Expressions
  'Everything' uses 'Perl Compatible Regular Expressions (PCRE)'.

  At this time I also did not mention the [WDX] PCREsearch" plugin from milo1012 ... Perl Compatible Regular Expressions (PCRE),
  as I assumed that you would have a look at the linked point 3, which contains both detailed background information and examples.
   
• The search tool 'Everything', which can be integrated into TC for search queries, delivers search results in real-time for
  files and folders, which are then prepared in TC in the search results window.
  The file contents are currently not yet indexed, i.e. the search takes much longer.

• I have no idea what I was looking at.

  Here are listed preconditions and examples for the interaction of Total Commander and the tool 'Everything' in a search query.
  One could also call it the attempt of a summary/documentation.
   
• Here is a picture for RegEx queries in TC with 'Everything' (Click into the picture to enlarge the view)

I hope this answers your question sufficiently.

[MarkFilipak]

I hope this answers your question sufficiently.

Oh, Thank You! That really helps.

ev:regex:((.+?)?(?=\.vbs))(.vbs)$
ed:files:regex:^(?!\d\d\d\d\-\d\d-\d\d_).+

So, Everything implements a form of protocol and requires the user to use that protocol on the 'Search for:' line as though the 'Search for:' line is a command line (which, I suppose, it is in this case). I guess that the protocol is in lieu of any other selection method (such as buttons). I don't object to that much, but better integration would be nice (such as an Everything pop-up).

Thanks for preparing that picture -- it was extremely kind of you -- and I hope my thoughts profit you.

[tuska]

ev:regex:((.+?)?(?=\.vbs))(.vbs)$
ed:files:regex:^(?!\d\d\d\d\-\d\d-\d\d_).+

So, Everything implements a form of protocol and requires the user to use that protocol on the 'Search for:' line
as though the 'Search for:' line is a command line (which, I suppose, it is in this case).
I guess that the protocol is in lieu of any other selection method (such as buttons).
I don't object to that much, but better integration would be nice (such as an Everything pop-up).

Well, I think the first text block of "Search queries in TC using 'Everything'" says it all about the integration of this tool into Total Commander.

You can also make queries directly in 'Everything' and open e.g. the folder in Total Commander instead of Windows Explorer -
see here: 'Everything' Important information + settings (1 of 2) --> comes from documentation, pt. 13. Link collection.
Buttons in TC, e.g. with saved searches are of course also possible.

'Everything' should be started with Windows(!) and you can have e.g. an 'Everything'-icon displayed in the system tray.
Important basic settings can be found in the topic linked above.

If 'Everything' is checked - without the prefix ev: or ed: - then the search criteria of TC (with the support of 'Everything') apply,
i.e. you still get the search result much faster than if 'Everything' is not checked.

Unfortunately the topic now leads further and further away from your actual request ("... What are limits of RegEx? ...").
So if you have further questions about 'Everything', I suggest that you open a separate topic.

[MarkFilipak]

Well, I think the first text block of "Search queries in TC using 'Everything'" says it all about the integration of this tool into Total Commander.

You do? You do! Did you write it?

"1. TC Parameter --> ed:" -- what does this try to say? I don't know? Is it a calling "parameter" when launching TotalCommander? I don't think so -- I know so now but that's what I thought.

In the drop-down box:

Code: Select all

Search for: 		    --> up from the current directory in the active TC window
ed:dm:today		    --> Search for folders and files that have been changed (dm:) today (today)
ed: *.msg dc:thisweek	    --> Search for emails (.msg) created (dc:) this week (thisweek)

I have absolutely no idea what any of that is trying to say. That's because I don't know what "ed:dm:today" means and where it would go and why it is being cited. After your informative picture, I do now, but I didn't then.

I know what it means now because of your picture and because I'm an engineer. But trust me, it's way, way, way too cryptic. Try English instead of Martian.

[tuska]

Try English instead of Martian.

Sorry, unfortunately this would not help you as I am not a native English speaker.
For me the topic 'Everything' is finished here.

Regards,
Karl

[MarkFilipak]

Try English instead of Martian.

Sorry, unfortunately this would not help you as I am not a native English speaker.

I could help...

For me the topic 'Everything' is finished here.

...or maybe not.