HEX Pattern searching using TC

English support forum

Moderators: white, Hacker, petermad, Stefan2

User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 »

devd wrote:do I need to change anything in "OP" or "value"
I've tried every possibility in "OP" but maybe need to add something in "VALUE"
Ah, I see your problem: you expect that TC will return the string returned by the plug-in. This is not the case, but instead, when using it in the search dialog like you did, you are actually trying to search *within* this string(s) returned by the plug-in, in your case you'd expect the string to be empty and it won't find anything of course. So, a string field is rarely useful for the main search function, but is most useful for custom columns or tooltips. I showed you a custom column with your search in my screenshot. If you need some basic help for how to create a custom column, you might try the wiki or search the forum, e.g. start here:
http://www.ghisler.ch/board/viewtopic.php?t=42433

devd wrote:is there a way I can use Hex in that online tool ?
I tried, but it interpret it as text
Unfortunately not, but you can get a good impression on how regex will work, in case you need to expand your search.
You can simply replace a hex value outside the printable ASCII range with a placeholder character and adapt your RegEx, e.g. use a RegEx

Code: Select all

..\x46\x49\x4c\x45[\x00-\xff]{10,100}..[\x00-\xff]{10,20}(\x01\x02|\x31\x38)\x31\x36
which would find the string

Code: Select all

__FILEfoofoofoofoo+_foofoofoo1816
TC plugins: PCREsearch and RegXtract
devd
Junior Member
Junior Member
Posts: 21
Joined: 2018-03-28, 12:23 UTC

Post by *devd »

milo1012 wrote:Ah, I see your problem: you expect that TC will return the string returned by the plug-in. This is not the case, but instead, when using it in the search dialog like you did, you are actually trying to search *within* this string(s) returned by the plug-in, in your case you'd expect the string to be empty and it won't find anything of course. So, a string field is rarely useful for the main search function, but is most useful for custom columns or tooltips. I showed you a custom column with your search in my screenshot. If you need some basic help for how to create a custom column, you might try the wiki or search the forum, e.g. start here:
http://www.ghisler.ch/board/viewtopic.php?t=42433
Now its working like charm
milo1012 wrote:Unfortunately not, but you can get a good impression on how regex will work, in case you need to expand your search.
You can simply replace a hex value outside the printable ASCII range with a placeholder character and adapt your RegEx, e.g. use a RegEx

Code: Select all

..\x46\x49\x4c\x45[\x00-\xff]{10,100}..[\x00-\xff]{10,20}(\x01\x02|\x31\x38)\x31\x36
which would find the string

Code: Select all

__FILEfoofoofoofoo+_foofoofoo1816
thanks for the idea

is there a way to display another position (found from the search) the third byte or the last byte found (not the first)?
(I thought the 0 from \xO"0" represents the location, So, I've tried \xO1, apparently I'm wrong)

EDIT: BTW playing around with your tool (PCREsearch) the last week, I came to a conclusion that its A very powerful tool
how do I get to be an expert on this tool

example if I search "Or" (\x01\x01|\x02\x02) and want to output what result is found + the location of it
or if I want to export the results to A file
is there A list of all the possibilities able to put in "String output"?
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 »

devd wrote:is there a way to display another position (found from the search) the third byte or the last byte found (not the first)?
Not really from my side (the way I implemented it in my plug-in, it will always display the offset of the WHOLE string that is found), but the PCRE engine has the ability of "lookaround assertions", which will assume a string without actually being part of the capturing, but IMO are not that easy to understand. E.g. for displaying the offset of the 7th byte, you could do sth like this:

Code: Select all

(?<=..\x46\x49\x4c\x45)[\x00-\xff]{10,100}..[\x00-\xff]{10,20}(\x01\x02|\x31\x38)\x31\x36
Explanation:
LOOKAHEAD AND LOOKBEHIND ASSERTIONS
(?=...) positive look ahead
(?!...) negative look ahead
(?<=...) positive look behind
(?<!...) negative look behind
Each top-level branch of a look behind must be of a fixed length.
devd wrote:if I search "Or" (\x01\x01|\x02\x02) and want to output what result is found + the location of it
Sure, you can mix it to your liking, simply use for the replacement string sth like this:

Code: Select all

@0x\xO0: $0 
Though this will cut everything after a possible null byte (due to being a C-String).

devd wrote:is there A list of all the possibilities able to put in "String output"?
Just open the "Replace Help" section in the config tool or the readme file.

devd wrote:or if I want to export the results to A file
Well, not with the plug-in itself, but when having a custom columns view open, you could mark the files you want and use either the TC menu:
Mark -> Copy To Clipboard With Path+Details
or use the internal TC command
cm_CopyFileDetailsToClip
if you don't need the full path. Now paste this clipboard content to any text editor or similar tools.
TC plugins: PCREsearch and RegXtract
devd
Junior Member
Junior Member
Posts: 21
Joined: 2018-03-28, 12:23 UTC

Post by *devd »

milo1012 wrote:
devd wrote:is there a way to display another position (found from the search) the third byte or the last byte found (not the first)?
Not really from my side (the way I implemented it in my plug-in, it will always display the offset of the WHOLE string that is found), but the PCRE engine has the ability of "lookaround assertions", which will assume a string without actually being part of the capturing, but IMO are not that easy to understand. E.g. for displaying the offset of the 7th byte, you could do sth like this:

Code: Select all

(?<=..\x46\x49\x4c\x45)[\x00-\xff]{10,100}..[\x00-\xff]{10,20}(\x01\x02|\x31\x38)\x31\x36
Explanation:
LOOKAHEAD AND LOOKBEHIND ASSERTIONS
(?=...) positive look ahead
(?!...) negative look ahead
(?<=...) positive look behind
(?<!...) negative look behind
Each top-level branch of a look behind must be of a fixed length.
devd wrote:if I search "Or" (\x01\x01|\x02\x02) and want to output what result is found + the location of it
Sure, you can mix it to your liking, simply use for the replacement string sth like this:

Code: Select all

@0x\xO0: $0 
Though this will cut everything after a possible null byte (due to being a C-String).

devd wrote:is there A list of all the possibilities able to put in "String output"?
Just open the "Replace Help" section in the config tool or the readme file.

will check them ^ out latter
milo1012 wrote:
devd wrote:or if I want to export the results to A file
Well, not with the plug-in itself, but when having a custom columns view open, you could mark the files you want and use either the TC menu:
Mark -> Copy To Clipboard With Path+Details
or use the internal TC command
cm_CopyFileDetailsToClip
if you don't need the full path. Now paste this clipboard content to any text editor or similar tools.
what I meant by export to A file, is NOT the result found (I.e. the location)
I meant the actual string that was found by the search, since the search was used using wildcards,

or maybe output the sting found to string output and then use the methods you've mentioned ?!
if so, how can I use to output the string found

example if the file contains ABCDEFG ABCDEGG (its just an example since we are talking about Regex) and I search for CD*G
the search will give 2 results
1. CDEFG
2. CDEGG

those 2 stings (CDEFG + CDEGG) I want them export to A file

Thanks in advanced
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 »

devd wrote:I meant the actual string that was found by the search, since the search was used using wildcards,
Yes, this would still be possible with the method I mentioned, though you'd have to create an output file manually.
Just use a replacement string like

Code: Select all

$0<space or other separator string>
and it would assemble an output string, with each resulting string separated by your separator in the replacement string.

But I think for your case, my RegXtract plug-in would be a better (faster) solution. It basically offers the same syntax as PCREsearch (though the file offset function is still missing), but it will output each result (in a new line) in a new output file.
TC plugins: PCREsearch and RegXtract
devd
Junior Member
Junior Member
Posts: 21
Joined: 2018-03-28, 12:23 UTC

Post by *devd »

milo1012 wrote:But I think for your case, my RegXtract plug-in would be a better (faster) solution. It basically offers the same syntax as PCREsearch (though the file offset function is still missing), but it will output each result (in a new line) in a new output file.
Oh, will check it out too

many many thanks
devd
Junior Member
Junior Member
Posts: 21
Joined: 2018-03-28, 12:23 UTC

Post by *devd »

milo1012 wrote:But I think for your case, my RegXtract plug-in would be a better (faster) solution. It basically offers the same syntax as PCREsearch (though the file offset function is still missing), but it will output each result (in a new line) in a new output file.
:( it says no support for Win 9x ?
I dont use win 9x (I wish I could still use it :) )
devd
Junior Member
Junior Member
Posts: 21
Joined: 2018-03-28, 12:23 UTC

Post by *devd »

devd wrote:if I search "Or" (\x01\x01|\x02\x02) and want to output what result is found + the location of it
milo1012 wrote:Sure, you can mix it to your liking, simply use for the replacement string sth like this:

Code: Select all

@0x\xO0: $0 
Though this will cut everything after a possible null byte (due to being a C-String).
is there A way to get the output as HEX (not A gibberish string)?

example:

now I'm getting:

Code: Select all

Test_File.bin	@0x0310D6: °­ÞÎÿ @0x0BC0D6: °­ÞÎÿ
how to get real HEX has an output like this:

Code: Select all

Test_File.bin	@0x0310D6: \xB0\x04... @0x0BC0D6: \xB0\x04...
or like this:

Code: Select all

Test_File.bin	@0x0310D6: B004... @0x0BC0D6: B004...
TIA
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 »

devd wrote:is there A way to get the output as HEX (not A gibberish string)?
No. The plug-in is based on text output, so you will always get the hex bytes interpreted as ANSI/UTF-8/UTF-16 characters.

But it's actually a good idea to implement an optional hex output. I might add it in the next version (though I can't say when I'll have time for it).
TC plugins: PCREsearch and RegXtract
devd
Junior Member
Junior Member
Posts: 21
Joined: 2018-03-28, 12:23 UTC

Post by *devd »

milo1012 wrote:
devd wrote:is there A way to get the output as HEX (not A gibberish string)?
No. The plug-in is based on text output, so you will always get the hex bytes interpreted as ANSI/UTF-8/UTF-16 characters.
thanks
milo1012 wrote:But it's actually a good idea to implement an optional hex output. I might add it in the next version (though I can't say when I'll have time for it).
very kind of you
I will wait patiently
Post Reply