HEX Pattern searching using TC

English support forum

Moderators: white, Hacker, petermad, Stefan2

.:hack3r2k:.
Junior Member
Junior Member
Posts: 2
Joined: 2015-10-01, 07:38 UTC

HEX Pattern searching using TC

Post by *.:hack3r2k:. »

Hi,

I need to look for a specific hex pattern in a bunch of files. Pattern looks like

78 56 34 12 ?? ?? ?? 00 04 24 00 21 where ?? can be any hex . How can i do that in TC. Tried replacing ?? by .., ** . Nothing seems to work.


Br;)
User avatar
Ovg
Power Member
Power Member
Posts: 756
Joined: 2014-01-06, 16:26 UTC

Post by *Ovg »

It's impossible to lead us astray for we don't care even to choose the way.
#259941, TC 11.01 x64, Windows 7 SP1 x64
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 »

Wildcards won't work in hex search AFAIK, you need to use RegEx.

But:
TC'S RegEx engine is limited, because it doesn't take newline characters into account.
(every line is searched individually, also in binary files)
This means that if your ?? consists of 0x0A, TC won't find it.

Additionally it seems that \x00 doesn't work either.

You may try:

Code: Select all

\x78\x56\x34\x12[\x00-\xff]{3}.\x04\x24.\x21
[x] RegEx (2)

But for file listing (F3) you'd have the same problem of course.




To use a precise search, use my PCREsearch (second link above).
Use PCREsearchConfig.exe.
Create a new field, name it to your liking, and insert
RegEx:

Code: Select all

\x78\x56\x34\x12[\x00-\xff]{3}\x00\x04\x24\x00\x21
Restart TC or use the internal command cm_UnloadPlugins,
and now activate the plug-in field in the search dialog ("Plugins" tab) and start your search.


You can even get the file offset(s):
Change field type to "String output".
Replace String:

Code: Select all

\xO0, 
You can now create a custom column, with that very field you just created.
In there you can see every file offset (hex) where the string is found, separated by a comma.
TC plugins: PCREsearch and RegXtract
.:hack3r2k:.
Junior Member
Junior Member
Posts: 2
Joined: 2015-10-01, 07:38 UTC

Post by *.:hack3r2k:. »

Thanks milo1012,

It worked perfectly !

Br;)
devd
Junior Member
Junior Member
Posts: 21
Joined: 2018-03-28, 12:23 UTC

Post by *devd »

milo1012 wrote:To use a precise search, use my PCREsearch (second link above).
Use PCREsearchConfig.exe.
Create a new field, name it to your liking, and insert
RegEx:

Code: Select all

\x78\x56\x34\x12[\x00-\xff]{3}\x00\x04\x24\x00\x21
Restart TC or use the internal command cm_UnloadPlugins,
and now activate the plug-in field in the search dialog ("Plugins" tab) and start your search.


You can even get the file offset(s):
Change field type to "String output".
Replace String:

Code: Select all

\xO0, 
You can now create a custom column, with that very field you just created.
In there you can see every file offset (hex) where the string is found, separated by a comma.
Hi,

first step says
milo1012 wrote:Create a new field
and I cant find A way to create A new field

can someone guide for a noob like me step by step how it's done ?
I tried to follow the above, and got lost in the first step

TIA
Last edited by devd on 2018-03-28, 13:36 UTC, edited 1 time in total.
User avatar
Stefan2
Power Member
Power Member
Posts: 4124
Joined: 2007-09-13, 22:20 UTC
Location: Europa

Post by *Stefan2 »

Hi and welcome, devd


Actually, 'first step says' : "use my PCREsearch"

.... have you installed the mentioned "PCREsearch" first?




 
devd
Junior Member
Junior Member
Posts: 21
Joined: 2018-03-28, 12:23 UTC

Post by *devd »

Stefan2 wrote:Actually, 'first step says' : "use my PCREsearch"
Right :D
Stefan2 wrote:.... have you installed the mentioned "PCREsearch" first?
yes
User avatar
Stefan2
Power Member
Power Member
Posts: 4124
Joined: 2007-09-13, 22:20 UTC
Location: Europa

Post by *Stefan2 »

devd wrote:
Stefan2 wrote:Actually, 'first step says' : "use my PCREsearch"
Right LOL
Stefan2 wrote:.... have you installed the mentioned "PCREsearch" first?
yes
Fine.
( For others > milo1012s' PCREsearch > http://ghisler.ch/board/viewtopic.php?t=38098 )



Launch the "PCREsearchConfig.exe".


I just took a look and it's not very clear what to do there next (even after reading the help).




As for what I see there, the List entry "PCRE-RegEx14" on the left hand seems to be a not already used "field".

See "PCREsearchConfig utility"-section in "TC\plugins\wdx\PCREsearch\Readme\Readme.html"




Use that entry (or the next free one) to create a "new field", an own entry:
- exchange the Regular Expression with the wanted one,
- clear or modify the Replace box,
- as Name enter what you like to indicate this entry.

Select at least "14" at the Fields box in the top left corner.
(or as many you need to get the '-->' mark on your 'devd' entry)

Click at [OK] to close the PCREsearchConfig.exe.

Restart TC or use the internal command cm_UnloadPlugins,

Now activate the plug-in field in the search dialog ("Plugins" tab)
- choose PCREsearch and
- select the Name (Property) you have chosen to indicate the new entry.




HTH? :D

If you need more info, wait till milo1012 visits the forum, another can help, or search the forum for other examples for this PCREsearch tool.



 
devd
Junior Member
Junior Member
Posts: 21
Joined: 2018-03-28, 12:23 UTC

Post by *devd »

Thanks

field type needs to be changed ?
(the default is A Boolean)

so far when doesn't chosen a Boolean or Encoding check it doesn't seem to work
what should I choose?
Stefan2 wrote: - clear or modify the Replace box,
I have cleared that, and even after saved
when next opened it appears again ($0\x20)
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 »

2devd
So you managed to create a new field and a boolean search worked for you, but you want what I described in the 2nd part of my post? In that case you need to do as I said:
Set the field type to "String output" and - most important - set the replace string to

Code: Select all

\xO0,
(note that the character trailing the x is not a zero, but the capital letter "O"!)
This would output the file offset(s) at which your search strings are found.
devd wrote:
Stefan2 wrote: - clear or modify the Replace box,
I have cleared that, and even after saved
when next opened it appears again ($0\x20)
Yes, this is the default replace string, i.e. all output results are separated by a space character.
Since it makes no sense to have no replace string at all (for string field types), this will reappear when you cleared it.
TC plugins: PCREsearch and RegXtract
devd
Junior Member
Junior Member
Posts: 21
Joined: 2018-03-28, 12:23 UTC

Post by *devd »

@milo
thanks for helping me
milo1012 wrote:So you managed to create a new field
yes, I have used one the field mentioned by @stefan2
milo1012 wrote:and a boolean search worked for you,
no, its not working
have try to play with it
but either I'm getting all the files in the directory chosen the search (which can't be)
Or I'm getting no founds at all (which is also not true)
milo1012 wrote:but you want what I described in the 2nd part of my post?
still not there yet
need to it work first
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 »

devd wrote:...
either I'm getting all the files in the directory chosen the search (which can't be)
Or I'm getting no founds at all (which is also not true)
I assume you restarted TC or used cm_UnloadPlugins after changing the regex for the field?
If this is not the problem, make sure the expression really *should* work, i.e. maybe post the RegEx you are using and what you expect it to match.
TC plugins: PCREsearch and RegXtract
devd
Junior Member
Junior Member
Posts: 21
Joined: 2018-03-28, 12:23 UTC

Post by *devd »

milo1012 wrote:I assume you restarted TC or used cm_UnloadPlugins after changing the regex for the field?
of course, after every change I tried/made
milo1012 wrote:If this is not the problem, make sure the expression really *should* work, i.e. maybe post the RegEx you are using and what you expect it to match.
Why not
but the system does not let me attach anything because of being a new member

Edit: it may let me attach code
will try
Last edited by devd on 2018-03-28, 20:58 UTC, edited 1 time in total.
devd
Junior Member
Junior Member
Posts: 21
Joined: 2018-03-28, 12:23 UTC

Post by *devd »

Oh, it does let me attach code

example of the (part) file tried to search in

Code: Select all

000014A0:   00 00 00 00 00 00 00 00|00 00 00 00 00 00 00 00 
000014B0:   46 49 4C 45 00 00 00 00|80 00 0E 02 01 00 00 00 
000014C0:   00 00 00 00 03 00 00 00|00 00 00 00 00 00 00 00 
000014D0:   00 00 00 00 03 00 00 00|00 00 00 00 18 00 68 00 
000014E0:   02 00 F7 0E 00 00 00 00|1F 88 F8 29 ED 1F 88 F8 
000014F0:   29 EA 00 00 00 00 00 1F|88 F8 29 EA 01 00 00 00 
00001500:   01 00 00 00 10 00 58 00|31 00 32 00 33 00 34 00 
00001410:   2E 00 74 00 78 00 74 00|80 00 00 00 00 00 00 00 
00001420:   00 00 00 00 00 00 00 00|00 00 00 00 00 00 00 00 
00001430:   4E 41 43 4C 02 00 20 00|5C 00 00 00 00 00 00 00 
00001440:   00 00 00 00 73 0D 01 00|04 80 14 00 00 00 1B EE
RegEx using:

Code: Select all

\x00\x00\x46\x49\x4c\x45[\x00-\xff]{52}\x1f\x88
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 »

2devd
Strange, it works for me perfectly. After I created a file with your hex sequence and with your RegEx, it will find results in/for that very file only, nothing else.

At first I thought that the hex snippet represents a file in UTF-16, but this is in fact not the case. But maybe your file is somehow caught by the xdoc2txt filter that PCREsearch is using by default. So is your binary file in fact a doc, odt, pdf, rtf, sxw or xml file? In that case you'd need to disable the filter to make binary search possible.

If this is still not helping, I can only guess that your PCREsearch settings don't fit. You may post the content of your PCREsearch.Sample.ini/PCREsearch.ini somewhere (either directly here as a code block if it's not too big, or on some extern site and link to it).
TC plugins: PCREsearch and RegXtract
Post Reply