MRT: regex that has possible zero-length is not checked at the end of the searched string

Bug reports will be moved here when the described bug has been fixed

Moderators: Hacker, petermad, Stefan2, white

Post Reply
User avatar
white
Power Member
Power Member
Posts: 5785
Joined: 2003-11-19, 08:16 UTC
Location: Netherlands

MRT: regex that has possible zero-length is not checked at the end of the searched string

Post by *white »

  • Create a file with name "12345".
  • Click the file in Total Commander and start the Multi-Rename Tool (Ctrl+M).
  • I want to add dashes around all digits, so I search for regular expression "(?:)" which means zero-length string and replace with "-"

Code: Select all

Filename:		12345

Search for:		(?:)
Replace with:		-
1x:			Disabled
Subst:			Disabled

Result:			-1-2-3-4-5
Expected result:	-1-2-3-4-5-
The result is missing a dash at the end.

On https://regex101.com/ you can see that 7 different flavors of regular expressions do match the position at the end.
Just fill in the following data and click the different flavors.

Code: Select all

REGULAR EXPRESSION:	(?:)
TEST STRING:		12345

Another example:
  • Create a file with name "1234_text_ABCD".
  • Click the file in Total Commander and start the Multi-Rename Tool (Ctrl+M).
  • I want to add two underscores around all chars and remove the text in between underscores in the original filename

Code: Select all

Filename:		1234_text_ABCD

Search for:		(_[a-zA-Z]+_)?
Replace with:		__
1x:			Disabled
Subst:			Disabled

Result:			__1__2__3__4____A__B__C__D
Expected result:	__1__2__3__4____A__B__C__D__

P.S. In older versions of Total Commander this problem doesn't occur. (?:) isn't supported in older versions, but A{0} and examples with ()? do work.
User avatar
nsp
Power Member
Power Member
Posts: 1913
Joined: 2005-12-04, 08:39 UTC
Location: Lyon (FRANCE)
Contact:

Re: MRT: regex that has possible zero-length is not checked at the end of the searched string

Post by *nsp »

This is probably a bug to report to Andrey V. Sorokin as TC use his Delphi library TRegExpr.

In both of your sample, you want to match empty string ! You have ambiguity depending of the engine implementation/optimization i.e no more match after latest char or infinite matches. Even if this his can be agreed by major RegExp implementation this is not the case for all (javascript do not even support such expression ).
Both sample illustrate the same interpretation: matching after latest char !

--edited for clarity
User avatar
white
Power Member
Power Member
Posts: 5785
Joined: 2003-11-19, 08:16 UTC
Location: Netherlands

Re: MRT: regex that has possible zero-length is not checked at the end of the searched string

Post by *white »

nsp wrote: 2022-03-14, 06:05 UTC You have ambiguity depending of the engine implementation/optimization i.e no more match after latest char or infinite matches.
The ambiguity is not related to the last char but to zero-length matches at any position. You either have infinite of such matches at any position or only one. See for example watch-out-for-zero-length-matches.
The implementation used by Total Commander matches one time and should also match at the end.
nsp wrote: 2022-03-14, 06:05 UTC Even if this his can be agreed by major RegExp implementation this is not the case for all (javascript do not even support such expression ).
I wouldn't point to javascript for a reference on regular expressions, but nowadays they seem to support it. Which javascript still doesn't?
nsp wrote: 2022-03-14, 06:05 UTC In both of your sample, you want to match empty string !
...
Both sample illustrate the same interpretation: matching after latest char !
Exactly, I gave two examples of the bug described by the topic title.
User avatar
white
Power Member
Power Member
Posts: 5785
Joined: 2003-11-19, 08:16 UTC
Location: Netherlands

Re: MRT: regex that has possible zero-length is not checked at the end of the searched string

Post by *white »

2ghisler(Author)
I see you moved this thread to forum TC Behaviour which will not be changed. I think this is a bug in the regex library. Have you contacted the author of the regex lib about this? Was his response it will not be changed?

Since this breaks compatibility with version 9.51 and older, are you going to notify users in the "What's new" that since version 10.00 regex works differently and that their regular expressions may no longer give the same results?
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 50475
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: MRT: regex that has possible zero-length is not checked at the end of the searched string

Post by *ghisler(Author) »

Unfortunately I can't change this myself because it's in the library, but I also think that it may be better to not change it...
Author of Total Commander
https://www.ghisler.com
User avatar
white
Power Member
Power Member
Posts: 5785
Joined: 2003-11-19, 08:16 UTC
Location: Netherlands

Re: MRT: regex that has possible zero-length is not checked at the end of the searched string

Post by *white »

ghisler(Author) wrote: 2022-04-25, 10:22 UTC Unfortunately I can't change this myself because it's in the library, but I also think that it may be better to not change it...
I find that really weird. Can you share your reasoning?
User avatar
white
Power Member
Power Member
Posts: 5785
Joined: 2003-11-19, 08:16 UTC
Location: Netherlands

Re: MRT: regex that has possible zero-length is not checked at the end of the searched string

Post by *white »

I created an issue on github now:
https://github.com/andgineer/TRegExpr/issues/265

It is suggested there that the last version of TRegexpr doesn't have this problem and that you should make sure you are using the last version of TRegexpr.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 50475
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: MRT: regex that has possible zero-length is not checked at the end of the searched string

Post by *ghisler(Author) »

Their latest release is from February 2017...

Edit: I used their latest trunk code and it seems to work. Please test when beta 3 comes out.
Author of Total Commander
https://www.ghisler.com
User avatar
white
Power Member
Power Member
Posts: 5785
Joined: 2003-11-19, 08:16 UTC
Location: Netherlands

Re: MRT: regex that has possible zero-length is not checked at the end of the searched string

Post by *white »

Tested OK using TC10.50b3 32bit and 64bit.
User avatar
white
Power Member
Power Member
Posts: 5785
Joined: 2003-11-19, 08:16 UTC
Location: Netherlands

Re: MRT: regex that has possible zero-length is not checked at the end of the searched string

Post by *white »

Moderator message from: white » 2022-04-27, 20:15 UTC

Moved to Fixed bugs
Post Reply