7zip native support

[MaxX]

2petermad
https://i.imgur.com/w1NXF5G.png
BCJ2 is used for executables - dll, exe, wcx/wdx/wfx/wlx, scr, etc...
Delta is used for uncompressed PCM 8\16 bit 1\2 channel.
There also exist ARM64 and few another types. I don't remember them all, need to read the help docs.

[petermad]

BCJ2 is used for executables - dll, exe, wcx/wdx/wfx/wlx, scr, etc..

but in my tests, packing wcx64 and wlx64 (but avoiding exe and dll) resulted in identical archives when using internal and external packer.

7zFM only shows differences in packed size for internal and extrenal packed archives - the methods are the same, the clock numbers are the same:
https://tcmd.madsenworld.dk/7zsizes.png 7zsizes.png
so where lies the cause of the difference????

[MaxX]

Here I found something on another forum.
Google translate:
Apparently, I found the problem - in TC, the object that implements the ISequentialInStream interface does NOT implement the IStreamGetSize interface. This interface is used to get the stream size, and the stream size is used (probably) to initialize some internal compression parameter, let's call it Z. And if there is no size, then Z is used with the default value, not with the value optimal for the stream size, hence the discrepancy in archive sizes. I conducted an experiment. In the proxy dll, I disabled getting IStreamGetSize, and after that 7-Zip began to create archives of the same size as TC.
May be that can help.

[ghisler(Author)]

Nice, implementing the IStreamGetSize interface does indeed improve the compression rate! Try it yourself:
https://www.totalcommander.ch/beta/tcbit7z_getsize.zip

However, I'm still getting different archives than with 7zg.exe. The reason is that 7zg is sorting the files differently than the dll: When you look inside and set sort order to "unsorted", then you can see that the dll sorts exe/dll files alphabetically together. However, 7zg.exe does not do that: Here it's putting some small files like SHARE_NT.EXE (TC directory) at the start, no idea why.

[petermad]

When you look inside and set sort order to "unsorted", then you can see that the dll sorts exe/dll files alphabetically together. However, 7zg.exe does not do that: Here it's putting some small files like SHARE_NT.EXE (TC directory) at the start, no idea why.

Hmm that is not what I see with my test scenario - here the files are in the same order in internally and extarenlly packed files: https://tcmd.madsenworld.dk/7zfileorder.png

I tested with the files from tc1150x32_64_b5.exe - and you are right - here the files SHARE_NT.EXE and WC32TO16.EXE are positioned differently in the internally and the externally packed file: https://tcmd.madsenworld.dk/7zfileorder2.png

[MaxX]

2ghisler(Author)
>> SHARE_NT.EXE (TC directory) at the start, no idea why

This file does not get BCJ2 filter used. The only reason. It should be this way when ok.

[petermad]

the FileInfo plugin shows this for the "Image file header" of the files SHARE_NT.EXE and WC32TO16.EXE

File Type Description : New Executable (NE) (Windows 3.1)
File Characteristics : File is a Windows application (MULTIPLEDATA)
File uses P.M. Windowing API

So maybe that is why 7zG.exe treats them differently... Those two files are the only ones with that header info


2MaxX

This file does not get BCJ2 filter used. The only reason. It should be this way when ok.

Hmm, So the internal packing is more correct than the external using 7zG.exe. It is the external packing that does not use BCJ2 on those two exe files

[lelik007]

2MaxX
2ghisler(Author)
If you think this is any filter affects it - you can check with -mmf=off (disables all filters) or vice versa -mmf=BCJ2 force BCJ2 for executables.
2ghisler(Author)
And maybe at some point it'd be a good Idea to do something that odd compression levels (that we usually use in 7zG.exe) to match corresponding levels of the internal packer.
https://www.upload.ee/image/17350725/7zdiff2.jpg
Solid block size for example for level 9 (ultra) is 16g in 7zG.exe, I set 2g for testing - why is it 2g in the internal packer?

[ghisler(Author)]

Solid block size for example for level 9 (ultra) is 16g in 7zG.exe, I set 2g for testing - why is it 2g in the internal packer?

There is no documentation of the default values anywhere. I tried to figure them out from the compress dialog, but I may have missed some values.

[lelik007]

2ghisler(Author)
Yes, I meant exactly what we see it in 7zG.exe (compress dialog) - some values don't match.

[petermad]

in 7-zip's dialog (7zFM.exe) I see these default values for solid block sizes:

LZMA2:
Compression 1 - 64MB
Compression 5 - 4GB
Compression 9 - 16GB

LZMA:
Compression 1 - 32MB
Compression 5 - 2GB
Compression 9 - 4GB

PPMd:
Compression 1 - 128MB
Compression 5 - 2GB
Compression 9 - 4GB

BZip2:
Compression 1 - 16MB
Compression 5 - 112500KB
Compression 9 - 112500KB

7zFM.exe does not have compression 6, so I listeted compression 5 (normal) instead

[lelik007]

I add these values for the solid block sizes.
LZMA2:
Compression 3 - 1GB
Compression 7 - 8 GB

LZMA:
Compression 3 - 512 MB
Compression 7 - 4 GB

PPMd:
Compression 3 - 512 MB
Compression 7 - 4 GB

BZip2:
Compression 3 - 62500 KB
Compression 7 - 112500 KB

[AntonyD]

2ghisler(Author)
Why don't you offer the author of the 7-zip development at least some reward to create a help section for you and
help you with the current implementation process?
It is very likely that he would even suggest some moment that is nowhere to be found, but he just made your work
a thousand times easier!
For example, perhaps he could tell you how to completely get rid of the intermediate library that you are currently
forced to use.

[ghisler(Author)]

Currently I'm not saving the solid block sizes separately for each compression rate.

I will add it, but not the odd values for BZIP2 (currently it's all stored in MBytes). I will use rounded values 64MB and 128MB.

[petermad]

2ghisler(Author)
Is it correct to assume that TC's values for compression rates corresponds 7zFM.exe's "Add to archive" dialog like this:
normal compression (6) = 5 - Normal ????
maximum compression (9) = 9 - Ultra
fastest compression (1) = 1 - Fastest

And if I manually (as "other") choose 6, what do I get then?

Wouldn't it be better if "normal compression" was set to 5, since that is what 7-Zip (7zFM.exe) uses for "Normal".