7zip native support

Here you can propose new features, make suggestions etc.

Moderators: Hacker, petermad, Stefan2, white

lelik007
Member
Member
Posts: 173
Joined: 2021-04-20, 06:37 UTC

Re: 7zip native support

Post by *lelik007 »

2ghisler(Author)
2. I see, I didn't checked Mb dictionaries because I usually use Gb dictionaries and that caught my eye immediately.
As for Mb dictionaries I don't know them to see the difference right away.
So I think TC should round up also GB.
lelik007
Member
Member
Posts: 173
Joined: 2021-04-20, 06:37 UTC

Re: 7zip native support

Post by *lelik007 »

2ghisler(Author)
1. Thank you, I checked the memory calculator and It shows the same GB values for the dictionaries as 7-zip.
2. As for this:
12.12.24 Fixed: Use internal 7-Zip packer to add files to a subdirectory in an archive also when packing with external 7-Zip is the chosen, because 7zg.exe doesn't have this functionality (32/64)
I'm an average user and don't know Igor's plans. But if we have a switch similar to rar's -ap in the future we'll be able to change this.
I'll try to let you know if it happens.
Last edited by lelik007 on 2024-12-19, 08:14 UTC, edited 1 time in total.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 50390
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: 7zip native support

Post by *ghisler(Author) »

Thanks, I will switch back to the external packer for packing to subdirectories when that happens.
Author of Total Commander
https://www.ghisler.com
lelik007
Member
Member
Posts: 173
Joined: 2021-04-20, 06:37 UTC

Re: 7zip native support

Post by *lelik007 »

2ghisler(Author)
Christian, if it could be done you should leave only 2 thread list for LZMA, this algo can't use more threads whatsoever. it's obvious, but I missed it somehow.

And as for the thread list for LZMA2, as I see TC's 7-zip module detects the maximum thread number available.
Maybe this thread list should be based on the maximum thread number detected and limited with this maximum?
I mean, why a user have 32 thread list if there's only 4 or 8 available for example.
This is not the same what 7-zip does but IDK exactly how it gives its choice.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 50390
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: 7zip native support

Post by *ghisler(Author) »

Christian, if it could be done you should leave only 2 thread list for LZMA, this algo can't use more threads whatsoever. it's obvious, but I missed it somehow.
I'm not getting any errors when passing more than two to the compressor for LZMA. Since I store the thread count in one variable in wincmd.ini, I will not change this behaviour for 11.50. Maybe later versions of the dll will support more threads for LZMA...
And as for the thread list for LZMA2, as I see TC's 7-zip module detects the maximum thread number available.
Maybe this thread list should be based on the maximum thread number detected and limited with this maximum?
I mean, why a user have 32 thread list if there's only 4 or 8 available for example.
This is not the same what 7-zip does but IDK exactly how it gives its choice.
I haven't found a good way to automatically set the thread count, so I will leave it entirely to the user.
7-Zip does something strange: It has some hard coded thread numbers, and in addition to that, it limits the thread count based on the available main memory. For example, if the user sets that limit to 90%, it will use 90% of the installed main memory for packing. It seems to ignore the number of processors or simultaneous threads.
Author of Total Commander
https://www.ghisler.com
User avatar
MaxX
Power Member
Power Member
Posts: 1166
Joined: 2012-03-23, 18:15 UTC
Location: UA

Re: 7zip native support

Post by *MaxX »

ghisler(Author) wrote: 2024-12-22, 08:58 UTC For example, if the user sets that limit to 90%, it will use 90% of the installed main memory for packing. It seems to ignore the number of processors or simultaneous threads.
Yes. 7zip has no need to use additional threads when is limited to RAM. The dictionary size and other options are much more important (the only important things at all).
And more I can say. Any additional thread causes worse compression than without them. The best result is 1 thread for LZMA and 2 threads for LZMA2. There is no more sense to use more threads if you need good compression.
Ukrainian Total Commander Translator. Feedback and discuss.
lelik007
Member
Member
Posts: 173
Joined: 2021-04-20, 06:37 UTC

Re: 7zip native support

Post by *lelik007 »

2ghisler(Author)
Since I store the thread count in one variable in wincmd.ini, I will not change this behaviour for 11.50.
These can be done in the future versions. After the testing of course.

1. The list itself for LZMA should be 2 somehow, in the case if LZMA is selected TC will only write 1 or 2 in "7zThreads=".
And if TC reads something like 7zThreads=6 for LZMA from .ini, no matter this, it's 2 in the list.
And the 2-nd way is the separate values in .ini, like: "7zThreadsLZMA=" , "7zThreadsLZMA2=" , "7zThreadsBZip2="
But what to do if somebody curious will put "7zThreadsLZMA=10" manually in .ini (this IDK)?

2. As for LZMA2 threads, take a look at 7-zip, to the right from the memory calculator, it's 80% in there.
https://www.upload.ee/image/17501594/TC7zCalc.jpg
This is a GUI presentation of memuse switch (Help - Command Line Version - Switches - -m (Method) - CTRL+F - memuse).
In GUI itself it's the last warning, but what this switch really does you can read in Help of 7-zip.
In addition to it, we had a hard time to understand what this thing is, so my nephew asked Igor and this was his answer.
https://sourceforge.net/p/sevenzip/discussion/45797/thread/c12ab8b42c/#346f/bd00
This can be for example: -mmemuse=30, Igor said this means 30% of all resources (not documented). The default value is -mmemuse=p80.

And this value is included in the process how 7-zip defines the thread count for its list.
But as for TC's list for LZMA2 and BZip2, if this doable I meant it could be maximum detected threads.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 50390
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: 7zip native support

Post by *ghisler(Author) »

The problem is when I limit it to 2 for LZMA and the user had set it to, say, 8 threads for LZMA2, then saving it for LZMA would change it to 2 only also for LZMA2. Therefore I will not change anything for 11.50 and think about a better solution for later versions, sorry.
Author of Total Commander
https://www.ghisler.com
lelik007
Member
Member
Posts: 173
Joined: 2021-04-20, 06:37 UTC

Re: 7zip native support

Post by *lelik007 »

2ghisler(Author)
Therefore I will not change anything for 11.50 and think about a better solution for later versions, sorry.
I understand there's no time for these things and they need to be tested. That were the plans for the minor versions after 11.50.
It's not not necessary to do anything right now because 7z.dll returns no errors for LZMA thread count more than 2.
But to show the difference between the algos and from the esthetic point of view it's nice to have them in the future.

As for 80% memory usage, TC supports -m and its sub-switches, so the users can redefine -mmemuse=N themselves.
lelik007
Member
Member
Posts: 173
Joined: 2021-04-20, 06:37 UTC

Re: 7zip native support

Post by *lelik007 »

2ghisler(Author)
It seems like a solid block selection in GUI doesn't work, for the internal packer at least. A value stays the default as I think and It's easy to see,
we can again repack:
https://download.cpuid.com/cpu-z/cpu-z_2.13-en.zip
select 1m solid block size in GUI - archive №1, then pack with -ms=1m in Extra Parameters - archive №2.
The archives should be the same because -ms=1m is the explicit definition of solid block size 1 m, but the archives aren't the same.
In extra parameters a definition of a solid block size works as it should.
Last edited by lelik007 on 2025-01-09, 16:18 UTC, edited 2 times in total.
User avatar
AntonyD
Power Member
Power Member
Posts: 1554
Joined: 2006-11-04, 15:30 UTC
Location: Russian Federation

Re: 7zip native support

Post by *AntonyD »

Maybe it's worth highlighting probably the last 6 or 7 posts from this topic (starting from #20 Dec 2024#)
and moving them to a new topic in the Bugs section?
It seems like it should apply specifically to them, rather than to suggestions.
#146217 personal license
lelik007
Member
Member
Posts: 173
Joined: 2021-04-20, 06:37 UTC

Re: 7zip native support

Post by *lelik007 »

2AntonyD
There are too many bugs or mismatches I found so I can't create a topic for every single one. I can bug report here or nowhere.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 50390
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: 7zip native support

Post by *ghisler(Author) »

It seems like a solid block selection in GUI doesn't work
Unfortunately you are right - TC 11.50 writes the solid block size to compression-dependent variables now, but still reads them from 7zSolidSize instead. I will fix it in 11.51.
Maybe it's worth highlighting probably the last 6 or 7 posts from this topic (starting from #20 Dec 2024#)
I don't see any bugs in these posts, please clarify.
Author of Total Commander
https://www.ghisler.com
User avatar
AntonyD
Power Member
Power Member
Posts: 1554
Joined: 2006-11-04, 15:30 UTC
Location: Russian Federation

Re: 7zip native support

Post by *AntonyD »

2ghisler(Author)
please clarify.
Pls, find above your initial words like this: "Since I store the thread count in one variable in wincmd.ini, I will not change this behaviour for 11.50."

2lelik007
So I didn't suggest you personally to deal with the problem of moving posts to another/appropriate topic in the right section of the forum.
That would have to be done by the moderators.
And already in a new topic you would describe all new/old found bugs/clarifications/mismatches.
The main thing is that the speech should always be conducted directly in the proper "Bugs" section,
and not as now - in the Suggestions...
#146217 personal license
lelik007
Member
Member
Posts: 173
Joined: 2021-04-20, 06:37 UTC

Re: 7zip native support

Post by *lelik007 »

2ghisler(Author)
Thank you, fix it please.
2AntonyD
I understood your idea, next time I'll create a topic in the "Bugs" section and ask Christian and the moderators to not close it while we're changing 7-zip module. I could enumerate the bugs or mismatches to identify what we deal with, here I'll leave just the suggestions.
Post Reply