speed up built in ZIP packer

[gonzo55]

current embedded packer is very slow, modern CPUs have 12-16 threads, built in packer can utilize only 3-5%
maybe add option to use WinRAR for ZIP compression, it can compress to ZIP format with 50% CPU load, and archives well compatible with stanard windows explorer

[ghisler(Author)]

I couldn't find any way to paralellize ZIP packing. The only thing I could do would be packing multiple files in parallel, but this would only be faster when using an SSD...

[gonzo55]

it's better than nothing

[Dalai]

You could just set a ZIP packer in Configuration > Options > Packer > Zip packer, and uncheck "Use internal ZIP packer" option.

Packing multiple files in parallel is a bad idea when they're on a harddrive.

Regards
Dalai

[gonzo55]

2Dalai
>and uncheck "Use internal ZIP packer" option
i already try it, i was set winrar.exe for ZIP packaging but it doesn't work, also i think pack optons "create separate archives..." and "leave out base directory..." will stop work with external program
>bad idea when they're on a harddrive
maybe for people without ssd, but we live in 2020 when ssd is very cheap, i don't know anybody from my friends who haven't it today and some of they without hdd!

[Dalai]

i was set winrar.exe for ZIP packaging but it doesn't work

Meaning?

also i think pack optons "create separate archives..." and "leave out base directory..." will stop work with external program

You can add command line parameters in the packer's edit field as well.

>bad idea when they're on a harddrive
maybe for people without ssd, but we live in 2020 when ssd is very cheap, i don't know anybody from my friends who haven't it today and some of they without hdd!

SSDs are excellent for storing the OS and perhaps most used programs, but certainly not for storing many large files like movies, large games and so on. If you can store such things on an SSD, good for you, but keep in mind that not all people have the money to buy large SSD, because they're still more expensive per GB than HDDs by several orders of magnitudes!

[EDIT]
Just to make it easier: WinRAR can still be used/called via the context menu, just like in Explorer. Meaning it's not necessary to set it as ZIP packer in TC.
[/EDIT]

Regards
Dalai

[petermad]

IF I configyre WINRAR.EXE (or RAR.EXE) as the ZIP-packer, I cannot avoid a password setting dialog (prompt).

[ghisler(Author)]

You cannot set WINRAR as ZIP packer because the parameters are completely different.

[vdijken]

Packing to a ZIP-archive on another device than the sources helps reducing the I/O on the same SSD or HDD.

[gonzo55]

Looked at the list of changes from beta 11 and found

16.12.22 Added: High speed packing and unpacking of ZIP files with modified libdeflate library (32/64)

Got excited and decided to give it a try. But no improvement, still 10% CPU utilisation at maximum compression level with the built-in ZIP packer. At the same time, packing ZIP in WinRAR with maximum compression loads the CPU by 45% with a corresponding increase in speed. If there are no plans to improve the built-in ZIP packer, then implement the connection as an external ZIP packer by WinRAR. There are now many obsolete formats in packer settings such as ARJ, LHA, UC2... I don't remember when I've seen them, maybe 15 or 20 years ago. Maybe replace them with something current, e.g. the one I suggested?

[Horst.Epp]

I suggest stopping that special handling of Zip archives.
Just allow to config any external packer or plugin to handle zip files
without the need for special tricks.

[ghisler(Author)]

Have you actually tried the new internal packer? In my tests, the 32-bit packer is more than twice the speed of the old packer, and the 64-bit packer is 3 times the speed (when packing large files), with the same CPU load.

[Dalai]

2gonzo55
Without having checked anything I think that the speed is improved. The change you quoted doesn't state anything about multi-threading leading to a higher CPU usage.

Regards
Dalai

[funkymonk]

I have done a few quick tests for TC11.00b1x32 on Win11b22621.
In average get 3x speedup on compress and 2x speedup on decompress (compared to TC 10.52).

[Galizza]

In average get 3x speedup on compress and 2x speedup on decompress (compared to TC 10.52).

Same results on my pc and the speed improvement on verify proccess is even better.