Page 1 of 2
speed up built in ZIP packer
Posted: 2020-02-20, 00:13 UTC
by gonzo55
current embedded packer is very slow, modern CPUs have 12-16 threads, built in packer can utilize only 3-5%
maybe add option to use WinRAR for ZIP compression, it can compress to ZIP format with 50% CPU load, and archives well compatible with stanard windows explorer
Re: speed up built in ZIP packer
Posted: 2020-02-20, 10:24 UTC
by ghisler(Author)
I couldn't find any way to paralellize ZIP packing. The only thing I could do would be packing multiple files in parallel, but this would only be faster when using an SSD...
Re: speed up built in ZIP packer
Posted: 2020-02-20, 14:07 UTC
by gonzo55
it's better than nothing
Re: speed up built in ZIP packer
Posted: 2020-02-20, 14:22 UTC
by Dalai
You could just set a ZIP packer in Configuration > Options > Packer > Zip packer, and uncheck "Use internal ZIP packer" option.
Packing multiple files in parallel is a bad idea when they're on a harddrive.
Regards
Dalai
Re: speed up built in ZIP packer
Posted: 2020-02-20, 15:24 UTC
by gonzo55
2Dalai
>and uncheck "Use internal ZIP packer" option
i already try it, i was set winrar.exe for ZIP packaging but it doesn't work, also i think pack optons "create separate archives..." and "leave out base directory..." will stop work with external program
>bad idea when they're on a harddrive
maybe for people without ssd, but we live in 2020 when ssd is very cheap, i don't know anybody from my friends who haven't it today and some of they without hdd!
Re: speed up built in ZIP packer
Posted: 2020-02-20, 15:51 UTC
by Dalai
gonzo55 wrote: 2020-02-20, 15:24 UTC
i was set winrar.exe for ZIP packaging but it doesn't work
Meaning?
gonzo55 wrote: 2020-02-20, 15:24 UTC
also i think pack optons "create separate archives..." and "leave out base directory..." will stop work with external program
You can add command line parameters in the packer's edit field as well.
gonzo55 wrote: 2020-02-20, 15:24 UTC>bad idea when they're on a harddrive
maybe for people without ssd, but we live in 2020 when ssd is very cheap, i don't know anybody from my friends who haven't it today and some of they without hdd!
SSDs are excellent for storing the OS and perhaps most used programs, but certainly not for storing many large files like movies, large games and so on. If you can store such things on an SSD, good for you, but keep in mind that not all people have the money to buy large SSD, because they're still more expensive per GB than HDDs by several orders of magnitudes!
[EDIT]
Just to make it easier: WinRAR can still be used/called via the context menu, just like in Explorer. Meaning it's not necessary to set it as ZIP packer in TC.
[/EDIT]
Regards
Dalai
Re: speed up built in ZIP packer
Posted: 2020-02-20, 18:39 UTC
by petermad
IF I configyre WINRAR.EXE (or RAR.EXE) as the ZIP-packer, I cannot avoid a password setting dialog (prompt).
Re: speed up built in ZIP packer
Posted: 2020-02-21, 14:18 UTC
by ghisler(Author)
You cannot set WINRAR as ZIP packer because the parameters are completely different.
Re: speed up built in ZIP packer
Posted: 2020-03-04, 08:50 UTC
by vdijken
Packing to a ZIP-archive on another device than the sources helps reducing the I/O on the same SSD or HDD.
Re: speed up built in ZIP packer
Posted: 2023-05-04, 16:32 UTC
by gonzo55
Looked at the list of changes from beta 11 and found
16.12.22 Added: High speed packing and unpacking of ZIP files with modified libdeflate library (32/64)
Got excited and decided to give it a try. But no improvement, still 10% CPU utilisation at maximum compression level with the built-in ZIP packer. At the same time, packing ZIP in WinRAR with maximum compression loads the CPU by 45% with a corresponding increase in speed. If there are no plans to improve the built-in ZIP packer, then implement the connection as an external ZIP packer by WinRAR. There are now many obsolete formats in packer settings such as ARJ, LHA, UC2... I don't remember when I've seen them, maybe 15 or 20 years ago. Maybe replace them with something current, e.g. the one I suggested?
Re: speed up built in ZIP packer
Posted: 2023-05-04, 17:02 UTC
by Horst.Epp
I suggest stopping that special handling of Zip archives.
Just allow to config any external packer or plugin to handle zip files
without the need for special tricks.
Re: speed up built in ZIP packer
Posted: 2023-05-04, 20:06 UTC
by ghisler(Author)
Have you actually tried the new internal packer? In my tests, the 32-bit packer is more than twice the speed of the old packer, and the 64-bit packer is 3 times the speed (when packing large files), with the same CPU load.
Re: speed up built in ZIP packer
Posted: 2023-05-04, 20:42 UTC
by Dalai
2gonzo55
Without having checked anything I think that the speed is improved. The change you quoted doesn't state anything about multi-threading leading to a higher CPU usage.
Regards
Dalai
Re: speed up built in ZIP packer
Posted: 2023-05-04, 20:51 UTC
by funkymonk
I have done a few quick tests for TC11.00b1x32 on Win11b22621.
In average get 3x speedup on compress and 2x speedup on decompress (compared to TC 10.52).
Re: speed up built in ZIP packer
Posted: 2023-05-04, 23:20 UTC
by Galizza
funkymonk wrote:
In average get 3x speedup on compress and 2x speedup on decompress (compared to TC 10.52).
Same results on my pc and the speed improvement on verify proccess is even better.