Improve Zip compression ratio

[vudu]

Hello,

I noticed that InfoZIP compression code in Totalcmd produces slightly larger ZIP archives than WinRAR's ZIP method, and noticeably larger archives (up to 10%) than 7-Zip implementation (which is still fully compatible with ZIP 2.0 standard, opens without problem on old Unixes and Windows XP using built-in ZIP support, etc).

I compared them using maximum "deflate" compression level in all programs (not "deflate64" or other algorithms). So obviously, InfoZIP implementation is not of highest quality regarding compression ratio, and there is room for improvement.

Is it possible for mr Ghisler to improve ZIP compression by examining 7-Zip method (since it's open-source) and implement those optimizations in Totalcmd?

This is far from high priority of course, but it will be nice nevertheless.

Thanks!

[hlloyge]

...or probably use 7-zip's zip routine, just tested it on 390 MB Documents folder - Total Commander compresses it to 301 MB in 43 seconds, 7-zip with zip routine and one core to 299 MB in 80 seconds, and with 4 cores to 299 MB in 40 seconds. All on Maximum.
Speed is OK with TC, could be better, though.

[Horst.Epp]

...or probably use 7-zip's zip routine, just tested it on 390 MB Documents folder - Total Commander compresses it to 301 MB in 43 seconds, 7-zip with zip routine and one core to 299 MB in 80 seconds, and with 4 cores to 299 MB in 40 seconds. All on Maximum.
Speed is OK with TC, could be better, though.

A few mega bytes better compression are not of any importance with todays hard disc sizes.
Time for compression is also not so importand because archives are most of the time read and not written.
Unfortunately TC config doesn't allow to replace Zip with a packer plugin like Total7zip.

[Hacker]

Horst.Epp,

A few mega bytes better compression are not of any importance with todays hard disc sizes.
Time for compression is also not so importand because archives are most of the time read and not written.

Ie. archives are not important nowadays anymore.

Roman

[MVV]

Why are you use outdated ZIP when e.g. 7Z provides much better compression ratio?

[milo1012]

You already called the problem by it's name:
the deflate algorithm.

Yes, it can be optimized, which is exactly what is done by 7zip.

A few mega bytes better compression are not of any importance with todays hard disc sizes.

I would agree, but zip is still the most "compatible" format out there (sadly, due to it's limitations).


Workaround:
use MultiArc and create a new entry, like:

Code: Select all

[7z_zip]
;ID=50 4B 03 04
IDPos=0
Extension=7z_zip
Archiver=%COMMANDER_PATH%\Plugins\Packer\7z.exe
List="%P -r0 l %AQA"
Start="^-------------------"
End="^-------------------"
Extract="%P e -y {%S} %AqA @%Lq"
ExtractWithPath="%P x -y {%S} %AqA @%Lq"
Test="%P t -y {%S} %AqA"
Delete="%P d -y -r {%S} %AqA @%Lq"
Add="%P a -tzip -mx=9 -mmt -mm=deflate -mfb=258 -mpass=15 -mtc=off -y -r {%S} %AqA @%LqA"
Description="7-Zip - www.7-zip.org"
Format0=yyyy tt dd hh mm ss aaaaa zzzzzzzzzzzz pppppppppppp  nnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn
IgnoreErrors=0
Debug=0
UnixPath=0
SkipDirsInFileList=0
SkipEmptyNames=1
BatchUnpack=1
SearchForUglyDirs=0

Disadvantage: you need to rename 7z_zip to zip afterwards.
(but you can still open the archive with Ctrl+PgDown)

Also take care before you start compressing large file sets, because it can be very slow,
but produces the smallest zip files I'm aware of.
(well, maybe except Zopfli)

[MVV]

Total7zip allows packing to any 7-Zip format (just edit archive extension in pack dialog), and supports background operations.

[milo1012]

Total7zip allows packing to any 7-Zip format (just edit archive extension in pack dialog), and supports background operations.

Sure.
But you'd always need to check and change the Total7zip options if you wanted to create a zip file with maximum compression.
Using MultiArc provides a form of "preset", while it still integrates with TC.

[MVV]

You can create TC command to start 7zG.exe with right arguments and flags, and it will be much faster than via pack dialog - just via a single hotkey for selected files (the only difference that ignore list won't be used).

[milo1012]

You can create TC command to start 7zG.exe with right arguments and flags...just via a single hotkey for selected files (the only difference that ignore list won't be used)

Well, good luck trying to emulate things like "recurse sub-dirs", "create separate archives..." and "leave out base-dir" with that.
I don't see a reason why I would want to squeeze my operations through such interface, since the "Pack..." dialog has all you need already.

it will be much faster than via pack dialog

Why would it be faster?
The speed limit is set by the 7z exe. It might have some advantages when having tons of small files, I doubt that it will be much though.

[MVV]

I mean that it will be faster in your actions: single hotkey press instead of pressing hotkey, selecting packer etc.

BTW, 7zG is able to pack subdirs w/o problems. But creating separate archives seems to be unsupported.