New PPM*, PAQ, BZIP* and HA packer plugin: PPMPackTC

Discuss and announce Total Commander plugins, addons and other useful tools here, both their usage and their development.

Moderators: white, Hacker, petermad, Stefan2

User avatar
alexanderwdark
Senior Member
Senior Member
Posts: 270
Joined: 2008-04-14, 07:20 UTC
Location: Russia
Contact:

Post by *alexanderwdark »

m^2 wrote:
Hacker wrote:alexanderwdark,
Very cool, thanks a lot! Hopefully finally a useful packer (that stores file dates and other attributes, etc.) that uses the PAQ algorithms. :)

Roman
:shock: What use for PAQ8 do you find?
PAQ8P has very strong compression, but very bad speed, PAQ9A - more good speed, but not so good compression. LPAQ8 - fast version of PAQ8 (implemented in text compression mode).

PAQ8P - for one-time compression and storing for years :)
User avatar
alexanderwdark
Senior Member
Senior Member
Posts: 270
Joined: 2008-04-14, 07:20 UTC
Location: Russia
Contact:

Post by *alexanderwdark »

Also impelemented Freehook (version 0.3 by Michael 'Eugene' Ortmann), based on Hook 0.8e, ADMC file compressor by Nania Francesco Antonio.
User avatar
alexanderwdark
Senior Member
Senior Member
Posts: 270
Joined: 2008-04-14, 07:20 UTC
Location: Russia
Contact:

Post by *alexanderwdark »

m^2 wrote:Did you get BALZ sources? Or did you use BALZ.exe?

Using exe - not so interesting for me. I have made some dlls from BALZ and Quad sources.
User avatar
m^2
Power Member
Power Member
Posts: 1413
Joined: 2006-07-12, 10:02 UTC
Location: Poland
Contact:

Post by *m^2 »

I know the PAQ family, though use different terms to describe their speed (LPAQ / PAQ9: very-extremely slow; PAQ8 - unusable).

"One time compression and storing for years"?
I.e. partition backup? Your computer spends a month compressing 20 GB. Then the partition fails. You need to wait another month to restore it. Not cool.

ADDED: BTW, do you have ECC memory? Because without it PAQ8 is unlikely to (de)compress a big file correctly.
Last edited by m^2 on 2008-12-29, 16:28 UTC, edited 1 time in total.
User avatar
alexanderwdark
Senior Member
Senior Member
Posts: 270
Joined: 2008-04-14, 07:20 UTC
Location: Russia
Contact:

Post by *alexanderwdark »

andres992 wrote:Great job! (I once tried to add PAQ8 to Multiarc but was not very successful.)

Is it possible to make a plugin for the *.pea format (PeaZip)?

Another very interesting challenge would be a plugin to produce the smallest standard (deflate) zip possible, using KZip (http://www.advsys.net/ken/utils.htm) with configurable options and DeflOpt.exe (http://www.walbeehm.com/download/) and AdvZip.exe (http://advancemame.sourceforge.net/doc-advzip.html) to further optimise some bytes... These utilities can usually beat 7zip's zips by ~5...10% and TC's own zips by even more.
Implementing format - to big job and not in plugin concept. Plugin works with its own open-text-header PPMP format, with universal structure (also used in DarkCryptTC, etc.), just as file - to file packer (or from TC to tar.ppmp - this is great TC's feature)

So, It's easy to add some algorithm, just for fun. But implementing format, (even very nice and great) with full multi-file structure - to much time and debugging for one coder.

For PEA - possible to make some implementation, but compatibility - must be another project.
User avatar
alexanderwdark
Senior Member
Senior Member
Posts: 270
Joined: 2008-04-14, 07:20 UTC
Location: Russia
Contact:

Post by *alexanderwdark »

m^2 wrote:I know the PAQ family, though use different terms to describe their speed (LPAQ / PAQ9: very-extremely slow; PAQ8 - unusable).

"One time compression and storing for years"?
I.e. partition backup? Your computer spends a month compressing 20 GB. Then the partition fails. You need to wait another month to restore it. Not cool.

ADDED: BTW, do you have ECC memory? Because without it PAQ8 is unlikely to (de)compress a big file correctly.
Do you have ECC memory - of course, not. Looking at some tests (maximumcompression, etc.), I see, testers had decompressed ENWiki and other file-setes by PAQ8 and CRC was ok. Plugin has not target to be some of state-of-art packers or commertial-quality product - this plugins stays with it's source, and someone can make stable releases. Implementing PAQ, etc. - just experimental, "just a hack"

P.S. Of course, LZMA, BZIP2 - state of art in packing. PPMD - king of text compression. For quick and high-level compression rar3 -m5 with it's ppmd text processor - better way.
User avatar
m^2
Power Member
Power Member
Posts: 1413
Joined: 2006-07-12, 10:02 UTC
Location: Poland
Contact:

Post by *m^2 »

According to Corsair, you can expect 1 memory error per 107 hours in 1800 MB that PAQ8 uses.
ENWIK9 takes 1 SIGB.
My computer (Pentium D@2.66ghz) gets 6-8 KB/s depending on a file when using PAQ8p1 exp04.
I didn't run it on ENWIK, but with 8 KB/s it's gonna take 33.9 hours to (de)compress and gives me ~68% of not encountering any error. Actually not every error is going to change compression output, so it's somewhat less problematic...and faster computers help too.

But the 20 GB example would indeed take a month. If Corsair is correct, it has a small chance of working well. But I'm not going to check.

ADDED:
rar -m1 is "rather fast" for me. :P

ADDED2:
PAQ8 is cool, fine that you added it. But I just can't see ANY practical use for it.
andres992
Member
Member
Posts: 124
Joined: 2006-04-12, 21:26 UTC

Post by *andres992 »

alexanderwdark wrote:
andres992 wrote:Is it possible to make a plugin for the *.pea format (PeaZip)?

Another very interesting challenge would be a plugin to produce the smallest standard (deflate) zip possible, using KZip...
Implementing format - to big job and not in plugin concept. Plugin works with its own open-text-header PPMP format, with universal structure (also used in DarkCryptTC, etc.), just as file - to file packer (or from TC to tar.ppmp - this is great TC's feature)

For PEA - possible to make some implementation, but compatibility - must be another project.
I was thinking of possible other/new plugins (not this one) anyway. The PEA format is a new one but offers quite nice compression and is yet unsupported by TC/plugins. Getting the smallest possible zip (e.g. in the case of smaller files like JavaSE files for mobile applications where there are strict size limits) could have big practical importance sometimes.

But I am not skilled enough to write such plugins myself, so my post was quite a theoretical offtopic. Sorry for that. :)
#68776 Personal Licence
User avatar
alexanderwdark
Senior Member
Senior Member
Posts: 270
Joined: 2008-04-14, 07:20 UTC
Location: Russia
Contact:

Post by *alexanderwdark »

m^2 wrote: PAQ8 is cool, fine that you added it. But I just can't see ANY practical use for it.
Yes, It's very cool algo, I think, PAQ can be optimized by it's authors, may be by some preprocessors and pre-sorters, and, sometime, whe can get algorithm with compression much better than rar for all typical datasets with rar's speed.
User avatar
Hacker
Moderator
Moderator
Posts: 13052
Joined: 2003-02-06, 14:56 UTC
Location: Bratislava, Slovakia

Post by *Hacker »

m^2,
What use for PAQ8 do you find?
Compression. ;)

Roman
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.
User avatar
m^2
Power Member
Power Member
Posts: 1413
Joined: 2006-07-12, 10:02 UTC
Location: Poland
Contact:

Post by *m^2 »

Hacker wrote:m^2,
What use for PAQ8 do you find?
Compression. ;)

Roman
Many thanks for your enlightening post. This is the answer, you're the man.
User avatar
alexanderwdark
Senior Member
Senior Member
Posts: 270
Joined: 2008-04-14, 07:20 UTC
Location: Russia
Contact:

Post by *alexanderwdark »

2008.12.29 Updated bzip2 engine to latest version, code now works faster, implemented also original bzip engine ver.0.21 by Julian Seward (from OS BSD ported version, compression better than in wide-used bzip2)
User avatar
alexanderwdark
Senior Member
Senior Member
Posts: 270
Joined: 2008-04-14, 07:20 UTC
Location: Russia
Contact:

Post by *alexanderwdark »

andres992 wrote:
I was thinking of possible other/new plugins (not this one) anyway. The PEA format is a new one but offers quite nice compression and is yet unsupported by TC/plugins. Getting the smallest possible zip (e.g. in the case of smaller files like JavaSE files for mobile applications where there are strict size limits) could have big practical importance sometimes.

But I am not skilled enough to write such plugins myself, so my post was quite a theoretical offtopic. Sorry for that. :)
Thanks for info about PEA, this is very good opensource project. And your idea is nice.
User avatar
alexanderwdark
Senior Member
Senior Member
Posts: 270
Joined: 2008-04-14, 07:20 UTC
Location: Russia
Contact:

Post by *alexanderwdark »

2008.12.31 Added PPMVC compressor (nice algorithm, based on PPMD, sometimes even better on large files) and LZPXj compressor.
User avatar
alexanderwdark
Senior Member
Senior Member
Posts: 270
Joined: 2008-04-14, 07:20 UTC
Location: Russia
Contact:

Post by *alexanderwdark »

Happy New Year!!!
2009.01.01 Added OCAMYD compressor.
Post Reply