New plugins: .xz and .lzma

[karlchen]

Hello, iana.

Thanks for your explanation. Now I start understanding why T.C. 8.0 x64 does not offer the usual minimal configuration dialogue for the lzma v2.0 packer plugin which T.C. 8.0 x32 does offer.

Kind regards,
Karl

[ghisler(Author)]

You are right, thanks! Fixed in version 2.1.

And a question about compiling, if someone uses vc6 why not use the /MD switch as vc 6 still links to the system crt msvcrt.dll and doesn't need newer msvcrt??.dll

The plugin is just 92k, so it's not really worth the troubles of dll hell...

[karlchen]

Hello, Christian.

Fixed in version 2.1.

Tested and confirmed. This was a fast fix. Thanks a lot.

Karl

[ghisler(Author)]

Thanks for confirming the fix! Unfortunately I forgot to digitally sign the plugin, so I uploaded an updated version yesterday.

[karlchen]

LZMA v2.1, digitally signed edition, seems to work fine still: x86 and x64.

[tbeu]

It would be nice if plugins have some version information (that can be shown in TC Plugins Manager).

[ghisler(Author)]

I'm sorry, I'm always forgetting to update them, so I prefer not to include them at all.

[iana]

Don't want to split hairs but the dll's have been updated,
for xz liblzma.dll is up to 5.0.4 (the included is 5.0.3) changelog:

5.0.4 (2012-06-22)

* liblzma:

- Fix lzma_index_init(). It could crash if memory allocation
failed.

- Fix the possibility of an incorrect LZMA_BUF_ERROR when a BCJ
filter is used and the application only provides exactly as
much output space as is the uncompressed size of the file.

- Fix a bug in doc/examples_old/xz_pipe_decompress.c. It didn't
check if the last call to lzma_code() really returned
LZMA_STREAM_END, which made the program think that truncated
files are valid.

- New example programs in doc/examples (old programs are now in
doc/examples_old). These have more comments and more detailed
error handling.

and the last stable version of TCMDLZMA.DLL (LZMA.DLL) is 9.09 (latest beta 9.22) while the included is 4.63
have there been changes to the dll call functions can we just rename Igor's dll and replace the one included with tc/(lzma plugin)?

update:
Replacing liblzma.dll for the xz plugin works fine,
but lzma.dll (TCMDLZMA.DLL) 9.22 has only 2 exported functions compared with 13 in the old 4.63 and simply replacing that dll won't work.

[ghisler(Author)]

liblzma.dll is lzma2 used by xz. tcmdlzma.dll is lzma1. It's a completely different format with different functions. I had to export various other functions to make it work in TC and the lzma plugin.

[iana]

found a bug in the lzma plugin, files compressed with the -f86 switch can not be unpacked.
to reproduce the problem, pack a pe file with the lzma command line utility using the -f86 switch
for example the unrar.dll that comes with tc
>lzma e unrar.dll unrar.dll.lzma -f86
the lzma plugin will not unpack this lzma file while the 7-zip file manager will.

Found this by accident while playing around with the sumatrapdf code, they use lzma to store the binaries in their installer and I found that tc would not unpack these files.
this is not a big deal as lzma with filters hasn't caught on but still a bug is a bug.

[ghisler(Author)]

I guess that this particular compression method is simply not supported, sorry.

[Dalai]

Not sure whether or not this is a bug, but I thought I'd report it anyway because it may be causing serious issues in certain scenarios.

Here are the steps to reproduce:

1. Install the current XZ plugin if not done so already (https://www.ghisler.com/plugins.htm)
2. Download this file (~ 77 MiB)
3. Open the downloaded file in TC (Enter or Ctrl+PageDown)
4. Look at the size of the linux-3.16.7.tar file: it will (most likely) say 176,144,384 bytes.
5. Extract the linux-3.16.7.tar file to any directory
6. Check the size of the extracted file. It will say 578,805,760 bytes

The current stable 7-zip (19.00) shows the correct size of the compressed .tar file as does the XZ command line tool (https://tukaani.org/xz/).

This is reproducible in any TC version, 32-bit and 64-bit. Updating the liblzma.dll with the one from the command line tool doesn't change anything. In fact, it's possible to delete that DLL and the plugin still lists .xz archive contents (haven't tried extracting because that wasn't the point).

Regards
Dalai

PS: The file format (and thus the plugin) really needs a new name . It's impossible to search for it in this forum. I needed to make use of an external search engine to find this thread.

[ghisler(Author)]

When I wrote this plugin, xz did not store the size of the internal packed file. Therefore I'm just reporting the size of the archive, for the progress dialog when unpacking. Maybe a newer revision of the xz format supports this now...

[Dalai]

Therefore I'm just reporting the size of the archive, for the progress dialog when unpacking.

That I don't understand. The archive size is around 77 MiB, the file inside the archive is given with around 168 MiB and the packed file's real size is around 552 MiB. Where does the 168 MiB value come from?

Maybe a newer revision of the xz format supports this now...

Do you intend to look into this? It's annoying when it's not possible to see the real size of the packed file. Just today I had it again when I checked some older Linux kernel versions (4.6 up to 4.8.1) and none of them showed the correct (real) size of the packed file. Note that the plugin also doesn't show time and date for newer .xz archives while it does for older ones - I suppose there have been changes to the .xz specification.

Regards
Dalai

[ghisler(Author)]

You are right, I have now checked the file, and the problem is elsewhere: The file contains multiple .xz files concatenated to each other. This is valid, but the size reported is just that of one of the parts. You can check this with F3, then search for:
7zXZ