WCX: CloseArchive and progress dialog

[milo1012]

I'm currently experimenting with a WCX plug-in for ZPAQ.

Unfortunately the same problem as with solid 7-Zip or Rar happens:
I can't and don't want to unpack each file in ProcessFile(), as this would read parts of the archive all over again and'd be horribly slow.
Therefore I collect all files to unpack and decompress them all in CloseArchive(), just like Christian suggested here.


Since ZPAQ is slow, I want to be able to abort operations and display progress via ProcessDataProc. Unfortunately it doesn't work in CloseArchive().
From the WCX interface help file:

Set Size to the number of bytes processed since the previous call to the function. For plugins which unpack in CloseArchive: Set size to negative percent value (-1..-100) to directly set first percent bar, -1000..-1100 for second percent bar (-1000=0%).

But it just doesn't work, no matter if I return a zero or negative value, and/or some valid custom string!
And yes, calling the function in ProcessFile() works (same pointer address).


So my questions are:

a) Why does the Callback function not work for me in CloseArchive, despite the documentation? What are the limiting factors? (I'm using the wide/Unicode function)

b) What is the reason that I can't send the progress dialog to background? (yes, BACKGROUND_UNPACK is set)
My impression is that I need to call ProcessDataProc at least once, but: see a)

c) Is there any special reason why there is STILL a -1 (or 0xFFFFFFFF) returned as Handle address in SetProcessDataProc(W)() when not using list or unpack?
See here (that thread is nearly 12(!) years old now).
It also took me nearly an hour until I found this in debug mode, and the wcx guide STILL says nothing about it.
(sorry, but you either use a null pointer or a valid address, but definitely not 0xFFFFFFFF)

[Lefteous]

2milo1012

I can't and don't want to unpack each file in ProcessFile(), as this would read parts of the archive all over again and'd be horribly slow.
Therefore I collect all files to unpack and decompress them all in CloseArchive()

From my point of view this is the main problem of the packer interface. It doesn't support these kind of unpackers properly. So we all have to use this workaround which then requires more workarounds...

So an alternate way of unpacking files would be a new function 'UnpackFiles'.

[ghisler(Author)]

This should definitely work - but you will probably have to call the message loop because you are running on the user interface thread:

while PeekMessage(&msg,0,0,0,PM_REMOVE) do begin
windows.TranslateMessage(&msg);
windows.DispatchMessage(&msg);
end;

And you are right, the interface does not work well with archive libraries where the list of names has to be provided before the unpacking even starts.

[milo1012]

while PeekMessage...

I'm not really comfortable in doing such things by myself.
I'll see if it helps though.

but you will probably have to call the message loop because you are running on the user interface thread

Is this the only explanation why it works in ProcessFile(), but not CloseArchive()?
So would it work if I start my own thread?
(I'll try that, as soon as I have time, but I don't think that an GUI update works from a different thread)

There still remains the questions why I can't send the progress dialog to background.

And BTW, I could name a few more ambiguous explanations in the documentation.
Maybe it's time to overhaul it and make it more clearly, so that not every plugin author needs to find the very same flaws on his own.

[milo1012]

Seems I found the culprit:
ZPAQ was silently creating threads where I didn't expect them, and so of course calling a GUI update in them didn't work.
The progress dialog works as expected now.
I need to collect and show errors for individual files by myself though.

But another thing regarding unpacking in CloseArchive():
Why isn't there any error message triggered when I return an error code?
I tried all of them, but none showed any message box.
So where is the point in returning them at all?
That's one of the ambiguous explanations in the wcx guide I'm talking about...

And one last question: what does TC officially expect for identifying directories in ReadHeader()?
A name with trailing backslash or without?
Normally it should be no problem, because I'd have and set the dir attribute anyway,
but I'm not so sure about some custom archives.

[MVV]

And one last question: what does TC officially expect for identifying directories in ReadHeader()?
A name with trailing backslash or without?
Normally it should be no problem, because I'd have and set the dir attribute anyway,
but I'm not so sure about some custom archives.

Of course only Directory flag for FileAtt field, this is the only flag describing item as a directory. FileName must not have any slashes.

[milo1012]

FileName must not have any slashes.

Sure, if you don't provide a backslash you need to set the attribute at least.
But in PackFiles() TC requests directories with a backslash itself.
So it's not that clear.

I didn't do full tests yet, but according to this it seems that they are expected.

I just want to hear what Christian expects or recommends,
since I might not be able to identify dirs by attributes when an archive was created on Unix machines

[MVV]

But in PackFiles() TC requests directories with a backslash itself.
So it's not that clear.

Because TC can only pass paths without attributes, so it adds backslashes for directories.

I didn't do full tests yet, but according to this it seems that they are expected.

I checked Multiarc in debugger, it doesn't bother with backslashes, it returns directories both with and without them... and it works fine because of attribute flag.

I might not be able to identify dirs by attributes when an archive was created on Unix machines

In such case there is a problem with detecting directories in your archive format but not with telling to TC that it is a directory.

[milo1012]

Because TC can only pass paths without attributes, so it adds backslashes for directories.

TC refers to an existing object with a valid path, so I could identify myself what it means. (GetFileAttributes et. al.)
Windows doesn't force you to identify dirs by a backslash, but works either way.

In such case there is a problem with detecting directories in your archive format but not with telling to TC that it is a directory.

I meant that I can't just copy the attribute DWORD, like with normal archives, where I don't need to bother further.
But of course I still know that it's a dir by the INTERNAL trailing backslash.
So I just wanted to know what Christian expects in such case (both attribute and backslash, or one thing only).

[MVV]

Why not just to check for backslash and set attribute flag?

[milo1012]

Why not just to check for backslash and set attribute flag?

For the third time:
I just want to know if it's really expected.
Mainly because it's not clearly documented.

Sure I could do it anyway and don't bother.

[milo1012]

2Ghisler
Since you didn't answer, I will ask again:

a)
Is it normal that CloseArchive() doesn't show error messages at all?
Any workarounds for this?

b)
What is your recommendation for identifying dirs: both attribute and backslash set, or one thing only ?

[ghisler(Author)]

a) Yes, errors returned from CloseArchive are ignored.

b) Recently I had a case with invlid ZIP files where the attributes were set to 0xFF (255) also for files. Therefore I would recommend this:
1. Attributes=255 -> ignore, go to 3
2. Attributes has directory bit set -> directory
3. Name has trailing slash or backslash -> directory
4. All others -> file

[milo1012]

a) Yes, errors returned from CloseArchive are ignored.

I see.
So I'd need to show my own MessageBox(), which would require translation with my own routines at some point.

Therefore I would recommend this...

Thanks. So I assume it doesn't matter to TC if I set either the trailing backslash or the (correct) dir attribute in ReadHeader(),
it will still identify that entry as a dir ?

[ghisler(Author)]

I recommend that you also set the attribute. My suggestion was just how your plugin should recognize files and folders when the archive data is unclear.