New packer plugin with web server counterpart

[ctiberg]

Hello!

This might take some explaining, but the rationale behind my idea is simple Using FTP to upload or download a very large amount of small files to/from a server can take forever and a day. I solved this problem a while ago by writing myself an packer/unpacker in PHP.

I'm now busy making a TC plugin that can make and read such archives. I post this here now because I'd like to get your input:

• Is this something you'd find useful?
• Anyone willing to do a server side thingy for other languages except PHP? I'd love to have one for .NET and one for servlet environments

[AlleyKat]

Yes, I'd indeed find that very useful, and I'm sure I'm not the only one. Some info on server requirements would be useful, just so I can check up on what I have and what I need.

I'm for one only interested in PHP. I imagine this being against PHP with zip gzip and/or bz2 modules and pack-the files-yourself-in-tc (I guess that would be a filesystem plugin) or 'directly' in a packer plugin...?

Options to upload either via (secure?) web interface or (secure?) FTP would be great. I'm using a rather weak system to this effect atm. uploading via a small file handling script with zip unpacking - it seriously lacks real control over where on the site I want files unpacked.

I'm looking forward to seeing this solution, even in its beta stages.

[ctiberg]

Well, the server part only requires standard PHP as yet. And it's dead simple, it uses gzcompress and gzuncompress for the actual compression.

I've used FTP upload to the server as yet, since the server we're using has a really tiny upload limit. I simply upload the file with FTP, and then run the decompression script on that file. No security whatever for the moment, I'm afraid

And the TC part is only a packer plugin as yet, since that's what I thought most useful...

[AlleyKat]

Sounds really good. About security I primarily meant HTTPS and SFTP (FTP over SSL). If its just a webscript its easy to add my own security either in php or via Apache.

[ctiberg]

That's what I've used, yes, and that's also why I didn' t add any in the scripts themselves

The server part mainly consists of one class file - that's what's doing the compress/uncompress. Then there are two dead simple wrappers to compress/decompress entire directories, but one could just chuck them and roll ones own

The server part is done, and most of the TC plugin too.. I'm currently getting AV's when calling ProcessDataProc, but I guess I'll be able to figure that out sometime soon...

[ghisler(Author)]

I'm currently getting AV's when calling ProcessDataProc, but I guess I'll be able to figure that out sometime soon...

The function is probably not declared correctly, it must be declared as stdcall.

Btw, please make sure that you don't have any buffer overflow vulnerabilities in the server! There should also be some user name/password authentication.

[ctiberg]

The function is probably not declared correctly, it must be declared as stdcall.

No, that wasn't it this time - it was even simpler - my first version was thus:

procedure SetProcessDataProc(hArcData: THandle; var ProcessDataProc: TProcessDataProc); stdcall; export;

When I turned it into this, it worked:

procedure SetProcessDataProc(hArcData: THandle; ProcessDataProc: TProcessDataProc); stdcall; export;

Btw, please make sure that you don't have any buffer overflow vulnerabilities in the server! There should also be some user name/password authentication.

As I said in another message, the server side code has no security at all - I've been using a .htaccess apache file for that.

And the server doesn't get the file to be unpacked by HTTP, since my (and probably other) web server has a ridiculously low upload limit. It reads the file to be unpacked from local disk, so one has to upload the file using FTP or some other means and then run the script to unpack it.

I've already release the first beta, as per this post: http://www.ghisler.ch/board/viewtopic.php?t=8130

[ctiberg]

Btw, does anyone know if gzcompress (or other gzip support) is available in .NET? I might take this chance to experiment a little with it if so, perhaps do an ASP.NET page for the packer plugin

But I'd vastly prefer if someone else did it, because then I'd have someone else to blame for it not working