7 ms·

Which is the consensus about better alternatives?

by Aldo_MX 10y ago

pkulak 10y ago
tar.gz ?
- Aldo_MX 10y ago
  tar.gz is not good with random access
  - duskwuff 10y ago
    "Not good" is an understatement. tar.gz is maximally bad at random access -- there is no way to uncompress a single file, or to list files in the archive, without decompressing the entire gzip stream.
    - Zardoz84 10y ago
      and gzip every file and tar it ? ie gz.tar
      - duskwuff 10y ago
        Gzipping each file individually makes the compressor have to start over with a fresh context on each file. This will result in a much worse compression ratio than if a single compressor is used for the whole archive, especially if the archive contains a lot of small, similar files. (Note that this is essentially what PKZIP does, though, so it's not awful.) Also, there are no tools I'm aware of that will handle that archive structure.
        mjevans 10y ago
        While a 'solid' archive wouldn't typically have the compression penalty or the 'listing' penalty it would still have the extraction penalty.
  - cwmma 10y ago
    But it's much better at streaming reads then zip so depending on the context might be better.
teilo 10y ago
DAR. GPL, indexed, file permissions, ACLs, alternate file streams, per-file compression, slices, hard and symbolic links, sparse files, encryption, parity records for recovery. Everything you could want. Not sure why it hasn't had much use. Maybe because it is usually thought of as a backup tool (at which it is excellent). But it is also an excellent archival tool. From the man page: "Dar can also use a sequential reading mode, in which dar acts like tar, just reading byte by byte the whole archive to know its contents and eventually extracting file at each step. In other words, the archive contents is located at both locations, all along the archive used for tar-like behavior suitable for sequential access media (tapes) and at the end for faster access, suitable for random access media (disks). ... Note also that the sequential reading mode let you extract data from a partially written archive (those that failed to complete due to a lack of disk space for example)." http://dar.linux.free.fr http://dar.linux.free.fr
- EpicEng 10y ago
  Yeah, and then you send your DAR file to Sally in accounting, who can't open it, and you're back to ZIP.
  - teilo 10y ago
    True of any alternative and thus irrelevant to the question I was answering.
- witty_username 10y ago
  From http://dar.linux.free.fr http://dar.linux.free.fr > Consequently, to use the API, your program must be released under the GPL as well. GPL is a bad thing for programs like this. It hampers usability. Especially at least the decoder should be BSD/MIT/etc.
  - teilo 10y ago
    A fair point. Lacking good GUI tools, integration libraries, etc., something like this will never gain traction. This looks to be a fairly complex code base. Re-implementing it in a library licensed under BSD/MIT would be non-trivial.
niftich 10y ago
There is no consensus about alternatives. There are plenty of open container file formats that can contain multiple files, random-access them, and optionally compress them -- with some, but not all, supporting filesystem attributes: CFS, 7z, PEA, ZPAQ, and as another commenter writes, DAR. A combined 'contain & compress' format is perhaps at odds with the unix philosophy of tools doing one thing well. Meanwhile, zip definitely exemplifies 'worse is better', in that despite the format's quirks and darker corners, it's good enough for most casual uses, and network effects and backwards compatibility help it win out.
user5994461 10y ago
Define "better". If you just want to compress some stuff, zip is perfectly fine and widely supported. (or gzip or tar) If you want fancy things with better compression, user rights/ACL (either linux or windows style), file dates, encryption, password protection. Well, I don't know what can do that.
- optionalparens 10y ago
  I'd also add compression time and decompression time to that list. Zip wasn't actually the best compression algorithm I remember seeing in its original days, it just was the best compromise of speed of compression, extraction, and compressed size. The TOC feature was a killer because even the fastest formats decompressed slow, not to mention space was more of a premium. An additional feature that sometimes is important as well is making itself stream friendly so you can read the metadata without the entire file (or easily before streaming) as it is streamed over some context or using some technology. The landscape is much different now and so some of features you'd design into a new format didn't exist then, while others that were concerns are less concerns now. As parent hints, the compatibility alone makes zip still very desirable and actual compression is good enough given size of disks today for most people. Obviously there are exceptions.
wolfgang42 10y ago
The main problem I see with any alternatives is that ZIP is the only format that seems to work everywhere. On NIX-type systems you also have tar+gzip, and some Windows users have something that can open RAR files, but if you want to send an archive and know that the other party will be able to open it without hassle ZIP is the only way to go. Unfortunately this is impossible to change on any reasonable time scale (less than a decade) since everyone needs to have the archiving tools immediately to hand.
LeoPanthera 10y ago
My favorite archive format is lzip. It's libre and extremely well documented, and there is a Public Domain unarchiver. And it uses modern LZMA compression. http://www.nongnu.org/lzip/lzip.html http://www.nongnu.org/lzip/lzip.html
aidenn0 10y ago
7z and rar are probably the most widely available alternatives today (The 7zip GUI utility on windows can decompress RAR files, while I don't think WinRAR can handle 7z files, so RAR is strictly more available than 7z on windows). I personally also like ZPAQ for generating archives, though I find it a very poor fit for its suggested use as a backup tool (doesn't handle symlinks, owner, xattr, ACL &c.). As an added bonus the reference implementation is completely composed of public domain and MIT/X11 licensed code, which makes it friendly for linking pretty much anywhere.
cwmma 10y ago
For a lot of things zip files are used for tar.gz isn't just an alternative but what you should have been using all along as you can do streaming reads, that being said it dosn't have as good support and can't do random access from disk like zip can so it depends on what you are trying to do.

Which is the consensus about better alternatives?

by Aldo_MX 10y ago

Which is the consensus about better alternatives?

pkulak 10y ago

tar.gz ?

Aldo_MX 10y ago
tar.gz is not good with random access
- duskwuff 10y ago
  "Not good" is an understatement. tar.gz is maximally bad at random access -- there is no way to uncompress a single file, or to list files in the archive, without decompressing the entire gzip stream.
  - Zardoz84 10y ago
    and gzip every file and tar it ? ie gz.tar
    - duskwuff 10y ago
      Gzipping each file individually makes the compressor have to start over with a fresh context on each file. This will result in a much worse compression ratio than if a single compressor is used for the whole archive, especially if the archive contains a lot of small, similar files. (Note that this is essentially what PKZIP does, though, so it's not awful.) Also, there are no tools I'm aware of that will handle that archive structure.
      - mjevans 10y ago
        While a 'solid' archive wouldn't typically have the compression penalty or the 'listing' penalty it would still have the extraction penalty.
- cwmma 10y ago
  But it's much better at streaming reads then zip so depending on the context might be better.

teilo 10y ago

DAR. GPL, indexed, file permissions, ACLs, alternate file streams, per-file compression, slices, hard and symbolic links, sparse files, encryption, parity records for recovery. Everything you could want. Not sure why it hasn't had much use. Maybe because it is usually thought of as a backup tool (at which it is excellent). But it is also an excellent archival tool. From the man page: "Dar can also use a sequential reading mode, in which dar acts like tar, just reading byte by byte the whole archive to know its contents and eventually extracting file at each step. In other words, the archive contents is located at both locations, all along the archive used for tar-like behavior suitable for sequential access media (tapes) and at the end for faster access, suitable for random access media (disks). ... Note also that the sequential reading mode let you extract data from a partially written archive (those that failed to complete due to a lack of disk space for example)." http://dar.linux.free.fr http://dar.linux.free.fr

EpicEng 10y ago
Yeah, and then you send your DAR file to Sally in accounting, who can't open it, and you're back to ZIP.
- teilo 10y ago
  True of any alternative and thus irrelevant to the question I was answering.
witty_username 10y ago
From http://dar.linux.free.fr http://dar.linux.free.fr > Consequently, to use the API, your program must be released under the GPL as well. GPL is a bad thing for programs like this. It hampers usability. Especially at least the decoder should be BSD/MIT/etc.
- teilo 10y ago
  A fair point. Lacking good GUI tools, integration libraries, etc., something like this will never gain traction. This looks to be a fairly complex code base. Re-implementing it in a library licensed under BSD/MIT would be non-trivial.

niftich 10y ago

There is no consensus about alternatives. There are plenty of open container file formats that can contain multiple files, random-access them, and optionally compress them -- with some, but not all, supporting filesystem attributes: CFS, 7z, PEA, ZPAQ, and as another commenter writes, DAR. A combined 'contain & compress' format is perhaps at odds with the unix philosophy of tools doing one thing well. Meanwhile, zip definitely exemplifies 'worse is better', in that despite the format's quirks and darker corners, it's good enough for most casual uses, and network effects and backwards compatibility help it win out.

user5994461 10y ago

Define "better". If you just want to compress some stuff, zip is perfectly fine and widely supported. (or gzip or tar) If you want fancy things with better compression, user rights/ACL (either linux or windows style), file dates, encryption, password protection. Well, I don't know what can do that.

optionalparens 10y ago
I'd also add compression time and decompression time to that list. Zip wasn't actually the best compression algorithm I remember seeing in its original days, it just was the best compromise of speed of compression, extraction, and compressed size. The TOC feature was a killer because even the fastest formats decompressed slow, not to mention space was more of a premium. An additional feature that sometimes is important as well is making itself stream friendly so you can read the metadata without the entire file (or easily before streaming) as it is streamed over some context or using some technology. The landscape is much different now and so some of features you'd design into a new format didn't exist then, while others that were concerns are less concerns now. As parent hints, the compatibility alone makes zip still very desirable and actual compression is good enough given size of disks today for most people. Obviously there are exceptions.

wolfgang42 10y ago

The main problem I see with any alternatives is that ZIP is the only format that seems to work everywhere. On NIX-type systems you also have tar+gzip, and some Windows users have something that can open RAR files, but if you want to send an archive and know that the other party will be able to open it without hassle ZIP is the only way to go. Unfortunately this is impossible to change on any reasonable time scale (less than a decade) since everyone needs to have the archiving tools immediately to hand.

LeoPanthera 10y ago

My favorite archive format is lzip. It's libre and extremely well documented, and there is a Public Domain unarchiver. And it uses modern LZMA compression. http://www.nongnu.org/lzip/lzip.html http://www.nongnu.org/lzip/lzip.html

aidenn0 10y ago

7z and rar are probably the most widely available alternatives today (The 7zip GUI utility on windows can decompress RAR files, while I don't think WinRAR can handle 7z files, so RAR is strictly more available than 7z on windows). I personally also like ZPAQ for generating archives, though I find it a very poor fit for its suggested use as a backup tool (doesn't handle symlinks, owner, xattr, ACL &c.). As an added bonus the reference implementation is completely composed of public domain and MIT/X11 licensed code, which makes it friendly for linking pretty much anywhere.

cwmma 10y ago

For a lot of things zip files are used for tar.gz isn't just an alternative but what you should have been using all along as you can do streaming reads, that being said it dosn't have as good support and can't do random access from disk like zip can so it depends on what you are trying to do.