3 ms·

I'm curious because I'm trying to convince people to switch from .rar to .tar.gz at work, wonder if you know the benefits, what does it mean that one is a block

I'm curious because I'm trying to convince people to switch from .rar to .tar.gz at work, wonder if you know the benefits, what does it mean that one is a block compressor?

chungy 7y ago
Generally speaking, there's two large categories of archivers in competition with one another: 1. Efficient random-access storage. If compression is supported, it is done on a per-member database. 2. Efficient all-at-once compression (block/solid compression). This is either handled explicitly by file format (RAR and 7z, for example), or the archive doesn't support compression (tar) and itself is processed through an external compressor. On the first category, I would argue Zip has clear superiority over most other file formats. It supports all metadata fields for both DOS/Windows and Unix operating systems, and has very wide support amongst everything. Zip does not support the second category of archival formats. In the second category, 7z (I don't know about RAR, but I suspect it holds true) only supports DOS attributes and modified dates, making it totally insufficient for backing up a lot of kinds of Unix file trees. tar is generally the de facto standard for Unix and naturally supports all metadata. I would argue gzip is long obsolete; zstd is much faster (it can nearly compete with xz too for file size). tar files don't really support random access for individual members, lacking a central directory, though uncompressed files can be seeked rather rapidly anyway as the tar program just skips over members (each member encodes its length). Compressed tar files make this worse, since the entire archive must be uncompressed in memory just to find anything. This is a large reason that tar is mostly restricted to being used when extracting individual members is not a generally useful operation. If that is a useful operation in your datasets, go for Zip.
- genpfault 7y ago
  RAR also has built-in forward error correction, something I sorely miss with 7z :(

I'm curious because I'm trying to convince people to switch from .rar to .tar.gz at work, wonder if you know the benefits, what does it mean that one is a block compressor?

chungy 7y ago

Generally speaking, there's two large categories of archivers in competition with one another: 1. Efficient random-access storage. If compression is supported, it is done on a per-member database. 2. Efficient all-at-once compression (block/solid compression). This is either handled explicitly by file format (RAR and 7z, for example), or the archive doesn't support compression (tar) and itself is processed through an external compressor. On the first category, I would argue Zip has clear superiority over most other file formats. It supports all metadata fields for both DOS/Windows and Unix operating systems, and has very wide support amongst everything. Zip does not support the second category of archival formats. In the second category, 7z (I don't know about RAR, but I suspect it holds true) only supports DOS attributes and modified dates, making it totally insufficient for backing up a lot of kinds of Unix file trees. tar is generally the de facto standard for Unix and naturally supports all metadata. I would argue gzip is long obsolete; zstd is much faster (it can nearly compete with xz too for file size). tar files don't really support random access for individual members, lacking a central directory, though uncompressed files can be seeked rather rapidly anyway as the tar program just skips over members (each member encodes its length). Compressed tar files make this worse, since the entire archive must be uncompressed in memory just to find anything. This is a large reason that tar is mostly restricted to being used when extracting individual members is not a generally useful operation. If that is a useful operation in your datasets, go for Zip.

genpfault 7y ago
RAR also has built-in forward error correction, something I sorely miss with 7z :(