4 ms·

I think for the common use case of distributing giga bytes of data in a reasonably efficient way, 7zip is fine (e.g. simheaven.com uses that for huge scenery do

I think for the common use case of distributing giga bytes of data in a reasonably efficient way, 7zip is fine (e.g. simheaven.com uses that for huge scenery downloads). Where things get tricky is splitting the resulting archive in multiple files. This used to be common with rar and I think 7zip also supports this. That is convenient when downloading over a slow/unstable connection (or if you don't know how to resume an aborted download with wget or ftp). Less of an issue these days but of course browsers don't support this. With zip files this also used to be a thing that some tools supported. I'm old enough to remember PKZip being a new thing :-). For as an actual backup format, tar is a common alternative. I'd probably use tar with bzip2, 7zip, or even gzip compression. It supports a few more compression algorithms of course. The main downside is that tar files are less common on windows and the whole double file extension thing confuses people with file managers that hide them (i.e. most users on windows or mac). Of course most of the archive/zip ui tools out there would support tar files anyway. Where it gets weird of course is with things like file permissions and other linux specific stuff like user and group ids. It's the reason zip is so popular, it doesn't support that at all. Tar is a bit over engineered for the simple use cases.

noAnswer 5y ago
The reason I'm a RAR fan is because TAR failed me multiple times! (The first one was ~21 years ago. I lost most of my home.tar.gz-Backup. Because tar has no index, if something bad comes along, everything after that can't be read.) 17 years ago, I took a deep dive into rar and replaced my tar.gz backup process with it. I had a cronjob that checked the tar.gz-Backups from the past week and there was a 50% change of "Unexpected EOF in archive". There where weeks, when all seven archives had and error! Disk, NFS, USB-Stick didn't mater. Also I wanted encryption. Which means another pipe. I have also seen bit flips on USB-Sticks. Where a copied jpg had reversed colours in the middle. So I wanted redundancy too. RAR has everything included, and I felt it had a low bus factor. Time moves on. 7zip feature creeps towards rar. Most if not all consumer OSs can open and create .zip out of the box. So, who cares ;-) Even I use Borg for my backups these days, since I don't have to care about a bus factor any more. RAR is still cool and my 17 year old licence is still valid.
- kiririn 5y ago
  It’s gzip that caused your issue, not tar. The lack of index in tar is actually a strength for surviving corruption, as there are no critical areas, ie you can lose any parts of a tar archive and recover useful data from the rest Tar.gz and tar.xz should never have been made a thing or become so popular, the only safe options are tar, tar.lz and to an extent tar.bz2
mmis1000 5y ago
No, the major problem of tar is 'it is not indexable'. In any case, browse a tar means you need to scan the whole achieve to locate all files and generate the entry list. And of course it get slower when the files are a lot or archive is big. This is a format even worse than zip.(Zip do have a central record section) And it also suffer the same problem from zip: the charset is undefined. Because they are older then the time utf-8 being the universal standard. Which means it mess up when two systems are in different charset and software is not smart enough to guess it. These problems are all addressed in later format(rar/7z). Newer archive format most enforcing utf-8 charset. So you won't have a problem that file name suddenly turn into garbage for no reason. It's probably the worst format you can chose if you need to distribute a file instead using it as a pack/extract on local only format(not even browse).
- duskwuff 5y ago
  Technically speaking, TAR is indexable -- the index is distributed through the archive, at the head of each file. So long as the TAR is uncompressed, you can skip from one header to the next by reading the size of each file. It ends up looking like a bunch of random I/O, but it's manageable. (The headers are even sector-aligned!) The problem arises when the TAR is compressed with a stream compressor (as most are). Since stream compression formats generally aren't seekable, there's no way to read each header without uncompressing the rest of the file in the process. > And it also suffer the same problem from zip: the charset is undefined. Newer archive format most enforcing utf-8 charset. On the flip side, this means that those archive formats can't archive the contents of a filesystem which contains inconsistently encoded (or flat-out nonsense) filenames. So long as filesystems haven't "solved" the encoding problem, I wouldn't fault an archive format for behaving similarly.
  - mmis1000 5y ago
    > Technically speaking, TAR is indexable -- the index is distributed through the archive, at the head of each file. Actually, zip works in the same way as tar. So the central dict is locate at the end of file because it can only be decided after all works finished. And zip actually struct the first part of file with file info before every block in layout similar to tar, so stream decoding is possible. (Although it IS a violation of specification to decode file based on per file info instead of central record in zip standard) > On the flip side, this means that those archive formats can't archive the contents of a filesystem which contains inconsistently encoded (or flat-out nonsense) filenames. It is a good thing IMO because you can't contribute to the problem more now (at the price not able to backup already screwed up disks. Well..., at least old format(tar) still works on old disk)

noAnswer 5y ago

The reason I'm a RAR fan is because TAR failed me multiple times! (The first one was ~21 years ago. I lost most of my home.tar.gz-Backup. Because tar has no index, if something bad comes along, everything after that can't be read.) 17 years ago, I took a deep dive into rar and replaced my tar.gz backup process with it. I had a cronjob that checked the tar.gz-Backups from the past week and there was a 50% change of "Unexpected EOF in archive". There where weeks, when all seven archives had and error! Disk, NFS, USB-Stick didn't mater. Also I wanted encryption. Which means another pipe. I have also seen bit flips on USB-Sticks. Where a copied jpg had reversed colours in the middle. So I wanted redundancy too. RAR has everything included, and I felt it had a low bus factor. Time moves on. 7zip feature creeps towards rar. Most if not all consumer OSs can open and create .zip out of the box. So, who cares ;-) Even I use Borg for my backups these days, since I don't have to care about a bus factor any more. RAR is still cool and my 17 year old licence is still valid.

kiririn 5y ago
It’s gzip that caused your issue, not tar. The lack of index in tar is actually a strength for surviving corruption, as there are no critical areas, ie you can lose any parts of a tar archive and recover useful data from the rest Tar.gz and tar.xz should never have been made a thing or become so popular, the only safe options are tar, tar.lz and to an extent tar.bz2

mmis1000 5y ago

No, the major problem of tar is 'it is not indexable'. In any case, browse a tar means you need to scan the whole achieve to locate all files and generate the entry list. And of course it get slower when the files are a lot or archive is big. This is a format even worse than zip.(Zip do have a central record section) And it also suffer the same problem from zip: the charset is undefined. Because they are older then the time utf-8 being the universal standard. Which means it mess up when two systems are in different charset and software is not smart enough to guess it. These problems are all addressed in later format(rar/7z). Newer archive format most enforcing utf-8 charset. So you won't have a problem that file name suddenly turn into garbage for no reason. It's probably the worst format you can chose if you need to distribute a file instead using it as a pack/extract on local only format(not even browse).

duskwuff 5y ago
Technically speaking, TAR is indexable -- the index is distributed through the archive, at the head of each file. So long as the TAR is uncompressed, you can skip from one header to the next by reading the size of each file. It ends up looking like a bunch of random I/O, but it's manageable. (The headers are even sector-aligned!) The problem arises when the TAR is compressed with a stream compressor (as most are). Since stream compression formats generally aren't seekable, there's no way to read each header without uncompressing the rest of the file in the process. > And it also suffer the same problem from zip: the charset is undefined. Newer archive format most enforcing utf-8 charset. On the flip side, this means that those archive formats can't archive the contents of a filesystem which contains inconsistently encoded (or flat-out nonsense) filenames. So long as filesystems haven't "solved" the encoding problem, I wouldn't fault an archive format for behaving similarly.
- mmis1000 5y ago
  > Technically speaking, TAR is indexable -- the index is distributed through the archive, at the head of each file. Actually, zip works in the same way as tar. So the central dict is locate at the end of file because it can only be decided after all works finished. And zip actually struct the first part of file with file info before every block in layout similar to tar, so stream decoding is possible. (Although it IS a violation of specification to decode file based on per file info instead of central record in zip standard) > On the flip side, this means that those archive formats can't archive the contents of a filesystem which contains inconsistently encoded (or flat-out nonsense) filenames. It is a good thing IMO because you can't contribute to the problem more now (at the price not able to backup already screwed up disks. Well..., at least old format(tar) still works on old disk)