4 ms·

I’ve always been confused by the popular choice of compression on tar archives. If a small portion of a gz, xz, zst, etc archive is corrupted you lose everythin

I’ve always been confused by the popular choice of compression on tar archives. If a small portion of a gz, xz, zst, etc archive is corrupted you lose everything after it, defeating the purpose of using a stream format like Tar. The algorithms that can recover data after the corruption like bzip2 and lzip don’t see much use these days

__s 6y ago
Two reasons: - if you know tar supports your directory / metadata, you don't have to lookup if zst will preserve that info - interfile compression I don't think people would say they use tar because it's a stream format. They use it because it packages files/directories into a file
- zeepzeep 6y ago
  I only use it for keeping file permissions
dTal 6y ago
99.9% of the time, if a .tar.gz archive is corrupted, I don't want to recover the data - I want to redownload it from a different mirror because it's source code and it probably won't work with any corruption at all.
- TonyTrapp 6y ago
  Sometimes you have no choice. Downloading historic archives of which there is no other copy, trying to recover of it as much as possible.
evil-olive 6y ago
In the common case of receiving a compressed tar over the network, it should hopefully be protected by at least a checksum or hash, if not a cryptographic signature. If I expect I may have to deal with some form of data corruption (such as long-term archival media), I'd rather add a layer such as PAR2 [0]. Rather than content myself with "if the tar file is corrupted, I can probably recover most of the contents", erasure coding gives me confidence that I can definitely recover all of the contents, with confidence they're not corrupted, as long as I have enough parity data. 0: https://en.wikipedia.org/wiki/Parchive https://en.wikipedia.org/wiki/Parchive
qalmakka 6y ago
Nobody uses Tar like that anymore. Tar is used as a more flexible replacement for ZIP nowadays, it could have easily been CPIO but `tar` had a much better command line interface.

__s 6y ago

Two reasons: - if you know tar supports your directory / metadata, you don't have to lookup if zst will preserve that info - interfile compression I don't think people would say they use tar because it's a stream format. They use it because it packages files/directories into a file

zeepzeep 6y ago
I only use it for keeping file permissions

dTal 6y ago

99.9% of the time, if a .tar.gz archive is corrupted, I don't want to recover the data - I want to redownload it from a different mirror because it's source code and it probably won't work with any corruption at all.

TonyTrapp 6y ago
Sometimes you have no choice. Downloading historic archives of which there is no other copy, trying to recover of it as much as possible.

evil-olive 6y ago

In the common case of receiving a compressed tar over the network, it should hopefully be protected by at least a checksum or hash, if not a cryptographic signature. If I expect I may have to deal with some form of data corruption (such as long-term archival media), I'd rather add a layer such as PAR2 [0]. Rather than content myself with "if the tar file is corrupted, I can probably recover most of the contents", erasure coding gives me confidence that I can definitely recover all of the contents, with confidence they're not corrupted, as long as I have enough parity data. 0: https://en.wikipedia.org/wiki/Parchive https://en.wikipedia.org/wiki/Parchive

qalmakka 6y ago

Nobody uses Tar like that anymore. Tar is used as a more flexible replacement for ZIP nowadays, it could have easily been CPIO but `tar` had a much better command line interface.