3 ms·

I can't stop wondering (sincerely, not for a "holywar" sake or anything) why we still use tar.whatever today, when ~99% of us have never seen a tape drive. .ta

I can't stop wondering (sincerely, not for a "holywar" sake or anything) why we still use tar.whatever today, when ~99% of us have never seen a tape drive. .tar.gz felt such a wild step backwards after 7z when I switched from Windows to Linux. Why can't we just introduce links and access rights metadata to 7z or something and use a modern, feature-rich archive format which wouldn't require you to decompress everything just to get a clue if there is a particularly named file in there?

vbezhenar 6y ago
Because tar gz works on any unix and 7z requires installing additional software. Whether you want that or not depends on your use-case. It's a good example of "good enough" software which is hard to phase out. That said, recently I've found that CentOS 8 minimal did not have tar installed. So may be it's worth to start using something modern like zip or even 7z.
ekimekim 6y ago
The simplicity of the tar format has certain advantages, such as ensuring an implementation in a variety of languages and platforms. The seperation of "group files into an archive" and "compress these bytes" can also be very useful. For example, it means I can play around with the latest and greatest compression formats without needing to wait for a version of tar that has that format built-in. Overall, I think it appeals to the "unix philosophy" - composing two independent tools (tar and gzip) instead of having one integrated solution. The "tape drive" part is legacy cruft, certainly, but in practice its removal doesn't improve things enough that people find it worth the compatability break. I'm not going to claim this is better than the more modern feature-rich archive formats, but hopefully this helps you understand the other side of the argument a little better.
- lifthrasiir 6y ago
  > Overall, I think it appeals to the "unix philosophy" - composing two independent tools (tar and gzip) instead of having one integrated solution. The unix philosophy in this case breaks down when you need to access individual files. One possible "unixy" solution would be .gz.tar as opposed to .tar.gz, but it is not universally supported as .tar.gz is.
  - heavenlyblue 6y ago
    You can have the first file in the tar archive as the "index" file
- pornel 6y ago
  There's no simplicity in tar. Even reading the file size is a guesswork. It's a baroque format with weird short-sighted decisions patched and extended in ad-hoc ways by multiple subtly incompatible implementations.
  - klodolph 6y ago
    Unless you're using PaX, isn't the file size just a fixed-size field in the header? Is there something I'm missing?
    - Hello71 6y ago
      fixed... ish... with two sizes and two formats: https://dev.gentoo.org/~mgorny/articles/portability-of-tar-features.html#large-file-sizes https://dev.gentoo.org/~mgorny/articles/portability-of-tar-f...
- pdw 6y ago
  It's unix philosophy alright, but it's the other unix philosophy -- worse is better. tar has a poorly designed base format with lots of incompatible extensions, but for most people most of the time it works fine. So why change?
  - fomine3 6y ago
    Is tar "worse" in the context?
eggsnbacon1 6y ago
I use Peazip on Linux https://peazip.github.io/ https://peazip.github.io/ tar.gz is ancient
dexen 6y ago
Your question is essentially one of "general composabilitiy vs specialized tool". There's two (opposing) philosophical answers, and three (opposing) practical answers, and who am I to say which is better for you? For me, Tar + compression is optimal for most of my use cases - transfer over network, software package distribution, long term storage. In case of transferring over fast network, uncompressed Tar is pretty good [0], and Lz4 compressed is pretty fast too. In case of distributing software, XZ compression is optimal (expensive, good level for small projects). In case of long term storage, Rzip compression is optimal (very expensive, good level for large projects). Besides the pragmatic considerations, as a software developer & maintainer, I honestly enjoy composability. Funnily enough, just yesterday I've read the specification of SQLite Archive Files[1]. A pretty cool format and yet - my first question was, "Why is there no support for other compression methods? Like Lz4 for high speed, XZ for high compression?" Please note that, unlike with Tar, extending this standard is pretty much a no-go. Meanwhile a .tar.gz / .tar.xz / .tar.rz / .tar.lz4 is a self-documenting file. You know what to do, and it's easy to write a tool that does the right thing automatically. I've been implementing a toy project recently, and one of the network communication protocols in it has grown into a poor man's re-implementation of Tar. Having realized that just now, I'll probably yank out several dozen lines of code and replace them with one call to tar on each side. However crufty Tar it might have gotten over decades, it's certainly less buggy than my hand-rolled code. Lastly, Tar is a much richer format than one might ordinarily expect. For example it supports multiple versions of the same file[2] - and you can add newer version(s) incrementally (append-only file; a much safer operation than re-writting/replacing files). Think a poor man's Git/SQLite/Wiki/whatever. Self-contained, self-documenting, all very nice. -- [0] uncompressed Tar supports fast seeking. If you expect working with an archive for a while, just create an uncompressed copy. Or a text file index, we don't judge. [1] https://www.sqlite.org/sqlar.html https://www.sqlite.org/sqlar.html [2] https://www.gnu.org/software/tar/manual/html_node/multiple.html https://www.gnu.org/software/tar/manual/html_node/multiple.h...
- itroot 6y ago
  I recently impletented an extension to sqlite - https://github.com/itroot/sqlite-brotli-compress https://github.com/itroot/sqlite-brotli-compress - that adds ability to compress blobs to sqlite via brotli, and I think it is trivial to add any compression format to sqlar (with some format extension, of course). But sqlar I believe is not very popular, and just an example of how sqlite can be used.

vbezhenar 6y ago

Because tar gz works on any unix and 7z requires installing additional software. Whether you want that or not depends on your use-case. It's a good example of "good enough" software which is hard to phase out. That said, recently I've found that CentOS 8 minimal did not have tar installed. So may be it's worth to start using something modern like zip or even 7z.

ekimekim 6y ago

The simplicity of the tar format has certain advantages, such as ensuring an implementation in a variety of languages and platforms. The seperation of "group files into an archive" and "compress these bytes" can also be very useful. For example, it means I can play around with the latest and greatest compression formats without needing to wait for a version of tar that has that format built-in. Overall, I think it appeals to the "unix philosophy" - composing two independent tools (tar and gzip) instead of having one integrated solution. The "tape drive" part is legacy cruft, certainly, but in practice its removal doesn't improve things enough that people find it worth the compatability break. I'm not going to claim this is better than the more modern feature-rich archive formats, but hopefully this helps you understand the other side of the argument a little better.

lifthrasiir 6y ago
> Overall, I think it appeals to the "unix philosophy" - composing two independent tools (tar and gzip) instead of having one integrated solution. The unix philosophy in this case breaks down when you need to access individual files. One possible "unixy" solution would be .gz.tar as opposed to .tar.gz, but it is not universally supported as .tar.gz is.
- heavenlyblue 6y ago
  You can have the first file in the tar archive as the "index" file
pornel 6y ago
There's no simplicity in tar. Even reading the file size is a guesswork. It's a baroque format with weird short-sighted decisions patched and extended in ad-hoc ways by multiple subtly incompatible implementations.
- klodolph 6y ago
  Unless you're using PaX, isn't the file size just a fixed-size field in the header? Is there something I'm missing?
  - Hello71 6y ago
    fixed... ish... with two sizes and two formats: https://dev.gentoo.org/~mgorny/articles/portability-of-tar-features.html#large-file-sizes https://dev.gentoo.org/~mgorny/articles/portability-of-tar-f...
pdw 6y ago
It's unix philosophy alright, but it's the other unix philosophy -- worse is better. tar has a poorly designed base format with lots of incompatible extensions, but for most people most of the time it works fine. So why change?
- fomine3 6y ago
  Is tar "worse" in the context?

eggsnbacon1 6y ago

I use Peazip on Linux https://peazip.github.io/ https://peazip.github.io/ tar.gz is ancient

dexen 6y ago

Your question is essentially one of "general composabilitiy vs specialized tool". There's two (opposing) philosophical answers, and three (opposing) practical answers, and who am I to say which is better for you? For me, Tar + compression is optimal for most of my use cases - transfer over network, software package distribution, long term storage. In case of transferring over fast network, uncompressed Tar is pretty good [0], and Lz4 compressed is pretty fast too. In case of distributing software, XZ compression is optimal (expensive, good level for small projects). In case of long term storage, Rzip compression is optimal (very expensive, good level for large projects). Besides the pragmatic considerations, as a software developer & maintainer, I honestly enjoy composability. Funnily enough, just yesterday I've read the specification of SQLite Archive Files[1]. A pretty cool format and yet - my first question was, "Why is there no support for other compression methods? Like Lz4 for high speed, XZ for high compression?" Please note that, unlike with Tar, extending this standard is pretty much a no-go. Meanwhile a .tar.gz / .tar.xz / .tar.rz / .tar.lz4 is a self-documenting file. You know what to do, and it's easy to write a tool that does the right thing automatically. I've been implementing a toy project recently, and one of the network communication protocols in it has grown into a poor man's re-implementation of Tar. Having realized that just now, I'll probably yank out several dozen lines of code and replace them with one call to tar on each side. However crufty Tar it might have gotten over decades, it's certainly less buggy than my hand-rolled code. Lastly, Tar is a much richer format than one might ordinarily expect. For example it supports multiple versions of the same file[2] - and you can add newer version(s) incrementally (append-only file; a much safer operation than re-writting/replacing files). Think a poor man's Git/SQLite/Wiki/whatever. Self-contained, self-documenting, all very nice. -- [0] uncompressed Tar supports fast seeking. If you expect working with an archive for a while, just create an uncompressed copy. Or a text file index, we don't judge. [1] https://www.sqlite.org/sqlar.html https://www.sqlite.org/sqlar.html [2] https://www.gnu.org/software/tar/manual/html_node/multiple.html https://www.gnu.org/software/tar/manual/html_node/multiple.h...

itroot 6y ago
I recently impletented an extension to sqlite - https://github.com/itroot/sqlite-brotli-compress https://github.com/itroot/sqlite-brotli-compress - that adds ability to compress blobs to sqlite via brotli, and I think it is trivial to add any compression format to sqlar (with some format extension, of course). But sqlar I believe is not very popular, and just an example of how sqlite can be used.