3 ms·
Your question is essentially one of "general composabilitiy vs specialized tool". There's two (opposing) philosophical answers, and three (opposing) practical a
by dexen 6y ago
Your question is essentially one of "general composabilitiy vs specialized tool". There's two (opposing) philosophical answers, and three (opposing) practical answers, and who am I to say which is better for you?
For me, Tar + compression is optimal for most of my use cases - transfer over network, software package distribution, long term storage. In case of transferring over fast network, uncompressed Tar is pretty good [0], and Lz4 compressed is pretty fast too. In case of distributing software, XZ compression is optimal (expensive, good level for small projects). In case of long term storage, Rzip compression is optimal (very expensive, good level for large projects).
Besides the pragmatic considerations, as a software developer & maintainer, I honestly enjoy composability.
Funnily enough, just yesterday I've read the specification of SQLite Archive Files[1]. A pretty cool format and yet - my first question was, "Why is there no support for other compression methods? Like Lz4 for high speed, XZ for high compression?" Please note that, unlike with Tar, extending this standard is pretty much a no-go. Meanwhile a .tar.gz / .tar.xz / .tar.rz / .tar.lz4 is a self-documenting file. You know what to do, and it's easy to write a tool that does the right thing automatically.
I've been implementing a toy project recently, and one of the network communication protocols in it has grown into a poor man's re-implementation of Tar. Having realized that just now, I'll probably yank out several dozen lines of code and replace them with one call to tar on each side. However crufty Tar it might have gotten over decades, it's certainly less buggy than my hand-rolled code.
Lastly, Tar is a much richer format than one might ordinarily expect. For example it supports multiple versions of the same file[2] - and you can add newer version(s) incrementally (append-only file; a much safer operation than re-writting/replacing files). Think a poor man's Git/SQLite/Wiki/whatever. Self-contained, self-documenting, all very nice.
--
[0] uncompressed Tar supports fast seeking. If you expect working with an archive for a while, just create an uncompressed copy. Or a text file index, we don't judge.
[1] https://www.sqlite.org/sqlar.html https://www.sqlite.org/sqlar.html
[2] https://www.gnu.org/software/tar/manual/html_node/multiple.html https://www.gnu.org/software/tar/manual/html_node/multiple.h...
- itroot 6y agoI recently impletented an extension to sqlite - https://github.com/itroot/sqlite-brotli-compress https://github.com/itroot/sqlite-brotli-compress - that adds ability to compress blobs to sqlite via brotli, and I think it is trivial to add any compression format to sqlar (with some format extension, of course). But sqlar I believe is not very popular, and just an example of how sqlite can be used.