4 ms·

Xz format inadequate for general use

22 points by Bogdanp 1y ago

usr1106 1y ago
Site says: Too many requests. At 3 points and 0 comments it can't be HN?
- JdeBP 1y ago
  It was also simultaneously submitted to lobste.rs (https://mastodon.social/@lobsters/115093242443397877 https://mastodon.social/@lobsters/115093242443397877) , and is on multiple Hacker News robot feeds such as https://mstdn.social/@hkrn/115093327746058961 https://mstdn.social/@hkrn/115093327746058961 , of course. At minimum, every node that has replicated that FediVerse post has done a fetch in order to construct a thumbnail (because the headlined URL is first).
rgovostes 1y ago
Nonetheless, xz has the best sshd integration.
arp242 1y ago
(2016); and used to be "Xz format inadequate for long-term archiving", but apparently the author's opinion of the xz format has worsened. Xz format inadequate for long-term archiving (2016) - https://news.ycombinator.com/item?id=39871914 https://news.ycombinator.com/item?id=39871914 - Mar 2024 (26 comments) Xz format inadequate for long-term archiving (2016) - https://news.ycombinator.com/item?id=39868810 https://news.ycombinator.com/item?id=39868810 - Mar 2024 (9 comments) Xz format considered inadequate for long-term archiving (2016) - https://news.ycombinator.com/item?id=32210438 https://news.ycombinator.com/item?id=32210438 - Jul 2022 (158 comments) Xz format inadequate for long-term archiving (2016) - https://news.ycombinator.com/item?id=20103255 https://news.ycombinator.com/item?id=20103255 - Jun 2019 (58 comments) Xz format inadequate for long-term archiving (2017) - https://news.ycombinator.com/item?id=16884832 https://news.ycombinator.com/item?id=16884832 - Apr 2018 (136 comments) Xz format inadequate for long-term archiving - https://news.ycombinator.com/item?id=12768425 https://news.ycombinator.com/item?id=12768425 - Oct 2016 (91 comments)
angst 1y ago
if i need xz i'll just use zstd instead.
- petre 1y ago
  Well, I did dome tests and settled on xz -0.
o11c 1y ago
As always when this is posted: this is quite overstated. If you're only using .xz on something already protected by a sha2sum or something, and you trust the source, almost all of this is immediately invalidated (and the rest is "whatever, good enough"). Now, "trust the source" does have a hole that most people might not think about - are you sure the archive you just created corresponds to the files you tried to add? Doing extraction comparison tests should be mandatory ... but the same applies to all other archive formats, and very few tools automate the check in a way that also generates the hash.
lifthrasiir 1y ago
I have to cringe every time this article is brought up, because while xz does have made some questionable design choices, it practically works well and doesn't do anything fundamentally inadaquate as a compression file format [1]. And yet about a half of this article is devoted to the xz's inadaquateness as an archival format for which xz was never designed (use a properly designed archival format if you want). At this point lzip should be avoided because of this article. Sigh. [1] Refers to file formats like .gz, .bz2 and of course .xz; doesn't include .tar, .zip or .7z.
arp242 1y ago
Most of these are not really serious issues in the real world. It's been critiqued extensively in the past (posted those discussions in a separate comment) and I'll not repeat it here. But I will add that lzip's "yo, here's a tarball, trust me bro"-model of distribution is no longer fit for 2025, for reasons that should be obvious. In my opinion this blows any concerns about xz out of the water. Technically this is about formats and not implementations, both de-facto have one dominant implementation, so that's a bit of an academic distinction. Anyway, zstd is the future for most use cases.
- lifthrasiir 1y ago
  I'm indeed interested in the author's opinion about the zstandard format. Because one seems to use xz as a kind of scapegoat...
  - quectophoton 1y ago
    And if for some reason zstd is not acceptable, the LZ4 format is still not too bad. Same author as zstd, relatively well supported, and the format seems even simpler to implement compared to Lzip. So if for some reason I can't use Zstd, I would still probably choose LZ4 instead of Lzip.
Milpotel 1y ago
The out-of-context quotes by Tony Hoare are at least questionable.

22 points by Bogdanp 1y ago

usr1106 1y ago

Site says: Too many requests. At 3 points and 0 comments it can't be HN?

JdeBP 1y ago
It was also simultaneously submitted to lobste.rs (https://mastodon.social/@lobsters/115093242443397877 https://mastodon.social/@lobsters/115093242443397877) , and is on multiple Hacker News robot feeds such as https://mstdn.social/@hkrn/115093327746058961 https://mstdn.social/@hkrn/115093327746058961 , of course. At minimum, every node that has replicated that FediVerse post has done a fetch in order to construct a thumbnail (because the headlined URL is first).

rgovostes 1y ago

Nonetheless, xz has the best sshd integration.

arp242 1y ago

(2016); and used to be "Xz format inadequate for long-term archiving", but apparently the author's opinion of the xz format has worsened. Xz format inadequate for long-term archiving (2016) - https://news.ycombinator.com/item?id=39871914 https://news.ycombinator.com/item?id=39871914 - Mar 2024 (26 comments) Xz format inadequate for long-term archiving (2016) - https://news.ycombinator.com/item?id=39868810 https://news.ycombinator.com/item?id=39868810 - Mar 2024 (9 comments) Xz format considered inadequate for long-term archiving (2016) - https://news.ycombinator.com/item?id=32210438 https://news.ycombinator.com/item?id=32210438 - Jul 2022 (158 comments) Xz format inadequate for long-term archiving (2016) - https://news.ycombinator.com/item?id=20103255 https://news.ycombinator.com/item?id=20103255 - Jun 2019 (58 comments) Xz format inadequate for long-term archiving (2017) - https://news.ycombinator.com/item?id=16884832 https://news.ycombinator.com/item?id=16884832 - Apr 2018 (136 comments) Xz format inadequate for long-term archiving - https://news.ycombinator.com/item?id=12768425 https://news.ycombinator.com/item?id=12768425 - Oct 2016 (91 comments)

angst 1y ago

if i need xz i'll just use zstd instead.

petre 1y ago
Well, I did dome tests and settled on xz -0.

o11c 1y ago

As always when this is posted: this is quite overstated. If you're only using .xz on something already protected by a sha2sum or something, and you trust the source, almost all of this is immediately invalidated (and the rest is "whatever, good enough"). Now, "trust the source" does have a hole that most people might not think about - are you sure the archive you just created corresponds to the files you tried to add? Doing extraction comparison tests should be mandatory ... but the same applies to all other archive formats, and very few tools automate the check in a way that also generates the hash.

lifthrasiir 1y ago

I have to cringe every time this article is brought up, because while xz does have made some questionable design choices, it practically works well and doesn't do anything fundamentally inadaquate as a compression file format [1]. And yet about a half of this article is devoted to the xz's inadaquateness as an archival format for which xz was never designed (use a properly designed archival format if you want). At this point lzip should be avoided because of this article. Sigh. [1] Refers to file formats like .gz, .bz2 and of course .xz; doesn't include .tar, .zip or .7z.

arp242 1y ago

Most of these are not really serious issues in the real world. It's been critiqued extensively in the past (posted those discussions in a separate comment) and I'll not repeat it here. But I will add that lzip's "yo, here's a tarball, trust me bro"-model of distribution is no longer fit for 2025, for reasons that should be obvious. In my opinion this blows any concerns about xz out of the water. Technically this is about formats and not implementations, both de-facto have one dominant implementation, so that's a bit of an academic distinction. Anyway, zstd is the future for most use cases.

lifthrasiir 1y ago
I'm indeed interested in the author's opinion about the zstandard format. Because one seems to use xz as a kind of scapegoat...
- quectophoton 1y ago
  And if for some reason zstd is not acceptable, the LZ4 format is still not too bad. Same author as zstd, relatively well supported, and the format seems even simpler to implement compared to Lzip. So if for some reason I can't use Zstd, I would still probably choose LZ4 instead of Lzip.

Milpotel 1y ago

The out-of-context quotes by Tony Hoare are at least questionable.