4 ms·

Are there any negatives to Zstandard (besides those listed in the article)? Specifically, I was thinking about using it to compress backup archives. I figure it

by mastax 10y ago

Are there any negatives to Zstandard (besides those listed in the article)? Specifically, I was thinking about using it to compress backup archives. I figure it might be annoying to use an algorithm that can't be extracted using default installed tools (though zstd is in Ubuntu 16.10 repos), but otherwise I don't see any downside.

koolba 10y ago
Standard tooling can be a big deal for some use cases. Having your data compressed with zlib means that you can stream it without decompressing to a client that can handle zlib streams. The biggest candidate for that is serving static assets to browsers which can natively decompress a zlib compressed response by checking the response Content-Encoding. Until browsers catch up, you'd have to either serve uncompressed traffic to the end user or recompress it for transport (which kind of kills the whole point right?).
hannob 10y ago
You usually get a tiny bit better compression with .xz, but it's much slower. So in scenarios where compression speed is mostly irrelevant and you only care about size, take .xz. Otherwise zstd is usually the best choice.
beagle3 10y ago
The PATENTS file. Calling it "zstandard" is a genius move - everyone assumes it's a replacement for zlib (royalty free, bsd-style copyright, patent free), where only the first two are in fact right. See the post by rdtsc above https://news.ycombinator.com/item?id=13814476 https://news.ycombinator.com/item?id=13814476
joosters 10y ago
I've hit a few problems using zstd from an early version (0.01?) - compressed files created by the original tools were not de-compressible using more recent programs :( I think the problem lies in the perl library Compress::Zstandard, rather than in the zstd code itself. The C code has flags to enable 'legacy' support and they they are on by default. The perl library doesn't seem to be using them... When I get the time, hopefully I can track this down further and submit a patch if this turns out to be the case.

by mastax 10y ago

koolba 10y ago

Standard tooling can be a big deal for some use cases. Having your data compressed with zlib means that you can stream it without decompressing to a client that can handle zlib streams. The biggest candidate for that is serving static assets to browsers which can natively decompress a zlib compressed response by checking the response Content-Encoding. Until browsers catch up, you'd have to either serve uncompressed traffic to the end user or recompress it for transport (which kind of kills the whole point right?).

hannob 10y ago

You usually get a tiny bit better compression with .xz, but it's much slower. So in scenarios where compression speed is mostly irrelevant and you only care about size, take .xz. Otherwise zstd is usually the best choice.

beagle3 10y ago

The PATENTS file. Calling it "zstandard" is a genius move - everyone assumes it's a replacement for zlib (royalty free, bsd-style copyright, patent free), where only the first two are in fact right. See the post by rdtsc above https://news.ycombinator.com/item?id=13814476 https://news.ycombinator.com/item?id=13814476

joosters 10y ago

I've hit a few problems using zstd from an early version (0.01?) - compressed files created by the original tools were not de-compressible using more recent programs :( I think the problem lies in the perl library Compress::Zstandard, rather than in the zstd code itself. The C code has flags to enable 'legacy' support and they they are on by default. The perl library doesn't seem to be using them... When I get the time, hopefully I can track this down further and submit a patch if this turns out to be the case.