3 ms·

gzip is very slow. I don't know why anyone would still use it today.

jgalt212 7y ago
really? what do you suggest that is faster? Our shop has tested a number of compression formats (xv, bz2, gzip, etc) and gzip is good enough and faster than the others we tested.
- thedance 7y ago
  Well yah, those are all ratio-tuned codecs that are very slow. LZMA beats bzip2 on both ratio and speed so you might as well forget about bzip2 forever. Zstd, snappy, LZ4, or even brotli are probably better choices than zlib for most people. Brotli has LZMA-like ratios at dramatically higher speeds.
- blattimwind 7y ago
  The algorithms you name are all rather outdated. Typical gzip decompression speed is somewhere in the 200-250 MB/s region, compression is much slower. LZ4 for example tends to compress at ~600-700 MB/s, and decompress at several GB/s. zstd is tweakable over a very wide range of ratio-speed trade-offs. LZMA(2) (xz) is a rather troubled format and should not be used any more. bzip2 has always been slower than gzip with usually marginally better compression. It has been irrelevant for a long time.
  - oefrha 7y ago
    > LZMA(2) (xv) is a rather troubled format and should not be used any more. You mean xz. Not sure where you got the idea that it’s a troubled format, but if you’re talking about the infamous “Xz format inadequate for long-term archiving”, IMO that’s just bzip2 authors taking a dump on xz for no good reason, and fortunately for us it’s bzip2 that’s basically irrelevant today, not xz.
  - gnulinux 7y ago
    > It has been irrelevant for a long time. Funny you say that, my company uses bz2 for compressing pretty much everything.
    - blattimwind 7y ago
      And many companies use fixed-width record formats for data exchange... what's your point exactly?
      - gnulinux 7y ago
        I guess that it's not irrelevant. Deprecated, old, etc sure, but irrelevant?
- deleted 7y ago
  [deleted]
- ants_a 7y ago
  https://quixdb.github.io/squash-benchmark/ https://quixdb.github.io/squash-benchmark/ Brotli is better, zstd is faster, lz4 is often good enough and a lot faster.
kbutler 7y ago
When you say, "very slow" what do you mean? gzip is fast relative to other compression algorithms and relative to internet speeds. gzip is slower than an ideal gigabit network, but faster than 100mb. https://www.rootusers.com/gzip-vs-bzip2-vs-xz-performance-comparison/ https://www.rootusers.com/gzip-vs-bzip2-vs-xz-performance-co...
- blattimwind 7y ago
  "gzip vs bzip2 vs xz performace comparison" is like comparing which centenarian can limp faster, it might be entertaining for some people, but is generally not relevant. > gzip is fast relative to other compression algorithms gzip looks fast perhaps compared to "xz -9", but not to anything modern.
  - abjKT26nO8 7y ago
    Your comment could benefit from pointing out the alternatives that outperform gzip.
- vlovich123 7y ago
  Check out zstd
willvarfar 7y ago
Gzip is old but still good. It’s not as good as Zstd nor cases where lz4 and ilk shine, but it’s not fair to claim its slow. Gzip is quite close to the Pareto frontier, meaning it is a good trade off of time and space. See the charts at http://mattmahoney.net/dc/text.html http://mattmahoney.net/dc/text.html (And read up on the Hutter Prize!)

really? what do you suggest that is faster? Our shop has tested a number of compression formats (xv, bz2, gzip, etc) and gzip is good enough and faster than the others we tested.

thedance 7y ago
Well yah, those are all ratio-tuned codecs that are very slow. LZMA beats bzip2 on both ratio and speed so you might as well forget about bzip2 forever. Zstd, snappy, LZ4, or even brotli are probably better choices than zlib for most people. Brotli has LZMA-like ratios at dramatically higher speeds.
blattimwind 7y ago
The algorithms you name are all rather outdated. Typical gzip decompression speed is somewhere in the 200-250 MB/s region, compression is much slower. LZ4 for example tends to compress at ~600-700 MB/s, and decompress at several GB/s. zstd is tweakable over a very wide range of ratio-speed trade-offs. LZMA(2) (xz) is a rather troubled format and should not be used any more. bzip2 has always been slower than gzip with usually marginally better compression. It has been irrelevant for a long time.
- oefrha 7y ago
  > LZMA(2) (xv) is a rather troubled format and should not be used any more. You mean xz. Not sure where you got the idea that it’s a troubled format, but if you’re talking about the infamous “Xz format inadequate for long-term archiving”, IMO that’s just bzip2 authors taking a dump on xz for no good reason, and fortunately for us it’s bzip2 that’s basically irrelevant today, not xz.
- gnulinux 7y ago
  > It has been irrelevant for a long time. Funny you say that, my company uses bz2 for compressing pretty much everything.
  - blattimwind 7y ago
    And many companies use fixed-width record formats for data exchange... what's your point exactly?
    - gnulinux 7y ago
      I guess that it's not irrelevant. Deprecated, old, etc sure, but irrelevant?
deleted 7y ago
[deleted]
ants_a 7y ago
https://quixdb.github.io/squash-benchmark/ https://quixdb.github.io/squash-benchmark/ Brotli is better, zstd is faster, lz4 is often good enough and a lot faster.

kbutler 7y ago

When you say, "very slow" what do you mean? gzip is fast relative to other compression algorithms and relative to internet speeds. gzip is slower than an ideal gigabit network, but faster than 100mb. https://www.rootusers.com/gzip-vs-bzip2-vs-xz-performance-comparison/ https://www.rootusers.com/gzip-vs-bzip2-vs-xz-performance-co...

blattimwind 7y ago
"gzip vs bzip2 vs xz performace comparison" is like comparing which centenarian can limp faster, it might be entertaining for some people, but is generally not relevant. > gzip is fast relative to other compression algorithms gzip looks fast perhaps compared to "xz -9", but not to anything modern.
- abjKT26nO8 7y ago
  Your comment could benefit from pointing out the alternatives that outperform gzip.
vlovich123 7y ago
Check out zstd

willvarfar 7y ago

Gzip is old but still good. It’s not as good as Zstd nor cases where lz4 and ilk shine, but it’s not fair to claim its slow. Gzip is quite close to the Pareto frontier, meaning it is a good trade off of time and space. See the charts at http://mattmahoney.net/dc/text.html http://mattmahoney.net/dc/text.html (And read up on the Hutter Prize!)