4 ms·

I wish lzma (xz) was integrated into the browser and curl as an Accept-Encoding. Would be amazing for us (clara.io), and I am sure a lot of others.

LeoPanthera 11y ago
Compressing xz is relatively slow, and would be expensive for web servers. Maybe something like LZO would be better? (Which OpenVPN uses to compress data in transit.)
- bhouston 11y ago
  I just want to serve large static data asserts from aws cloudfront. :)
wmf 11y ago
Browsers are getting Brotli which is comparable to xz: https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/xdVm8c2GOMQ/DsIZc8mhkPcJ https://groups.google.com/a/chromium.org/forum/#!msg/blink-d...
- bhouston 11y ago
  My tests with brotli is that is it overrated - it is slow and has poor compression ratios compared to xz. It confuses me why it is being pushed so hard... https://github.com/google/brotli/issues/165 https://github.com/google/brotli/issues/165
  - wmf 11y ago
    Eh, Google gave one example where Brotli does well and you gave one where it does poorly; we're not exactly in science territory here.
    - kijin 11y ago
      Yeah, the example given by GP involves large binary streams. Brotli was designed for small text documents with lots of English words in them, as we often see on the web.
      - zurn 11y ago
        Where does it say that Brotli is for small English text documents? I didn't see anything like that in the draft spec or the Google blog post. The spec doesn't say much on the subject but has this item in the Purpouse section: "Compresses data with a compression ratio comparable to the best currently available general-purpose compression methods and in particular considerably better than the gzip program"
        wmf 11y ago
        Brotli includes a built-in dictionary that contains a lot of English words, HTML tags, etc. so it will give better compression for that kind of input.
        zurn 11y ago
        Yes, it has that optimization for short data (though it's not restricted to English), but the PR and specs say it's still meant to be a general purpouse compressor. And it does very well on most types of large data.
    - Confusion 11y ago
      Brotli does not work well for bhoustons use case, so his original wish stands and your helpful suggestion that he should be able to use Brotli in the near future does unfortunately not fulfill his wish.
    - zurn 11y ago
      It's a real problem in a compressor proposed for general purpouse use when it's shown that a naturally occurring major class of data has this bad performance.
donatj 11y ago
Decompression of xz is slow and quite memory intense. I'd argue that the light memory footprint of gz decompression is better suited to the web, particularly mobile where you need to balance battery v bandwidth.

LeoPanthera 11y ago

Compressing xz is relatively slow, and would be expensive for web servers. Maybe something like LZO would be better? (Which OpenVPN uses to compress data in transit.)

bhouston 11y ago
I just want to serve large static data asserts from aws cloudfront. :)

wmf 11y ago

Browsers are getting Brotli which is comparable to xz: https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/xdVm8c2GOMQ/DsIZc8mhkPcJ https://groups.google.com/a/chromium.org/forum/#!msg/blink-d...

bhouston 11y ago
My tests with brotli is that is it overrated - it is slow and has poor compression ratios compared to xz. It confuses me why it is being pushed so hard... https://github.com/google/brotli/issues/165 https://github.com/google/brotli/issues/165
- wmf 11y ago
  Eh, Google gave one example where Brotli does well and you gave one where it does poorly; we're not exactly in science territory here.
  - kijin 11y ago
    Yeah, the example given by GP involves large binary streams. Brotli was designed for small text documents with lots of English words in them, as we often see on the web.
    - zurn 11y ago
      Where does it say that Brotli is for small English text documents? I didn't see anything like that in the draft spec or the Google blog post. The spec doesn't say much on the subject but has this item in the Purpouse section: "Compresses data with a compression ratio comparable to the best currently available general-purpose compression methods and in particular considerably better than the gzip program"
      - wmf 11y ago
        Brotli includes a built-in dictionary that contains a lot of English words, HTML tags, etc. so it will give better compression for that kind of input.
        zurn 11y ago
        Yes, it has that optimization for short data (though it's not restricted to English), but the PR and specs say it's still meant to be a general purpouse compressor. And it does very well on most types of large data.
  - Confusion 11y ago
    Brotli does not work well for bhoustons use case, so his original wish stands and your helpful suggestion that he should be able to use Brotli in the near future does unfortunately not fulfill his wish.
  - zurn 11y ago
    It's a real problem in a compressor proposed for general purpouse use when it's shown that a naturally occurring major class of data has this bad performance.

donatj 11y ago

Decompression of xz is slow and quite memory intense. I'd argue that the light memory footprint of gz decompression is better suited to the web, particularly mobile where you need to balance battery v bandwidth.