3 ms·

Chromies got in the way of it for a very long time, but zstd is now coming to the web too, as it’s finally landed in chrome - now we’ve gotta get safari onboard

by raggi 2y ago

CharlesW 2y ago
Looks like it's on the To Do list, at least: https://webkit.org/standards-positions/#position-168 https://webkit.org/standards-positions/#position-168
mananaysiempre 2y ago
I’d love to go all-Zstandard, but in this particular case, as far as I know, Brotli and Zstandard are basically on par at identical values of decompressor memory consumption.
- simondotau 2y ago
  Realistically, everything should just support everything. There’s no reason why every (full featured) web server and every (full featured) web browser couldn’t support all compelling data compression algorithms. Unfortunately we live in a world where Google decides to rip JPEG-XL support out of Chrome for seemingly no reason other than spite. If the reason was a lack of maturity in the underlying library, fine, but that wasn’t the reason they offered.
  - madeofpalk 2y ago
    > There’s no reason Of course, there is - and it's really boring. Prioritisation, and maintenance. It's a big pain to add, say, 100 compressions formats and support them indefinitely, especially with little differentiation between them. Once we agree on what the upper bound of useless formats is, we can start to negotiate what the lower limit is.
    - raggi 2y ago
      It’s not prioritization of the code - it’s relatively little implementation and low maintenance but security is critical here and everyone is very afraid of compressors because gzip, jpeg and various others were pretty bad. Zstd, unlike lz4 before it (at least early on) has a good amount of tests and fuzzing. The implementation could probably be safer (with fairly substantial effort) but having test and fuzzing coverage is a huge step forward over prior industry norms
    - simondotau 2y ago
      I qualified it with compelling which means only including formats/encodings which have demonstrably superior performance in some non-trivial respect. And I qualified it with mature implementation because I agree that if there is no implementation which has a clear specification, is well written, actively maintained, and free of jank, then it ought not qualify. Relative to the current status quo, I would only imagine the number of data compression, image compression, and media compression options to increase by a handful. Single digits. But the sooner we add them, the sooner they can become sufficiently widely deployed as to be useful.
  - rafaelmn 2y ago
    How many CVEs came out of different file format handling across platforms ? Including shit in browsers has insane impact.
- raggi 2y ago
  Brotli and zstd are close, each trading in various configurations and cases - but zstd is muuuuuch more usable in a practical sense, because the code base can be easily compiled and linked against in a wider number of places with less effort, and the cli tools turn up all over for similar reasons. Brotli like many Google libraries in the last decade are infected with googlisms. Brotli is less bad, in a way, for example the C comes with zero build system at all, which is marginally better than a bazel disaster of generated stuff, but it’s still not in a state the average distro contributor gives a care to go turn into a library, prepare pkgconf rules for and whatnot - plus no versioning and so on. Oh and the tests are currently failing.

Chromies got in the way of it for a very long time, but zstd is now coming to the web too, as it’s finally landed in chrome - now we’ve gotta get safari onboard

by raggi 2y ago

Chromies got in the way of it for a very long time, but zstd is now coming to the web too, as it’s finally landed in chrome - now we’ve gotta get safari onboard

CharlesW 2y ago

Looks like it's on the To Do list, at least: https://webkit.org/standards-positions/#position-168 https://webkit.org/standards-positions/#position-168

mananaysiempre 2y ago

I’d love to go all-Zstandard, but in this particular case, as far as I know, Brotli and Zstandard are basically on par at identical values of decompressor memory consumption.

simondotau 2y ago
Realistically, everything should just support everything. There’s no reason why every (full featured) web server and every (full featured) web browser couldn’t support all compelling data compression algorithms. Unfortunately we live in a world where Google decides to rip JPEG-XL support out of Chrome for seemingly no reason other than spite. If the reason was a lack of maturity in the underlying library, fine, but that wasn’t the reason they offered.
- madeofpalk 2y ago
  > There’s no reason Of course, there is - and it's really boring. Prioritisation, and maintenance. It's a big pain to add, say, 100 compressions formats and support them indefinitely, especially with little differentiation between them. Once we agree on what the upper bound of useless formats is, we can start to negotiate what the lower limit is.
  - raggi 2y ago
    It’s not prioritization of the code - it’s relatively little implementation and low maintenance but security is critical here and everyone is very afraid of compressors because gzip, jpeg and various others were pretty bad. Zstd, unlike lz4 before it (at least early on) has a good amount of tests and fuzzing. The implementation could probably be safer (with fairly substantial effort) but having test and fuzzing coverage is a huge step forward over prior industry norms
  - simondotau 2y ago
    I qualified it with compelling which means only including formats/encodings which have demonstrably superior performance in some non-trivial respect. And I qualified it with mature implementation because I agree that if there is no implementation which has a clear specification, is well written, actively maintained, and free of jank, then it ought not qualify. Relative to the current status quo, I would only imagine the number of data compression, image compression, and media compression options to increase by a handful. Single digits. But the sooner we add them, the sooner they can become sufficiently widely deployed as to be useful.
- rafaelmn 2y ago
  How many CVEs came out of different file format handling across platforms ? Including shit in browsers has insane impact.
raggi 2y ago
Brotli and zstd are close, each trading in various configurations and cases - but zstd is muuuuuch more usable in a practical sense, because the code base can be easily compiled and linked against in a wider number of places with less effort, and the cli tools turn up all over for similar reasons. Brotli like many Google libraries in the last decade are infected with googlisms. Brotli is less bad, in a way, for example the C comes with zero build system at all, which is marginally better than a bazel disaster of generated stuff, but it’s still not in a state the average distro contributor gives a care to go turn into a library, prepare pkgconf rules for and whatnot - plus no versioning and so on. Oh and the tests are currently failing.