4 ms·

We have been live with zstd encoding for several weeks now and last week public traffic really started to use it. We're now at 50% brotli, 40% zstd, 9% gzip. Ou

We have been live with zstd encoding for several weeks now and last week public traffic really started to use it. We're now at 50% brotli, 40% zstd, 9% gzip. Our payload is almost exclusively JavaScript and compression rates are 73% zstd (level 6), 72% brotli (level 4) and 65% gzip (level 4). However, for 95th percentile encoding speed zstd is 4.3ms, brotli is 2.1ms, and gzip is 0.1ms. At the 75th percentile they're all essentially the same. This is a Golang application and we use a cgo brotli library but a pure-go zstd library.

saurik 3y ago
Are you unable to cache the zstd-compressed JavaScript in your app server (so as to save the 4.3ms of CPU on every request)? If not, are you sure the 4.3ms of CPU costs less than the 1% of bandwidth? I saw a linked discussion here with a Brotli author talking to a zstd author that started making me stare into these questions more.
- fastest963 3y ago
  We compile/build JS on the fly per-request so we wouldn't be able to cache the JS ahead of time. The difference between brotli and zstd is 2.2ms at the 95th percentile, at lower percentiles the differences are much smaller. This is still an area of active investigation/development for us. There are possibly content sizes where one is better than another. There are also probably tweaks on the server-side to improve the encoding time.
  - eddd-ddde 3y ago
    Unrelated but I'm really curious what's the use case for sending custom js per request?
    - fastest963 3y ago
      We offer a lot of customization in our product so we can eliminate whole blocks or code if we know the customer doesn't have x turned on. We also have several geo-based products and so we know we don't need to send GDPR-related code to users in the USA for example. The goal is to ship the minimal amount of code to the end-user.
      - gcr 3y ago
        I’d love to poke at this if you don’t mind. One advantage of downloading everything is that you only download it once. If the user is enabling/disabling features, you’ll have to then re-ship a new js bundle, no?
        eddd-ddde 3y ago
        Also my thoughts. With something like Qwik you get tiny chunks that you only download when needed and are highly cacheable. All automated!
        fastest963 3y ago
        Not impossible for us to use something like that but remember that we are a vendor that sits on customer sites with very limited objectives we aren't powering a full-on webapp or multi-page experience. Typically we want to load our experience/component as quickly as possible after some sort of user interaction and the user interaction might have been from the customer's app/website.
        eddd-ddde 3y ago
        Ah I see, that makes a lot of sense.
        fastest963 3y ago
        At the scale we operate at we would probably have billions, if not tens or hundreds of billions, of downloads before any feature status is changed. Typically our customers are not turning on and off features very frequently.
      - vlovich123 3y ago
        That still doesn’t explain why you can’t cache. I would think that each code snippet itself is still cacheable. Or do you amalgamate all of them into a single download? Even still, I would expect most users to be fetching the same amalgamation, in which case caching should be beneficial…
netol 3y ago
I fail to understand why this, now. For these minimal gains (compared to Brotli), servers and CDNs will need to increase memory and disk space to store the cached responses in zstd.
- fastest963 3y ago
  We don't use brotli for any responses under 25kb and in those cases, for us, zstd is the clear winner. We are still tweaking zstd and expect to improve the performance to bring it inline with brotli. Additionally, our biggest expense per month is egress, any savings goes a long way for us.
valyala 3y ago
Try using cgo-based zstd package https://github.com/valyala/gozstd https://github.com/valyala/gozstd . It is usually faster than pure-Go package https://github.com/klauspost/compress/tree/master/zstd https://github.com/klauspost/compress/tree/master/zstd
- fastest963 3y ago
  Yep, this was validated earlier but deemed not worth the effort. We might revisit it down the road but we already deal with cgo for Brotli and it's a pain. We run Linux servers but Macs for development and so we have to keep the toolchain working across both and it isn't free.
- everybackdoor 3y ago
  [dead]

saurik 3y ago

Are you unable to cache the zstd-compressed JavaScript in your app server (so as to save the 4.3ms of CPU on every request)? If not, are you sure the 4.3ms of CPU costs less than the 1% of bandwidth? I saw a linked discussion here with a Brotli author talking to a zstd author that started making me stare into these questions more.

fastest963 3y ago
We compile/build JS on the fly per-request so we wouldn't be able to cache the JS ahead of time. The difference between brotli and zstd is 2.2ms at the 95th percentile, at lower percentiles the differences are much smaller. This is still an area of active investigation/development for us. There are possibly content sizes where one is better than another. There are also probably tweaks on the server-side to improve the encoding time.
- eddd-ddde 3y ago
  Unrelated but I'm really curious what's the use case for sending custom js per request?
  - fastest963 3y ago
    We offer a lot of customization in our product so we can eliminate whole blocks or code if we know the customer doesn't have x turned on. We also have several geo-based products and so we know we don't need to send GDPR-related code to users in the USA for example. The goal is to ship the minimal amount of code to the end-user.
    - gcr 3y ago
      I’d love to poke at this if you don’t mind. One advantage of downloading everything is that you only download it once. If the user is enabling/disabling features, you’ll have to then re-ship a new js bundle, no?
      - eddd-ddde 3y ago
        Also my thoughts. With something like Qwik you get tiny chunks that you only download when needed and are highly cacheable. All automated!
        fastest963 3y ago
        Not impossible for us to use something like that but remember that we are a vendor that sits on customer sites with very limited objectives we aren't powering a full-on webapp or multi-page experience. Typically we want to load our experience/component as quickly as possible after some sort of user interaction and the user interaction might have been from the customer's app/website.
        eddd-ddde 3y ago
        Ah I see, that makes a lot of sense.
      - fastest963 3y ago
        At the scale we operate at we would probably have billions, if not tens or hundreds of billions, of downloads before any feature status is changed. Typically our customers are not turning on and off features very frequently.
    - vlovich123 3y ago
      That still doesn’t explain why you can’t cache. I would think that each code snippet itself is still cacheable. Or do you amalgamate all of them into a single download? Even still, I would expect most users to be fetching the same amalgamation, in which case caching should be beneficial…

netol 3y ago

I fail to understand why this, now. For these minimal gains (compared to Brotli), servers and CDNs will need to increase memory and disk space to store the cached responses in zstd.

fastest963 3y ago
We don't use brotli for any responses under 25kb and in those cases, for us, zstd is the clear winner. We are still tweaking zstd and expect to improve the performance to bring it inline with brotli. Additionally, our biggest expense per month is egress, any savings goes a long way for us.

valyala 3y ago

Try using cgo-based zstd package https://github.com/valyala/gozstd https://github.com/valyala/gozstd . It is usually faster than pure-Go package https://github.com/klauspost/compress/tree/master/zstd https://github.com/klauspost/compress/tree/master/zstd

fastest963 3y ago
Yep, this was validated earlier but deemed not worth the effort. We might revisit it down the road but we already deal with cgo for Brotli and it's a pain. We run Linux servers but Macs for development and so we have to keep the toolchain working across both and it isn't free.
everybackdoor 3y ago
[dead]