4 ms·

How to Improve Rails Caching with Brotli Compression

50 points by pawurb 3y ago

out_of_protocol 3y ago
If that compression is internal-only, i'd start looking at zstd, which is much better at both speed and compression level
- pawurb 3y ago
  Thanks for suggestion. Gem supports custom compression algorithms. I'll try to add more compression types to the benchmarks.
  - out_of_protocol 3y ago
    Note zstd can be fine-tuned to almost any ratio between performance and compression, worth trying multiple different settings https://raw.githubusercontent.com/facebook/zstd/v1.3.4/doc/images/CSpeed2.png https://raw.githubusercontent.com/facebook/zstd/v1.3.4/doc/i...
- e12e 3y ago
  From TFA: > Optionally, you can hook up a custom compression algorithm. For example, Google Snappy offers even better performance than Brotli for the cost of worse compression ratios. ZSTD by Facebook is also worth checking out. (...) > The great news is that support for custom cache compression algorithms has recently been merged into the Rails main branch. But it may take a while before it’s available in a production Rails release.
jonathanhefner 3y ago
Upcoming in Rails 7.1, you can specify a custom compressor for your cache, out of the box. See the `:compressor` option of `ActiveSupport::Cache::Store.new`: https://api.rubyonrails.org/v7.1/classes/ActiveSupport/Cache/Store.html#method-c-new https://api.rubyonrails.org/v7.1/classes/ActiveSupport/Cache...
- pawurb 3y ago
  Thanks, I wasn't sure when it will be live. I'll update the post to reflect it.
- booleanbetrayal 3y ago
  Thanks for this. Will likely defer until this lands, but looking forward to trying out Brotli and Zstd for this use case!
miohtama 3y ago
Important to understand about Brotli vs. other compression is that Brotli has been pretrained with HTML/CSS/JS code. It efficiently compresses English text and source code, especially small files, due to its static dictionary. https://chromium.googlesource.com/external/font-compression-reference/+/2733d6c0c2d6774e4c4d7a233ad200c5a832a133/brotli/enc/dictionary.h https://chromium.googlesource.com/external/font-compression-... There are other factors that make Brotli good, but this is good to understand when dealing with web frontends.

50 points by pawurb 3y ago

If that compression is internal-only, i'd start looking at zstd, which is much better at both speed and compression level

pawurb 3y ago
Thanks for suggestion. Gem supports custom compression algorithms. I'll try to add more compression types to the benchmarks.
- out_of_protocol 3y ago
  Note zstd can be fine-tuned to almost any ratio between performance and compression, worth trying multiple different settings https://raw.githubusercontent.com/facebook/zstd/v1.3.4/doc/images/CSpeed2.png https://raw.githubusercontent.com/facebook/zstd/v1.3.4/doc/i...
e12e 3y ago
From TFA: > Optionally, you can hook up a custom compression algorithm. For example, Google Snappy offers even better performance than Brotli for the cost of worse compression ratios. ZSTD by Facebook is also worth checking out. (...) > The great news is that support for custom cache compression algorithms has recently been merged into the Rails main branch. But it may take a while before it’s available in a production Rails release.

jonathanhefner 3y ago

Upcoming in Rails 7.1, you can specify a custom compressor for your cache, out of the box. See the `:compressor` option of `ActiveSupport::Cache::Store.new`: https://api.rubyonrails.org/v7.1/classes/ActiveSupport/Cache/Store.html#method-c-new https://api.rubyonrails.org/v7.1/classes/ActiveSupport/Cache...

pawurb 3y ago
Thanks, I wasn't sure when it will be live. I'll update the post to reflect it.
booleanbetrayal 3y ago
Thanks for this. Will likely defer until this lands, but looking forward to trying out Brotli and Zstd for this use case!

miohtama 3y ago

Important to understand about Brotli vs. other compression is that Brotli has been pretrained with HTML/CSS/JS code. It efficiently compresses English text and source code, especially small files, due to its static dictionary. https://chromium.googlesource.com/external/font-compression-reference/+/2733d6c0c2d6774e4c4d7a233ad200c5a832a133/brotli/enc/dictionary.h https://chromium.googlesource.com/external/font-compression-... There are other factors that make Brotli good, but this is good to understand when dealing with web frontends.