10 ms·

There's a lot of stuff greater then 64k nowadays ;)

mwpmaybe 11y ago
Especially those pesky XML and JSON payloads!
clarinois 11y ago
Exactly :) If Cloudflare decides it’s worth several man-days to offer this feature to their clients who deliver larger files than 64k, feel free to reach out to us, we’ll be happy to share our experience :) devs@cdn77.com
- jgrahamc 11y ago
  Thanks. We actually wrote and open sourced our nginx modifications for choosing between Brotli and gzip. https://github.com/cloudflare/ngx_brotli_module https://github.com/cloudflare/ngx_brotli_module
- deleted 11y ago
  [deleted]
acqq 11y ago
From the Cloduflare's tests: "Most files are smaller than 64KB, and if we look only at those files then Brotli 4 is actually 1.48X slower than zlib level 8!" And the faster zlib levels (1-7) are even faster! See the table. I especially like zlib 1.
- jqueryin 11y ago
  Keep a rolling average of each user's request filesize and if it exceeds 64KB (image, video, pdf, downloads) then notify them of a possible performance boost to switch to Brotli.
  - acqq 11y ago
    > if it exceeds 64KB (image, video, pdf, downloads) then notify them All these can't be compressed more than they already are, forcing any lossless (like zlib and brotli) compressor on them is just the loss of CPU time. Try and compare. Compression 101.
    - TTPrograms 11y ago
      Well they might compress more, but if so they should publish ;)
    - cbr 11y ago
      There are actually some cases where you can gain by losslessly compressing these. For example, EXIF data in a JPEG is plain text, so gzip can help: $ curl -sS http://www.exiv2.org/include/img_1771.jpg | wc -c 32764 $ curl -sS http://www.exiv2.org/include/img_1771.jpg | gzip -9 | wc -c 31323 In this case, though, it often makes more sense to just remove the EXIF data. PDF files can be even more compressible, at least ones that are text and layout as opposed to embedded images: $ curl -sS http://www.polyu.edu.hk/iaee/files/pdf-sample.pdf | wc -c 7945 $ curl -sS http://www.polyu.edu.hk/iaee/files/pdf-sample.pdf | gzip -9 | wc -c 4336
      - acqq 11y ago
        I agree, there is a small subset of such files where small pieces (or even the whole) can be compressed. If we just say "images" or "videos" there are also "uncompressed" variants of such, but such images or videos are always those which aren't even prepared to be on the web. E.g. "uncompressed AVI" or "Windows BMP." Those prepared for the web (e.g. MP4 or PNG) certainly aren't losslessly compressible. Those that want to deliver the videos or pictures should use the proper format and not depend on Brotli or zlib.
  - snug 11y ago
    Gzip/brotli compression is not done for media, typically only for plain text files.

Especially those pesky XML and JSON payloads!

Exactly :) If Cloudflare decides it’s worth several man-days to offer this feature to their clients who deliver larger files than 64k, feel free to reach out to us, we’ll be happy to share our experience :) devs@cdn77.com

jgrahamc 11y ago
Thanks. We actually wrote and open sourced our nginx modifications for choosing between Brotli and gzip. https://github.com/cloudflare/ngx_brotli_module https://github.com/cloudflare/ngx_brotli_module
deleted 11y ago
[deleted]

acqq 11y ago

From the Cloduflare's tests: "Most files are smaller than 64KB, and if we look only at those files then Brotli 4 is actually 1.48X slower than zlib level 8!" And the faster zlib levels (1-7) are even faster! See the table. I especially like zlib 1.

jqueryin 11y ago
Keep a rolling average of each user's request filesize and if it exceeds 64KB (image, video, pdf, downloads) then notify them of a possible performance boost to switch to Brotli.
- acqq 11y ago
  > if it exceeds 64KB (image, video, pdf, downloads) then notify them All these can't be compressed more than they already are, forcing any lossless (like zlib and brotli) compressor on them is just the loss of CPU time. Try and compare. Compression 101.
  - TTPrograms 11y ago
    Well they might compress more, but if so they should publish ;)
  - cbr 11y ago
    There are actually some cases where you can gain by losslessly compressing these. For example, EXIF data in a JPEG is plain text, so gzip can help: $ curl -sS http://www.exiv2.org/include/img_1771.jpg | wc -c 32764 $ curl -sS http://www.exiv2.org/include/img_1771.jpg | gzip -9 | wc -c 31323 In this case, though, it often makes more sense to just remove the EXIF data. PDF files can be even more compressible, at least ones that are text and layout as opposed to embedded images: $ curl -sS http://www.polyu.edu.hk/iaee/files/pdf-sample.pdf | wc -c 7945 $ curl -sS http://www.polyu.edu.hk/iaee/files/pdf-sample.pdf | gzip -9 | wc -c 4336
    - acqq 11y ago
      I agree, there is a small subset of such files where small pieces (or even the whole) can be compressed. If we just say "images" or "videos" there are also "uncompressed" variants of such, but such images or videos are always those which aren't even prepared to be on the web. E.g. "uncompressed AVI" or "Windows BMP." Those prepared for the web (e.g. MP4 or PNG) certainly aren't losslessly compressible. Those that want to deliver the videos or pictures should use the proper format and not depend on Brotli or zlib.
- snug 11y ago
  Gzip/brotli compression is not done for media, typically only for plain text files.