3 ms·
Chromium is kind of stuck with zlib because it's the algorithm that's in the standards, but if you're making your own protocol, you can do even better than this
by jrockway 2y ago
Chromium is kind of stuck with zlib because it's the algorithm that's in the standards, but if you're making your own protocol, you can do even better than this by picking a better algorithm. Zstandard is faster and compresses better. LZ4 is much faster, but not quite as small.
Some reading: https://jolynch.github.io/posts/use_fast_data_algorithms/ https://jolynch.github.io/posts/use_fast_data_algorithms/
(As an aside, at my last job container pushes / pulls were in the development critical path for a lot of workflows. It turns out that sha256 and gzip are responsible for a lot of the time spent during container startup. Fortunately, Zstandard is allowed, and blake3 digests will be allowed soon.)
- jeffbee 2y agoYeah I just discovered this a few days ago. All the docker-era tools default to gzip but if using, say, bazel rules_oci instead of rules_docker you can turn on zstd for large speedups in push/pull time.
- jeroenhd 2y ago`Content-Encoding: zstd` was added to Chromium a while ago: https://chromestatus.com/feature/6186023867908096 https://chromestatus.com/feature/6186023867908096 You can still use deflate for compression, but Brotli and Zstd have been available in all modern browsers for quite some time.
- amaranth 2y agoSafari doesn't support zstd, that means if you want to use it you have to support multiple formats.
- j16sdiz 2y agoChromium supports brotli and zstd
- cesarb 2y ago> Zstandard is faster and compresses better. However, keep in mind that zstd also needs much more memory. IIRC, it uses by default 8 megabytes as its buffer size (and can be configured to use many times more than that), while zlib uses at most 32 kilobytes, allowing it to run even on small 16-bit processors.