4 ms·
They compare pretty closely. The squash benchmark is a nice interactive comparison tool that usually agrees with my own benchmarks. See for example [1]. (Note t
by felixhandte 8y ago
They compare pretty closely. The squash benchmark is a nice interactive comparison tool that usually agrees with my own benchmarks. See for example [1]. (Note that you should experiment with different input texts to get a sense for the variability in relative performance.)
I am working on that very question! Zstd's support for creating and using custom dictionaries opens the door to significant efficiencies. As described in the post though, dictionaries make compression a more complicated thing to use, and so there are lots of questions about how to apply that to the public internet in a way that's cross-compatible and secure. In short: it's something we're actively exploring.
[1] http://quixdb.github.io/squash-benchmark/unstable/?dataset=enwik8&machine=s-desktop&speed-scale=logarithmic&visible-plugins=zstd,brotli,lz4,lzma,zlib http://quixdb.github.io/squash-benchmark/unstable/?dataset=e...
- pmarreck 8y agoBrowsers could cache dictionaries per MIME type per domain which are provided by the server. Would be super cool, if a bit more complicated.
- felixhandte 8y agoYep, that's in the vicinity of the solutions we're thinking about. There are a few proposals out there, and prior art like SDCH [1]. The hard part is that compression is already an attack vector for the web (e.g., CRIME [2], BREACH [3], et al.). We want to make sure that we're not eroding or unduly complicating that situation [4]. [1] https://tools.ietf.org/html/draft-lee-sdch-spec-00 https://tools.ietf.org/html/draft-lee-sdch-spec-00 [2] https://en.wikipedia.org/wiki/CRIME https://en.wikipedia.org/wiki/CRIME [3] https://en.wikipedia.org/wiki/BREACH https://en.wikipedia.org/wiki/BREACH [4] https://tools.ietf.org/html/draft-kucherawy-httpbis-dict-sec-00 https://tools.ietf.org/html/draft-kucherawy-httpbis-dict-sec...
- JyrkiAlakuijala 8y agoBrotli added custom dictionaries in 2014 or 2015 (before zstd format was even frozen). We are currently working on a new more efficient custom dictionary that uses context and transforms to ramp up the impact of the custom dictionary (by about 20%).