5 ms·

> Brotli's fastest compression is slightly faster than zstd's. Come on, this is not serious. Brotli's fastest compression algorithm is still significantly slo

> Brotli's fastest compression is slightly faster than zstd's. Come on, this is not serious. Brotli's fastest compression algorithm is still significantly slower than zstd. And more importantly, it compresses _much worse_. For a 3rd party evaluation, one can try [TurboBench](https://github.com/powturbo/TurboBench https://github.com/powturbo/TurboBench) or even [lzbench](https://github.com/inikep/lzbench https://github.com/inikep/lzbench) which are open-sourced. Squash introduces a wrapper layer with distortions which makes it less reliable, and more complex to use and install, quite a pity given the graphical presentation is very good. I'm interested in speed, and in this area, all benchmarks point in the same direction : for a given speed budget, Zstandard offers better ratio (and decompresses much faster).

emn13 10y ago
The two benchmarks you mention aren't very useful for comparing brotli and zstd, because they deal with large datasets. Per-call overhead matters a lot for small files, and I imagine (particularly brotli) is aimed at small files. Zstd (usefully!) calls out dictionary compression, which hints that small files matter for it too, but I'm not positive there's any specific use case for zstd in mind. In any case, neither the compression ratios nor the speed of large-file compression necessarily say much about small file performance. There's just much more context to search in a 100MB file than there is in a 10KB file. Having said that, there's no reason to assume brotli is better for small files; there's just no way to tell given the links you provide. I'm not affiliated with nor use neither zstd nor brotli.
JyrkiAlakuijala 10y ago
TurboBench didn't compile straigth after git clone. lzbench did: I get a test file by: wget https://web.archive.org/web/20151222062543/http://www.microsoft.com/surface/th-th/devices/surface-pro-4?ocid=OCTEVENT_MSCOM https://web.archive.org/web/20151222062543/http://www.micros... The test file is 267253 bytes. $ ./lzbench -ebrotli,0,1,2,5,7,9,11/zstd,1,22 testfile brotli 0.4.0 -0 compresses 783 MB/s and decompresses 809 MB/s zstd 0.7.1 -1 compresses 586 MB/s and decompresses 1691 MB/s brotli 0.4.0 -7 compresses 57 MB/s, decompresses 873 MB/s to 28185 bytes brotli 0.4.0 -11 compresses to 25413 bytes zstd 0.7.1 -22 compresses in 4.01 MB/s to 28363 bytes Of course it is an unfair example because of the static dictionary that brotli uses, but it is not a pathological example: Thai is not part of the static dictionary. The numbers are on a i7-4790K@4.00 GHz. Brotli's fastest compression is faster than that of zstd, at least as shown with lzbench and this file. Also brotli wins in compression density. In this file the win is 10.5 % less bytes for brotli -11 than for zstd -22.
- blank_state 10y ago
  > Of course it is an unfair example because of the static dictionary that brotli uses It is certainly a favorable ground for Brotli. Brotli claims an advantage in html compression, thanks to its integrated specialized dictionary. The real pb though is the suggested conclusion that these favorable results are broadly applicable everywhere else. That's a terrible suggestion. We need more examples, not just "html files" which happen to be Brotli's best case. > brotli 0.4.0 -11 compresses to 25413 bytes > zstd 0.7.1 -22 compresses in 4.01 MB/s to 28363 bytes Why you don't disclose the compression time of brotli ? Of course it does matter : everyone understand that an algorithm that spend 10x more cpu has the budget to compress more. > brotli 0.4.0 -0 compresses 783 MB/s and decompresses 809 MB/s > zstd 0.7.1 -1 compresses 586 MB/s and decompresses 1691 MB/s Here, you don't disclose the compression ratio of both algorithm, implying they are equal. By such standard, LZ4 is probably the best : it's so much faster ! Of course, they do not compress the same... I was initially thrilled at your detailed answer, but now, quite frankly, I feel cheated. Grossly so. This is really disappointing. I was so much vexed that I decided to run the tests myself. Downloading and using __the same html file__, the same lzbench, same library versions, just a different computer and compiler, here is what it produced : | Algo | compressed size | compression speed | decompression speed | | --------- | --------------- | ----------------- | ------------------- | | brotli -2 | 36223 | 220 MB/s | 670 MB/s | | zstd -1 | 36655 | 480 MB/ | 1400 MB/s | | brotli -1 | 38292 | 360 MB/s | 650 MB/s | | brotli -0 | 41141 | 560 MB/s | 610 MB/s | __Conclusion__ : brotli -0 is indeed fast, faster than in my previous tests. It seems to be tuned to reach this objective, but throw away a lot of compression ratio to get there. Consequently, brotli -0 is not comparable to zstd -1, it takes brotli -2 to produce an equivalent compressed size . By that time though, zstd is much, much faster. Which is exactly the question I'm trying to get answers to : which algorithm compresses better for a given speed budget ? That's what matters, at least in our datacenter. I'm not interested in ultra slow mode, but while at it, I wanted to complete the picture with the missing compression speed of brotli - 11. It produced : zstd -22 : 2.95 MB/s brotli -11 : 0.53 MB/s So that's > 5x difference. It surely helps to reach better compression ratios. I also wanted an answer to "by how much the dictionary helps ?". Fortunately, TurboBench can help, thanks to a special mode which turns off dictionary compression. By using it on the very same sample, brotli -11 compressed size increases from 25413 to 26639 bytes. 5% larger, clearly not negligible. Still good, but it cuts the advertised size difference in half. Anyway, clearly I feel disappointed to have to redo the tests myself, because some inconvenient results were intentionally undisclosed (or not produced). This really undermines my trust in future publications. That learned me something : trust only benchmark done by yourself. And now, I should probably benchmark even more ...

emn13 10y ago

The two benchmarks you mention aren't very useful for comparing brotli and zstd, because they deal with large datasets. Per-call overhead matters a lot for small files, and I imagine (particularly brotli) is aimed at small files. Zstd (usefully!) calls out dictionary compression, which hints that small files matter for it too, but I'm not positive there's any specific use case for zstd in mind. In any case, neither the compression ratios nor the speed of large-file compression necessarily say much about small file performance. There's just much more context to search in a 100MB file than there is in a 10KB file. Having said that, there's no reason to assume brotli is better for small files; there's just no way to tell given the links you provide. I'm not affiliated with nor use neither zstd nor brotli.

JyrkiAlakuijala 10y ago

TurboBench didn't compile straigth after git clone. lzbench did: I get a test file by: wget https://web.archive.org/web/20151222062543/http://www.microsoft.com/surface/th-th/devices/surface-pro-4?ocid=OCTEVENT_MSCOM https://web.archive.org/web/20151222062543/http://www.micros... The test file is 267253 bytes. $ ./lzbench -ebrotli,0,1,2,5,7,9,11/zstd,1,22 testfile brotli 0.4.0 -0 compresses 783 MB/s and decompresses 809 MB/s zstd 0.7.1 -1 compresses 586 MB/s and decompresses 1691 MB/s brotli 0.4.0 -7 compresses 57 MB/s, decompresses 873 MB/s to 28185 bytes brotli 0.4.0 -11 compresses to 25413 bytes zstd 0.7.1 -22 compresses in 4.01 MB/s to 28363 bytes Of course it is an unfair example because of the static dictionary that brotli uses, but it is not a pathological example: Thai is not part of the static dictionary. The numbers are on a i7-4790K@4.00 GHz. Brotli's fastest compression is faster than that of zstd, at least as shown with lzbench and this file. Also brotli wins in compression density. In this file the win is 10.5 % less bytes for brotli -11 than for zstd -22.

blank_state 10y ago
> Of course it is an unfair example because of the static dictionary that brotli uses It is certainly a favorable ground for Brotli. Brotli claims an advantage in html compression, thanks to its integrated specialized dictionary. The real pb though is the suggested conclusion that these favorable results are broadly applicable everywhere else. That's a terrible suggestion. We need more examples, not just "html files" which happen to be Brotli's best case. > brotli 0.4.0 -11 compresses to 25413 bytes > zstd 0.7.1 -22 compresses in 4.01 MB/s to 28363 bytes Why you don't disclose the compression time of brotli ? Of course it does matter : everyone understand that an algorithm that spend 10x more cpu has the budget to compress more. > brotli 0.4.0 -0 compresses 783 MB/s and decompresses 809 MB/s > zstd 0.7.1 -1 compresses 586 MB/s and decompresses 1691 MB/s Here, you don't disclose the compression ratio of both algorithm, implying they are equal. By such standard, LZ4 is probably the best : it's so much faster ! Of course, they do not compress the same... I was initially thrilled at your detailed answer, but now, quite frankly, I feel cheated. Grossly so. This is really disappointing. I was so much vexed that I decided to run the tests myself. Downloading and using __the same html file__, the same lzbench, same library versions, just a different computer and compiler, here is what it produced : | Algo | compressed size | compression speed | decompression speed | | --------- | --------------- | ----------------- | ------------------- | | brotli -2 | 36223 | 220 MB/s | 670 MB/s | | zstd -1 | 36655 | 480 MB/ | 1400 MB/s | | brotli -1 | 38292 | 360 MB/s | 650 MB/s | | brotli -0 | 41141 | 560 MB/s | 610 MB/s | __Conclusion__ : brotli -0 is indeed fast, faster than in my previous tests. It seems to be tuned to reach this objective, but throw away a lot of compression ratio to get there. Consequently, brotli -0 is not comparable to zstd -1, it takes brotli -2 to produce an equivalent compressed size . By that time though, zstd is much, much faster. Which is exactly the question I'm trying to get answers to : which algorithm compresses better for a given speed budget ? That's what matters, at least in our datacenter. I'm not interested in ultra slow mode, but while at it, I wanted to complete the picture with the missing compression speed of brotli - 11. It produced : zstd -22 : 2.95 MB/s brotli -11 : 0.53 MB/s So that's > 5x difference. It surely helps to reach better compression ratios. I also wanted an answer to "by how much the dictionary helps ?". Fortunately, TurboBench can help, thanks to a special mode which turns off dictionary compression. By using it on the very same sample, brotli -11 compressed size increases from 25413 to 26639 bytes. 5% larger, clearly not negligible. Still good, but it cuts the advertised size difference in half. Anyway, clearly I feel disappointed to have to redo the tests myself, because some inconvenient results were intentionally undisclosed (or not produced). This really undermines my trust in future publications. That learned me something : trust only benchmark done by yourself. And now, I should probably benchmark even more ...