4 ms·
Brotli makes a bit of sense considering this is a static asset; it compresses somewhat more than zstd. This is why brotli is pretty ubiquitous for precompressed
by adzm 9mo ago
Brotli makes a bit of sense considering this is a static asset; it compresses somewhat more than zstd. This is why brotli is pretty ubiquitous for precompressed static assets on the Web.
That said, I personally prefer zstd as well, it's been a great general use lib.
- dist-epoch 9mo agoYou need to crank up zstd compression level. zstd is Pareto better than brotli - compresses better and faster
- jeffbee 9mo agoAre you sure? Admittedly I only have 1 PDF in my homedir, but no combination of flags to zstd gets it to match the size of brotli's output on that particular file. Even zstd --long --ultra -22.
- xxs 9mo agoon max compression (11 vs zstd's 22) of text brotli will be around 3-4% denser... and a lot slower. Decompression wise zstd is over 2x faster. The pdfs you have are already compressed with deflate (zip).
- atiedebee 9mo agoI thought the same, so I ran brotli and zstd on some PDFs I had laying around. brotli 1.0.7 args: -q 11 -w 24 zstd v1.5.0 args: --ultra -22 --long=31 | Original | zstd | brotli RandomBook.pdf | 15M | 4.6M | 4.5M Invoice.pdf | 19.3K | 16.3K | 16.1K I made a table because I wanted to test more files, but almost all PDFs I downloaded/had stored locally were already compressed and I couldn't quickly find a way to decompress them. Brotli seemed to have a very slight edge over zstd, even on the larger pdf, which I did not expect.
- order-matters 9mo agoWhats the assumption we can potentially target as reason for the counter-intuitive result? that data in pdf files are noisy and zstd should perform better on noisy files?
- jeffbee 9mo agoWhat's counter-intuitive about this outcome?
- order-matters 9mo agomaybe that was too strongly worded but there was an expectation for zstd to outperform. So the fact it didnt means the result was unexpected. i generally find it helpful to understand why something performs better than expected.
- mort96 9mo agoIsn't zstd primarily designed to provide decent compression ratios at amazing speeds? The reason it's exciting is mainly that you can add compression to places where it didn't necessarily make sense before because it's almost free in terms of CPU and memory consumption. I don't think it has ever had a stated goal of beating compression ratio focused algorithms like brotli on compression ratio.
- sgerenser 9mo agoI actually thought zstd was supposed to be better than Brotli in most cases, but a bit of searching reveals you're right... Brotli, especially at the highest compression levels (10/11), often exceeds zstd at the highest compression levels (20-22). Both are very slow at those levels, although perfectly suitable for "compress once, decompress many" applications which the PDF spec is obviously one of them.
- mort96 9mo agoEDIT: Something weird is going on here. When compressing zstd in parallel it produces the garbage results seen here, but when compressing on a single core, it produces result competitive with Brotli (37M). See: https://news.ycombinator.com/item?id=46723158 https://news.ycombinator.com/item?id=46723158 I did my own testing where Brotli also ended up better than ZSTD: https://news.ycombinator.com/item?id=46722044 https://news.ycombinator.com/item?id=46722044 Results by compression type across 55 PDFs: +------+------+-----+------+--------+ | none | zstd | xz | gzip | brotli | +------|------|-----|------|--------| | 47M | 45M | 39M | 38M | 37M | +------+------+-----+------+--------+
- dchest 9mo agoNot with small files.
- Dylan16807 9mo agoIf that's about using predefined dictionaries, zstd can use them too. If brotli has a different advantage on small source files, you have my curiosity. If you're talking about max compression, zstd likely loses out there, the answer seems to vary based on the tests I look at, but it seems to be better across a very wide range.
- dchest 9mo agoNo, it's literally just compressing small files without training zstd dict or plugging external dictionaries (not counting the built-in one that brotli has). Especially for English text, brotli at the same speed as zstd gives better results for small data (in kilobyte to a few of megabyte range).
- DetroitThrow 9mo agoI love zstd but this isn't necessarily true.
- itsdesmond 9mo ago> Pareto I don’t think you’re using that correctly.
- wizzwizz4 9mo agoIt's correct use of Pareto, short for Pareto frontier, if the claim being made is "for every needed compression ratio, zstd is faster; and for every needed time budget, zstd is faster". (Whether this claim is true is another matter.)
- stonogo 9mo agobrotli is ubiquitous because Google recommends it. While Deflate definitely sucks and is old, Google ships brotli in Chrome, and since Chrome is the de facto default platform nowadays, I'd imagine it was chosen because it was the lowest-effort lift. Nevertheless, I expect this to be JBIG2 all over again: almost nobody will use this because we've got decades of devices and software in the wild that can't, and 20% filesize savings is pointless if your destination can't read the damn thing.