3 ms·

> advanced compression (finite-state entropy, like ZStandard) I've not tried it on images, but wouldn't zstandard be exceedingly bad at gradients? It completel

by Aachen 1y ago

> advanced compression (finite-state entropy, like ZStandard) I've not tried it on images, but wouldn't zstandard be exceedingly bad at gradients? It completely fails to compress numbers that change at a fixed rate Bzip2 does that fine, not sure why https://chaos.social/@luc/114531687791022934 https://chaos.social/@luc/114531687791022934 The two variables (inner and outer loop) could be two color channels that change at different rates. Real-world data will never be a clean i++ like it is here, but more noise surely isn't going to help the algorithm compared to this clean example

Retr0id 1y ago
zlib/deflate already has the same issue. It is mitigated somewhat by PNG row filters.
adgjlsfhk1 1y ago
the FSE layer isn't responsible for finding these sorts of patterns in an image codec. The domain modeling turns that sort of pattern into repeated data and then the FSE goes to town on the output.
wongarsu 1y ago
PNG's basic idea is to store the difference between the current pixel and the pixel above it, left of it or to the top-left (chosen once per row), then apply standard deflate compression to that. The first step basically turns gradients into repeating patterns of small numbers, which compress great. You can get decent improvements by just switching deflate for zstd
- Aachen 1y ago
  > You can get decent improvements by just switching deflate for zstd Maybe I sounded too critical of zstd. To be clear: I use it for general-purpose compression where available, the only exception would be where eeking out the last % gain is important and slow decompression is acceptable and it has one of these patterns that Bzip2 does better in the first place That it's better than deflate (afaik aka gzip and zlib, just with different header fields) is not surprising since that was iirc the defined goal of Zstandard project
spider-mario 1y ago
JPEG XL is not “apply zstd to the pixel data”. What the parent comment meant is that JPEG XL, like zstd, happens to use ANS among its set of techniques. It handles gradients just fine.

by Aachen 1y ago

Retr0id 1y ago

zlib/deflate already has the same issue. It is mitigated somewhat by PNG row filters.

adgjlsfhk1 1y ago

the FSE layer isn't responsible for finding these sorts of patterns in an image codec. The domain modeling turns that sort of pattern into repeated data and then the FSE goes to town on the output.

wongarsu 1y ago

PNG's basic idea is to store the difference between the current pixel and the pixel above it, left of it or to the top-left (chosen once per row), then apply standard deflate compression to that. The first step basically turns gradients into repeating patterns of small numbers, which compress great. You can get decent improvements by just switching deflate for zstd

Aachen 1y ago
> You can get decent improvements by just switching deflate for zstd Maybe I sounded too critical of zstd. To be clear: I use it for general-purpose compression where available, the only exception would be where eeking out the last % gain is important and slow decompression is acceptable and it has one of these patterns that Bzip2 does better in the first place That it's better than deflate (afaik aka gzip and zlib, just with different header fields) is not surprising since that was iirc the defined goal of Zstandard project

spider-mario 1y ago

JPEG XL is not “apply zstd to the pixel data”. What the parent comment meant is that JPEG XL, like zstd, happens to use ANS among its set of techniques. It handles gradients just fine.