4 ms·
Zstd is very different - it includes an entropy coder. LZ4 only finds repeated matches, but then doesn't encode them very efficiently. To put it simplistically
by unsigner 6y ago
Zstd is very different - it includes an entropy coder.
LZ4 only finds repeated matches, but then doesn't encode them very efficiently.
To put it simplistically, if you have a file which is a (good) random mix of an equal number A and B characters, LZ4 won't be able to compress it significantly, while Zstd will compress it 8:1 converging to an encoding where a '1' bit is A, and a '0' bit is B.
- YetAnotherNick 6y ago> To put it simplistically, if you have a file which is a (good) random mix of an equal number A and B characters, LZ4 won't be able to compress it significantly I checked it. LZ4 is still reducing the size to half, no idea why half. So for 10 MB file it compresses to 5 MB. Edit: checked with highest compression and it compresses 1MB file to 185KB. So what the parent wrote is false.
- not2b 6y agoYes, if I take the 8 combinations aaa, aab, aba etc and assign each of them a 9 bit codeword I replace each 24 bit sequence with a 9 bit sequence. So arithmetic coders have no problem with cases like this.
- unsigner 6y agobut LZ4 doesn't have a arithmetic coder, or any other statistical encoding - it's just matches and literals. Puzzling...
- livvelupp 6y agoYetAnother*ick