3 ms·
> compression tends to increase the entropy of a bit stream. Does it? Encryption increases entropy, but not sure about compression.
by puttycat 1y ago
> compression tends to increase the entropy of a bit stream.
Does it?
Encryption increases entropy, but not sure about compression.
- JCBird1012 1y agoI can see what was meant with that statement. I do think compression increases Shannon entropy by virtue of it removing repeating patterns of data - Shannon entropy per byte of compressed data increases since it’s now more “random” - all the non-random patterns have been compressed out. Total information entropy - no. The amount of information conveyed remains the same.
- gary_0 1y agoTechnically with lossy compression, the amount of information conveyed will likely change. It could even increase the amount of information of the decompressed image, for instance if you compress a cartoon with simple lines and colors, a lossy algorithm might introduce artifacts that appear as noise.
- gregdeon 1y agoYes: the reason why some data can be compressed is because many of its bits are predictable, meaning that it has low entropy per bit.