I'm pretty sure all PNG is doing here is exploiting the fact that some characters are much more common than others. It's implicitly creating a unigram character

I'm pretty sure all PNG is doing here is exploiting the fact that some characters are much more common than others. It's implicitly creating a unigram character model and using that to encode. There may also be a tiny bit of benefit related to the fact that in English if you see a letter, you're slightly more likely to see that letter again immediately after, as in words like "battle", "sonnet", "thee", etc. I'm surprised that PNG does better than ZIP; I'd be willing to be that ZIP does better if you scale up to the full Shakespeare corpus. ZIP will start to utilize word probabilities if you give it enough text (e.g. it will learn that "the" is more common than "eth"). > Image compression algorithms are, generally, pretty good at finding patterns in images and squashing them down. This really isn't true in practice. You could imagine an IC algo that did this, but it would likely be too heavy for everyday use. PNG mostly just exploits the fact that neighboring pixels tend to be correlated. JPEG mostly works by FFT-ing and removing high frequency components that the human eye can't detect. If you want to play around with lossless compression, try out my "BYOM" (Bring Your Own Model) library for arithmetic encoding: https://github.com/comperical/MirrorEncode https://github.com/comperical/MirrorEncode