In information theory you can't come up with an encoding that will give you more than a bit of information per bit of source on average. Of course, the informat

In information theory you can't come up with an encoding that will give you more than a bit of information per bit of source on average. Of course, the information we usually care about is usually highly structured, so we use encodings that compress the highly structured parts we care about and dont care if in that same encoding it would take many bits more to encode some other part. Ever used some sort of library or framework where in the beginning you get a ton of progress made in a small amount of time but when it comes to getting it to do exactly what you want you find your code balooning around the corner cases? Same idea in my opinion.