I imagine the timing of this post is correlated with release of the documentary Bit Player about Claude Shannon. Haven't seen it yet but looking forward to it.

I imagine the timing of this post is correlated with release of the documentary Bit Player about Claude Shannon. Haven't seen it yet but looking forward to it. The article does a decent job at graphing and laying out some of the concepts of entropy for information theory, but I'm not sure who the target reader is, since prereqs are perhaps only slightly narrower than what one needs to read Shannon's paper[0] and the article is really illustrating only a fraction of the concept. It can perhaps work as a primer for what shows up starting on pages 10-11 of the original document, in any case, provided you grasp the mathematical definition of entropy through thermodynamics, and the microstates-based definition through Boltzman, as well as "basic probabilities" (expected value, typical discrete distributions, terms like "i.i.d"), you should be good to go. But then you might already know all this.. And if you do, and you like what you read, then the full original thing by Shannon is a delight to explore to truly grasp what has been so foundational to a lot of things since 1948. [0] http://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf http://people.math.harvard.edu/~ctm/home/text/others/shannon...