Shannon was certainly one of the great scientists. His paper "The Mathematical Theory of Communication" [1] is on the one hand accessible but profound. His deri

Shannon was certainly one of the great scientists. His paper "The Mathematical Theory of Communication" [1] is on the one hand accessible but profound. His derivation of entropy and the the limits of communication over a noisy channel are almost simple but I believe this is true only because of hindsight and is, in my opinion, the hallmark of a deep idea. We've only recently been able to get close to the Shannon limit with Turbo Codes [2] and Low Parity Density Checks (LPDC) [3]. For anyone interested, I wrote a short post on deriving entropy and channel capacity [4]. [1] http://math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf http://math.harvard.edu/~ctm/home/text/others/shannon/entrop... [2] https://en.wikipedia.org/wiki/Turbo_code https://en.wikipedia.org/wiki/Turbo_code [3] https://en.wikipedia.org/wiki/Low-density_parity-check_code https://en.wikipedia.org/wiki/Low-density_parity-check_code [4] https://mechaelephant.com/dev/Shannon-Entropy/ https://mechaelephant.com/dev/Shannon-Entropy/