4 ms·

It's incredible how Shannon published "A Mathematical Theory of Communication", while actually working on the "Communication Theory of Secrecy Systems": https:

It's incredible how Shannon published "A Mathematical Theory of Communication", while actually working on the "Communication Theory of Secrecy Systems": https://pages.cs.wisc.edu/~rist/642-spring-2014/shannon-secrecy.pdf https://pages.cs.wisc.edu/~rist/642-spring-2014/shannon-secr... https://pure.mpg.de/rest/items/item_2383164/component/file_2383163/content https://pure.mpg.de/rest/items/item_2383164/component/file_2... What an absolute genius who disrupted two fields in two years. I was exposed to Shannon's channel capacity theorem of a binary erasure channel in a grad comms theory course, and totally fell in love with the beauty and elegance of his mathematical formulations. Since that day, teaching Information Theory has been my most favorite hobby--which also used to a career. I love to see students' eyes pop when you eventually show how elegantly simple it is to express disorder (entropy) and to find fundamental limits on source and channel codes using this expression.

cosmojg 3y ago
Heh, I think I'm going to start bringing up this paper whenever somebody tells me they majored in communications. This stuff is fascinating! Shannon really was a genius.
UncleOxidant 3y ago
3 fields - Don't forget his Masters thesis which revolutionized digital circuit design.
- nickpeterson 3y ago
  Yeah, I just re-encountered Shannon while reading about the board game Hex. The Shannon Switching game is the base case of Hex. Guy was firing on all cylinders ;)
- akhayam 3y ago
  Yes, yes. How did I forget this one? Thanks for calling it out. Wish we had more MS theses today that at least attempted something as ambitious as this one, instead of just checking a box :(
  - UncleOxidant 3y ago
    Shannon's Masters thesis was probably the most consequential Masters thesis of the 20th century (and the 21st century so far).
    - akhayam 3y ago
      100% agree. Jack Kilby's Masters thesis that realized an actual integrated circuit was another one that makes my list of GOAT.
padobson 3y ago
I read Gleick's The Information[0] a few years ago and was riveted. Do you have any suggestions for further reading if I wanted to take a hobbyist interest? [0]https://en.wikipedia.org/wiki/The_Information:_A_History,_a_Theory,_a_Flood https://en.wikipedia.org/wiki/The_Information:_A_History,_a_...
- akhayam 3y ago
  I would read "Lifespan" by David Sinclair, who used Shannon's information theory to model epigenetic changes that cause us to age. Fascinating read and goes to show how far reaching impacts a foundational mathematical theory can have.
pythonguython 3y ago
In EE we of course learned about the Shannon capacity of a channel, but when I read his proof using higher dimensional sphere packing I was amazed. Inspired me to attempt to look further into the math we see in engineering. I highly recommend anyone interested read his paper.
jhncls 3y ago
Claude Shannon also had early ideas about machine learning. AT&T has a 1950s video of the maestro himself demonstrating a maze solver with physical output. A true genius. In those days, geeks were dressed in suit. 0: https://techchannel.att.com/play-video.cfm/2010/3/16/In-Their-Own-Words-Claude-Shannon-Demonstrates-Machine-Learning https://techchannel.att.com/play-video.cfm/2010/3/16/In-Thei...
akhayam 3y ago
Since we are on the topic, does anyone know what Shannon was up to after disrupting 3 fields? ... He got into ball juggling and built the mathematical theory that led to creation of an automated juggling machine: https://cs.stanford.edu/people/eroberts/courses/soco/projects/1999-00/information-theory/juggling.html https://cs.stanford.edu/people/eroberts/courses/soco/project.... He actually showed that ball juggling is one of the most complex cognitive tasks that a human can undertake: "The physical constraints that affect mastery and limit the number of objects juggled arise from gravity - more specifically, Newtonian mechanics (h=1/2gt2). Each ball must be thrown sufficiently high to allow the juggler time to deal with the other balls. The need for either speed or height increases rapidly with the number of objects juggled."
- abecedarius 3y ago
  What else? Well, one thing was making a lot of money as an investor. I don't remember quite what Fortune's Formula said about his overall rate of return over many decades, but iirc he made less than Warren Buffett mainly because he started with much less. (That book is about information theory and betting.)
- aborsy 3y ago
  He was down for a period of time in his 40s or 50s (not sure of exact age), which might have been a prelude to what came next: Alzheimer’s disease.

cosmojg 3y ago

Heh, I think I'm going to start bringing up this paper whenever somebody tells me they majored in communications. This stuff is fascinating! Shannon really was a genius.

UncleOxidant 3y ago

3 fields - Don't forget his Masters thesis which revolutionized digital circuit design.

nickpeterson 3y ago
Yeah, I just re-encountered Shannon while reading about the board game Hex. The Shannon Switching game is the base case of Hex. Guy was firing on all cylinders ;)
akhayam 3y ago
Yes, yes. How did I forget this one? Thanks for calling it out. Wish we had more MS theses today that at least attempted something as ambitious as this one, instead of just checking a box :(
- UncleOxidant 3y ago
  Shannon's Masters thesis was probably the most consequential Masters thesis of the 20th century (and the 21st century so far).
  - akhayam 3y ago
    100% agree. Jack Kilby's Masters thesis that realized an actual integrated circuit was another one that makes my list of GOAT.

padobson 3y ago

I read Gleick's The Information[0] a few years ago and was riveted. Do you have any suggestions for further reading if I wanted to take a hobbyist interest? [0]https://en.wikipedia.org/wiki/The_Information:_A_History,_a_Theory,_a_Flood https://en.wikipedia.org/wiki/The_Information:_A_History,_a_...

akhayam 3y ago
I would read "Lifespan" by David Sinclair, who used Shannon's information theory to model epigenetic changes that cause us to age. Fascinating read and goes to show how far reaching impacts a foundational mathematical theory can have.

pythonguython 3y ago

In EE we of course learned about the Shannon capacity of a channel, but when I read his proof using higher dimensional sphere packing I was amazed. Inspired me to attempt to look further into the math we see in engineering. I highly recommend anyone interested read his paper.

jhncls 3y ago

Claude Shannon also had early ideas about machine learning. AT&T has a 1950s video of the maestro himself demonstrating a maze solver with physical output. A true genius. In those days, geeks were dressed in suit. 0: https://techchannel.att.com/play-video.cfm/2010/3/16/In-Their-Own-Words-Claude-Shannon-Demonstrates-Machine-Learning https://techchannel.att.com/play-video.cfm/2010/3/16/In-Thei...

akhayam 3y ago

Since we are on the topic, does anyone know what Shannon was up to after disrupting 3 fields? ... He got into ball juggling and built the mathematical theory that led to creation of an automated juggling machine: https://cs.stanford.edu/people/eroberts/courses/soco/projects/1999-00/information-theory/juggling.html https://cs.stanford.edu/people/eroberts/courses/soco/project.... He actually showed that ball juggling is one of the most complex cognitive tasks that a human can undertake: "The physical constraints that affect mastery and limit the number of objects juggled arise from gravity - more specifically, Newtonian mechanics (h=1/2gt2). Each ball must be thrown sufficiently high to allow the juggler time to deal with the other balls. The need for either speed or height increases rapidly with the number of objects juggled."

abecedarius 3y ago
What else? Well, one thing was making a lot of money as an investor. I don't remember quite what Fortune's Formula said about his overall rate of return over many decades, but iirc he made less than Warren Buffett mainly because he started with much less. (That book is about information theory and betting.)
aborsy 3y ago
He was down for a period of time in his 40s or 50s (not sure of exact age), which might have been a prelude to what came next: Alzheimer’s disease.