4 ms·
Claude Shannon’s research laid foundations for modern communications (2020)
- rogerkirkness 4y agoCertainly it seems like Shannon is turning out to be the lindy Nicola Tesla equivalently impactful person but in the realm of bits.
- 082349872349872 4y ago> ...in the realm of bits. Especially as an OG (first author to use the word?): https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf https://people.math.harvard.edu/~ctm/home/text/others/shanno... > If the base 2 is used the resulting units may be called binary digits, or more briefly bits, a word suggested by J. W. Tukey. [Edit: also, consider the section "3. THE SERIES OF APPROXIMATIONS TO ENGLISH" to be an early exploration of MLMs: Microscopic Language Models]
- kkylin 4y agoJohn Tukey also invented FFT and apparently was the first to use "software" in a publication: https://en.wikipedia.org/wiki/John_Tukey https://en.wikipedia.org/wiki/John_Tukey
- yesenadam 4y agoGauss invented FFT, in 1805. And "Between 1805 and 1965, some versions of FFT were published by other authors" https://en.wikipedia.org/wiki/Fast_Fourier_transform#History https://en.wikipedia.org/wiki/Fast_Fourier_transform#History p.s. I started reading Tukey's textbook Exploratory Data Analysis just yesterday. It's wonderful so far, a pleasure to read. "It is important to understand what you CAN DO before you learn to measure how WELL you seem to have DONE it." "The greatest value of a picture is when it forces us to notice what we never expected to see."
- disconcision 4y agoyes but colloquially we take 'inventor' to mean whoever invented it next after guass
- yesenadam 4y ago:-) It seems Cooley and Tukey were the first to realize/state that it's O(N log N), which is something.
- amelius 4y agoNot just communication, but many other fields. E.g. deep learning is rooted in information theory.
- LudwigNagasena 4y agoThe Kelly criterion that is widely used in finance was invented at the Bell Labs. It was based on the Shannon’s theory.
- mhh__ 4y ago> widely used A nice idea but not really.
- User23 4y agoI use it every time I’m in Vegas.
- sn41 4y agoKelly criterion was originally invented by Daniel Bernoulli (using logarithmic utility) as a resolution to the St. Petersburg Paradox, IIRC: https://en.wikipedia.org/wiki/St._Petersburg_paradox https://en.wikipedia.org/wiki/St._Petersburg_paradox
- kqr 4y agoNot specifically a solution to the St. Petersburg Paradox because, well, it's not a solution to the most general formulation of it (and Bernoulli admits as much in his paper, IIRC). At least the way I read Bernoulli's paper, it was more of a general musing on how to reduce risk and make insurance decisions.
- aborsy 4y agoWell, the notion of bit as the unit of information, as well as digitization and digital communication, came primarily out of Shannon’s work.
- notlukesky 4y agoThe suggestion to use the word entropy by von Neumann is a great story: My greatest concern was what to call it. I thought of calling it 'information,' but the word was overly used, so I decided to call it 'uncertainty.' When I discussed it with John von Neumann, he had a better idea. Von Neumann told me, 'You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, no one really knows what entropy really is, so in a debate you will always have the advantage.' https://mathoverflow.net/questions/403036/john-von-neumanns-remark-on-entropy https://mathoverflow.net/questions/403036/john-von-neumanns-...
- mitthrowaway2 4y agoIt's a funny story, but now that we do understand better what entropy is it's clear that information and thermodynamic entropy are the same concept, so it was a very good call.
- User23 4y agoVon Neumann was incredibly clever.
- bee_rider 4y agovon Neumann was an alien sent to Earth when we were about to get/were in the process of getting nukes, to make sure we advanced fast enough to not blow ourselves up. And the wild thing is that you, the reader, are only like 75% sure I’m kidding and don’t actually believe this.
- mitthrowaway2 4y agoYes, it's likely he also understood that they were the same concept, and his recommendation was less of a joke than his words suggest.
- bee_rider 4y agoShannon’s work has already produced a whole lot of course, but I don’t think we’re anywhere near seeing the full outcome of his contributions yet. Currently our computers operate by filling the transistors with charge and/or dumping it to ground. Who even cares about in information-theoretic efficiency in that case? The cost of the actual work done is dwarfed by the ancillary cost of running the machine. If we ever move on to something less brute-force, like reversible quantum cellular automata, I think we’ll see him as an invaluable part in the chain of formalizing what information and computation mean physically. Kelvin/Maxwell -> Shannon -> Landauer -> maybe Bennett
- 2devnull 4y agoWas rereading “Fortune’s Formula” this morning after dreaming about Shannon last night. A name that should be more widely recognized.
- dang 4y agoDiscussed at the time (of the article): Claude Shannon Invented the Future - https://news.ycombinator.com/item?id=25507627 https://news.ycombinator.com/item?id=25507627 - Dec 2020 (68 comments)
- pm90 4y agoSlightly OT: have you considered using some kind of automation to generate these messages? ( or perhaps this is an automated message).
- dang 4y agoI use software to help build the lists quickly. They're still hand-picked, though, because an automated solution tends to come up with a lot of duds (e.g. threads with no interesting commments).
- rdxm 4y ago[dead]
- tsuru 4y agoI learned recently from Alvy Ray Smith's _A Biography of the Pixel_ that Kotelnikov[1] was first to discover sampling theorem before Shannon independently did. And Shannon made it accessible to the West. It's pretty fascinating history. [1] https://en.m.wikipedia.org/wiki/Vladimir_Kotelnikov https://en.m.wikipedia.org/wiki/Vladimir_Kotelnikov
- manv1 4y agoEvery year I read Shannon's paper, and every year I realize I have no idea WTF he was talking about.