6 ms·

Claude Shannon at Bell Labs

139 points by woodandsteel 9y ago

dabber 9y ago
Hmm, ieee.org still has Flash ads [Edit] Sorry maybe not a flash ad. Upon further inspection it was an iframe whose contents were blocked by my ad blocker, presenting me with that grey box in chrome. I'm not going to disable it to check what it actually is.
- nilved 9y ago
  That's "professional associations" for you
- jasonkostempski 9y ago
  It looks exactly like AdFly at first glance. Thought I was being directed to a Minecraft Mod.
chubot 9y ago
This sounds like a promising book. I read the The Idea Factory a few years ago, which is a related and fantastic book about the history of Bell Labs. Around that time I came across an interesting idea. I don't remember if it was in the Idea Factory, or in material I read afterward, but it's related to one of central ideas from this excerpt: The sender no longer mattered, the intent no longer mattered, the medium no longer mattered, not even the meaning mattered: A phone conversation, a snatch of Morse telegraphy, a page from a detective novel were all brought under a common code. The idea I came across is that: Shannon's information theory, devised at AT&T, indirectly led to the demise of AT&T's monopoly. Before Shannon, there was no concept of the information-carrying capacity of a wire. And AT&T's monopoly was largely due to it having the biggest set of wires, which as you can imagine were expensive to deploy in the 19th/20th century. I remember that calling California from NYC was a huge achievement, precisely because of the number of physical wires that had to be connected. AT&T was the first to offer that service. So I think the argument was that it made economic sense for a single organization to own all the wires, so it could maintain them with a set of common specifications and processes. But if you can reduce every wire to a single number -- its information-carrying capacity -- then this argument goes out the window. You can use all sorts of heterogeneous links made by different manufacturers and maintained by different companies. (I'm not sure if this is historically accurate, but technically it sounds true.) So my thought was that there's an analogous breakthrough waiting to happen with respect to cloud computing. Google and Facebook have information monopolies based on centralized processing of big data in custom-built data centers. Likewise, AWS has a strong network effect, and is hard to compete with even if you have billions of dollars to spend. So my question is: Is it possible there will be a breakthrough in decentralized distributed computing? And could it make obsolete the centralized cloud computing that Google/Facebook/Amazon practice? Just like AT&T had no reason to be a monopoly after Shannon, maybe a technological breakthrough will be the end of Google/Facebook/Amazon. Maybe this idea is too cute, and you can poke holes in it on a number of fronts, e.g.: - Shannon's ideas were profound, but they didn't actually bring down AT&T. AT&T was forcibly broken apart, and there are still network effects today that make re-mergers rational. - Centralized distributed computing will always be more efficient than decentralized distributed computing (?) I'm not aware of any fundamental theorems here but it seems within the realm of possibility. (EDIT: On further reflection, the main difference between centralized and decentralized is trust, so maybe they're not comparable. Decentralized algorithms always do more work because they have to deal with security and conflicting intentions.) But still I like the idea that merely an idea could end an industry :) Relatedly, I also recall that Paul Graham argued that there will be more startups because of decreasing transaction costs between companies, or something like that. But it still feels like the computer industry is inherently prone to monopolies, and despite what pg said, the big companies still control as much of the industry that Microsoft did back in the day, or maybe more.
- andars 9y ago
  I believe you are thinking of The Idea Factory.
  - chubot 9y ago
    Oops thanks, corrected. The Genius Factory was also a good book, about a crazy effort to make a Nobel Prize sperm bank. :) If I recall correctly, that book included a distant relative of Edward Teller, or possibly it was some other Bell Labs luminary.
- zrm 9y ago
  > So I think the argument was that it made economic sense for a single organization to own all the wires, so it could maintain them with a set of common specifications and processes. But if you can reduce every wire to a single number -- its information-carrying capacity -- then this argument goes out the window. You can use all sorts of heterogeneous links made by different manufacturers and maintained by different companies. > (I'm not an electrical engineer, so I have no idea if this is all true, but it sounds plausible.) You're describing the internet. :) > Is it possible there will be a breakthrough in decentralized distributed computing? The hard problem is security. Right now you have to trust Amazon et al with your data, which is not really what you want, but even that is better than having to trust Some Guy running a host out of his garage. This isn't a real problem for static content. That you can just encrypt, throw it up on IPFS or similar and add federated authentication for access to the decryption keys. But data processing is something else. There are people trying to solve that with encryption too, but it's hard, and to a large extent equivalent to making effective DRM, which is not a desirable thing for your problem to be equivalent to. A different approach is to trust people who you actually trust. We're now at the point that a fast processor consumes a single digit number of watts. So you can run your own server at home, and so can your friends and family, which allows you to pool capacity for load sharing and higher availability. Then services become software you install on your trusted pool of servers. There is no technical reason that can't exist, people just haven't done it yet (or they have but the future is not evenly distributed).
  - 9y ago
B1FF_PSUVM 9y ago
> many of Shannon’s colleagues found themselves working six days a week. Hmm. Was the 5 day work week common by 1940? I have a notion that we went from Sunday off only, to Sunday plus Saturday afternoon, then Sunday plus Saturday off. Not sure when that happened, or where it started.
keithpeter 9y ago
Interesting links off the OA to the generation after Shannon who implemented a lot of the ideas. >> Reflecting on this time later, he remembered the flashes of intuition. The work wasn’t linear; ideas came when they came. “One night I remember I woke up in the middle of the night and I had an idea and I stayed up all night working on that.” << Do people think that putting Shannon somewhere like the Institute for Advanced Study would actually have quickened up his thinking? Or is a level of distracting background activity actually helpful?
- gwern 9y ago
  I don't think it would have helped, but I suspect it's less a distracting background offering serendipity/weak contacts and more basic expectations of finishing and communicating things. The IAS has something of a bad reputation for fostering rest rather than revolution (I think both Hamming and Feynman criticized it in rather strong terms). You should also remember that Shannon was given carte blanch at MIT because he was so famous and respected, and that's exactly when his public output went to zero. Not that he didn't keep himself busy (he did a ton of stock trading and other things which is where Shannon's volatility harvesting comes from etc) but without any kind of external constraint or direction... I was shocked to learn that Claude Shannon died in 2001, because from how he drops out of all histories in the '50s-60s, I always sort of assumed he had died around then, relatively young. (Probably a fair number of HNers could learn from Shannon's bad example: shipping matters!)
  - chubot 9y ago
    Huh? He did more than a lifetime of work, and happened to do it early in life. It sounds like he worked on whatever was interesting to him. If those things weren't interesting to others, so be it. I find it very odd that you would call this a bad example and relate it to the relatively prosaic idea of "shipping". Shipping is good when you need feedback, but that's not the kind work he was doing.
    - gwern 9y ago
      He did less than a lifetime of work because he did almost all of his work early in his lifetime. Then he selfishly betrayed his agreement with society and MIT by spending decades of tenure fiddling with his toys, and underachieved his potential. He probably could have done so much more with his talent if he had just some more structure to his life. That is why he is a cautionary example - he was a victim of his own success. You can have all the talent in the world and casually revolutionize fields as different as genetics and electronics when you bother to finish something, but if you don't put in the work, nothing will happen and you will squander it all. > Shipping is good when you need feedback, but that's not the kind work he was doing. No, Shannon was a genius and didn't need much in the way of feedback. Shipping is good because it requires you to complete the project and take it from 99% finished & only 1% useful to 100% finished & 100% useful, and it makes it available to the rest of the world, instead of buried in your estate's papers for a journalist like William Poundstone to uncover decades later after it would have been useful if it had been completed & published in a timely fashion.
losteverything 9y ago
What jumped out was the mere mention of the draft (wwii). I even looked it up. The narrative my wwii relatives give is everyone enlisted. If you didnt there was shame that required an explanation
- tyingq 9y ago
  ~60% of US military in the WW II time period were draftees. https://www.nationalww2museum.org/students-teachers/student-resources/research-starters/research-starters-us-military-numbers https://www.nationalww2museum.org/students-teachers/student-... Doesn't refute the idea that shame might have been involved, but...
  - losteverything 9y ago
    My history could be very distorted but the "war is bad and to be avoided" idea came to life (in my lifetime) during Vietnam years. I cant imagine the us population rallying around a new war like they did in the '40's
    - tyingq 9y ago
      Only about 25% of US military in Vietnam were draftees. Suggests there were 2 distinct trains of thought on "war is bad".
    - maroonblazer 9y ago
      If something akin to a "Pearl Harbor" struck us today I could totally see us rallying around a new war.
    - thearn4 9y ago
      We saw similar rallying in the year following 9-11 and the initial invasion of Afghanistan. With the invasion of Iraq in 2003, it became more complicated very quickly.
- joatmon-snoo 9y ago
  The Idea Factory does note that there were Bell Labs engineers who were chided on the street by strangers for not being overseas.

139 points by woodandsteel 9y ago

Hmm, ieee.org still has Flash ads [Edit] Sorry maybe not a flash ad. Upon further inspection it was an iframe whose contents were blocked by my ad blocker, presenting me with that grey box in chrome. I'm not going to disable it to check what it actually is.

nilved 9y ago
That's "professional associations" for you
jasonkostempski 9y ago
It looks exactly like AdFly at first glance. Thought I was being directed to a Minecraft Mod.

chubot 9y ago

This sounds like a promising book. I read the The Idea Factory a few years ago, which is a related and fantastic book about the history of Bell Labs. Around that time I came across an interesting idea. I don't remember if it was in the Idea Factory, or in material I read afterward, but it's related to one of central ideas from this excerpt: The sender no longer mattered, the intent no longer mattered, the medium no longer mattered, not even the meaning mattered: A phone conversation, a snatch of Morse telegraphy, a page from a detective novel were all brought under a common code. The idea I came across is that: Shannon's information theory, devised at AT&T, indirectly led to the demise of AT&T's monopoly. Before Shannon, there was no concept of the information-carrying capacity of a wire. And AT&T's monopoly was largely due to it having the biggest set of wires, which as you can imagine were expensive to deploy in the 19th/20th century. I remember that calling California from NYC was a huge achievement, precisely because of the number of physical wires that had to be connected. AT&T was the first to offer that service. So I think the argument was that it made economic sense for a single organization to own all the wires, so it could maintain them with a set of common specifications and processes. But if you can reduce every wire to a single number -- its information-carrying capacity -- then this argument goes out the window. You can use all sorts of heterogeneous links made by different manufacturers and maintained by different companies. (I'm not sure if this is historically accurate, but technically it sounds true.) So my thought was that there's an analogous breakthrough waiting to happen with respect to cloud computing. Google and Facebook have information monopolies based on centralized processing of big data in custom-built data centers. Likewise, AWS has a strong network effect, and is hard to compete with even if you have billions of dollars to spend. So my question is: Is it possible there will be a breakthrough in decentralized distributed computing? And could it make obsolete the centralized cloud computing that Google/Facebook/Amazon practice? Just like AT&T had no reason to be a monopoly after Shannon, maybe a technological breakthrough will be the end of Google/Facebook/Amazon. Maybe this idea is too cute, and you can poke holes in it on a number of fronts, e.g.: - Shannon's ideas were profound, but they didn't actually bring down AT&T. AT&T was forcibly broken apart, and there are still network effects today that make re-mergers rational. - Centralized distributed computing will always be more efficient than decentralized distributed computing (?) I'm not aware of any fundamental theorems here but it seems within the realm of possibility. (EDIT: On further reflection, the main difference between centralized and decentralized is trust, so maybe they're not comparable. Decentralized algorithms always do more work because they have to deal with security and conflicting intentions.) But still I like the idea that merely an idea could end an industry :) Relatedly, I also recall that Paul Graham argued that there will be more startups because of decreasing transaction costs between companies, or something like that. But it still feels like the computer industry is inherently prone to monopolies, and despite what pg said, the big companies still control as much of the industry that Microsoft did back in the day, or maybe more.

andars 9y ago
I believe you are thinking of The Idea Factory.
- chubot 9y ago
  Oops thanks, corrected. The Genius Factory was also a good book, about a crazy effort to make a Nobel Prize sperm bank. :) If I recall correctly, that book included a distant relative of Edward Teller, or possibly it was some other Bell Labs luminary.
zrm 9y ago
> So I think the argument was that it made economic sense for a single organization to own all the wires, so it could maintain them with a set of common specifications and processes. But if you can reduce every wire to a single number -- its information-carrying capacity -- then this argument goes out the window. You can use all sorts of heterogeneous links made by different manufacturers and maintained by different companies. > (I'm not an electrical engineer, so I have no idea if this is all true, but it sounds plausible.) You're describing the internet. :) > Is it possible there will be a breakthrough in decentralized distributed computing? The hard problem is security. Right now you have to trust Amazon et al with your data, which is not really what you want, but even that is better than having to trust Some Guy running a host out of his garage. This isn't a real problem for static content. That you can just encrypt, throw it up on IPFS or similar and add federated authentication for access to the decryption keys. But data processing is something else. There are people trying to solve that with encryption too, but it's hard, and to a large extent equivalent to making effective DRM, which is not a desirable thing for your problem to be equivalent to. A different approach is to trust people who you actually trust. We're now at the point that a fast processor consumes a single digit number of watts. So you can run your own server at home, and so can your friends and family, which allows you to pool capacity for load sharing and higher availability. Then services become software you install on your trusted pool of servers. There is no technical reason that can't exist, people just haven't done it yet (or they have but the future is not evenly distributed).
- 9y ago

B1FF_PSUVM 9y ago

> many of Shannon’s colleagues found themselves working six days a week. Hmm. Was the 5 day work week common by 1940? I have a notion that we went from Sunday off only, to Sunday plus Saturday afternoon, then Sunday plus Saturday off. Not sure when that happened, or where it started.

keithpeter 9y ago

Interesting links off the OA to the generation after Shannon who implemented a lot of the ideas. >> Reflecting on this time later, he remembered the flashes of intuition. The work wasn’t linear; ideas came when they came. “One night I remember I woke up in the middle of the night and I had an idea and I stayed up all night working on that.” << Do people think that putting Shannon somewhere like the Institute for Advanced Study would actually have quickened up his thinking? Or is a level of distracting background activity actually helpful?

gwern 9y ago
I don't think it would have helped, but I suspect it's less a distracting background offering serendipity/weak contacts and more basic expectations of finishing and communicating things. The IAS has something of a bad reputation for fostering rest rather than revolution (I think both Hamming and Feynman criticized it in rather strong terms). You should also remember that Shannon was given carte blanch at MIT because he was so famous and respected, and that's exactly when his public output went to zero. Not that he didn't keep himself busy (he did a ton of stock trading and other things which is where Shannon's volatility harvesting comes from etc) but without any kind of external constraint or direction... I was shocked to learn that Claude Shannon died in 2001, because from how he drops out of all histories in the '50s-60s, I always sort of assumed he had died around then, relatively young. (Probably a fair number of HNers could learn from Shannon's bad example: shipping matters!)
- chubot 9y ago
  Huh? He did more than a lifetime of work, and happened to do it early in life. It sounds like he worked on whatever was interesting to him. If those things weren't interesting to others, so be it. I find it very odd that you would call this a bad example and relate it to the relatively prosaic idea of "shipping". Shipping is good when you need feedback, but that's not the kind work he was doing.
  - gwern 9y ago
    He did less than a lifetime of work because he did almost all of his work early in his lifetime. Then he selfishly betrayed his agreement with society and MIT by spending decades of tenure fiddling with his toys, and underachieved his potential. He probably could have done so much more with his talent if he had just some more structure to his life. That is why he is a cautionary example - he was a victim of his own success. You can have all the talent in the world and casually revolutionize fields as different as genetics and electronics when you bother to finish something, but if you don't put in the work, nothing will happen and you will squander it all. > Shipping is good when you need feedback, but that's not the kind work he was doing. No, Shannon was a genius and didn't need much in the way of feedback. Shipping is good because it requires you to complete the project and take it from 99% finished & only 1% useful to 100% finished & 100% useful, and it makes it available to the rest of the world, instead of buried in your estate's papers for a journalist like William Poundstone to uncover decades later after it would have been useful if it had been completed & published in a timely fashion.

losteverything 9y ago

What jumped out was the mere mention of the draft (wwii). I even looked it up. The narrative my wwii relatives give is everyone enlisted. If you didnt there was shame that required an explanation

tyingq 9y ago
~60% of US military in the WW II time period were draftees. https://www.nationalww2museum.org/students-teachers/student-resources/research-starters/research-starters-us-military-numbers https://www.nationalww2museum.org/students-teachers/student-... Doesn't refute the idea that shame might have been involved, but...
- losteverything 9y ago
  My history could be very distorted but the "war is bad and to be avoided" idea came to life (in my lifetime) during Vietnam years. I cant imagine the us population rallying around a new war like they did in the '40's
  - tyingq 9y ago
    Only about 25% of US military in Vietnam were draftees. Suggests there were 2 distinct trains of thought on "war is bad".
  - maroonblazer 9y ago
    If something akin to a "Pearl Harbor" struck us today I could totally see us rallying around a new war.
  - thearn4 9y ago
    We saw similar rallying in the year following 9-11 and the initial invasion of Afghanistan. With the invasion of Iraq in 2003, it became more complicated very quickly.
joatmon-snoo 9y ago
The Idea Factory does note that there were Bell Labs engineers who were chided on the street by strangers for not being overseas.