4 ms·

What's an intelligence explosion? Intelligence dissemination or deletion? EDIT: Nevermind, just looked it up and found this was a real thing. I thought you w

What's an intelligence explosion? Intelligence dissemination or deletion? EDIT: Nevermind, just looked it up and found this was a real thing. I thought you were joking.

astrofinch 11y ago
It sounds less weird if you think: "AI has automated some jobs, and eventually it may automate away most of them, in the same way (pending AlphaGo victory) it's now automated winning at most board games... So what happens if the job of programmer gets automated away?" (I'm convinced that programmers will take the AI-automating-all-the-jobs idea more seriously once it's their own jobs that are on the line) There are a few ways I can think of to object to this line of reasoning: 1) You could argue that programming will be a job that never gets automated away. But this seems unlikely--previous intellectual tasks (Chess, Jeopardy, Go) were thought to be "essentially human", and once they were solved by computers, got redefined as not "essentially human" (and therefore not AI). My opinion: In the same way we originally thought tool use made humans unique, then realized that other animals use tools too, we'll eventually learn that there's no fundamental uniqueness to human cognition. It'll be matter & algorithms all the way down. Of course, the algorithms could be really darn complicated. But the fact the Deepmind team won at both Go and Atari using the same approach suggests the existence of important general-purpose algorithms that are within the reach of human software engineers. 2) You could argue that programming will be automated away but in a sense that isn't meaningful (e.g. you need an expensive server farm to replace a relatively cheap human programmer). This is certainly possible. But in the same way calculators do arithmetic much faster & better than humans do, there's the possibility that automated computer programmers will program much faster & better than humans. (Honestly humans suck so hard at programming https://twitter.com/bramcohen/status/51714087842877440 https://twitter.com/bramcohen/status/51714087842877440 that I think this one will happen.) And all the jobs that we've automated 'til now have been automated in this "meaningful" sense. Neither of these objections hold much water IMO, which is why I take the intelligence explosion scenario described by Oxford philosopher Nick Bostrom seriously: http://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0199678111/ http://www.amazon.com/Superintelligence-Dangers-Strategies-N...
- studentrob 11y ago
  > (I'm convinced that programmers will take the AI-automating-all-the-jobs idea more seriously once it's their own jobs that are on the line) I assure you we do not. We would all love to be the one to create such a program. But don't worry it isn't happening anytime soon in the form of singularity. Just like other people in other jobs, we sometimes come up with more efficient ways of doing things than our bosses thought possible, and then we have some extra free time to do what we like.
  - astrofinch 11y ago
    "But don't worry it isn't happening anytime soon in the form of singularity." Probably not soon. Worth noting that the Go victory happened a decade or two before it was predicted to though. (To clarify, I was not trying to establish that any intelligence explosion is on the immediate horizon. Rather, I was trying to establish that it's a pretty sensible concept when you think about it, and has a solid chance of happening at some point.) >Just like other people in other jobs, we sometimes come up with more efficient ways of doing things than our bosses thought possible, and then we have some extra free time to do what we like. Yes, I'm a programmer (UC Berkeley computer science)... I know. But don't listen to me. Listen to Stuart Russell: https://www.cs.berkeley.edu/~russell/research/future/ https://www.cs.berkeley.edu/~russell/research/future/
    - studentrob 11y ago
      > (To clarify, I was not trying to establish that any intelligence explosion is on the immediate horizon. Rather, I was trying to establish that it's a pretty sensible concept when you think about it, and has a solid chance of happening at some point.) Well first you were saying programmers may hesitate to create an AI singularity because it may cost them their job. I said we would love to but probably won't anytime soon. Now you're saying that it's likely that it will happen some day. I'm not sure these points follow a single train of thought.
      - astrofinch 11y ago

What's an intelligence explosion? Intelligence dissemination or deletion? EDIT: Nevermind, just looked it up and found this was a real thing. I thought you were joking.

astrofinch 11y ago

It sounds less weird if you think: "AI has automated some jobs, and eventually it may automate away most of them, in the same way (pending AlphaGo victory) it's now automated winning at most board games... So what happens if the job of programmer gets automated away?" (I'm convinced that programmers will take the AI-automating-all-the-jobs idea more seriously once it's their own jobs that are on the line) There are a few ways I can think of to object to this line of reasoning: 1) You could argue that programming will be a job that never gets automated away. But this seems unlikely--previous intellectual tasks (Chess, Jeopardy, Go) were thought to be "essentially human", and once they were solved by computers, got redefined as not "essentially human" (and therefore not AI). My opinion: In the same way we originally thought tool use made humans unique, then realized that other animals use tools too, we'll eventually learn that there's no fundamental uniqueness to human cognition. It'll be matter & algorithms all the way down. Of course, the algorithms could be really darn complicated. But the fact the Deepmind team won at both Go and Atari using the same approach suggests the existence of important general-purpose algorithms that are within the reach of human software engineers. 2) You could argue that programming will be automated away but in a sense that isn't meaningful (e.g. you need an expensive server farm to replace a relatively cheap human programmer). This is certainly possible. But in the same way calculators do arithmetic much faster & better than humans do, there's the possibility that automated computer programmers will program much faster & better than humans. (Honestly humans suck so hard at programming https://twitter.com/bramcohen/status/51714087842877440 https://twitter.com/bramcohen/status/51714087842877440 that I think this one will happen.) And all the jobs that we've automated 'til now have been automated in this "meaningful" sense. Neither of these objections hold much water IMO, which is why I take the intelligence explosion scenario described by Oxford philosopher Nick Bostrom seriously: http://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0199678111/ http://www.amazon.com/Superintelligence-Dangers-Strategies-N...

studentrob 11y ago
> (I'm convinced that programmers will take the AI-automating-all-the-jobs idea more seriously once it's their own jobs that are on the line) I assure you we do not. We would all love to be the one to create such a program. But don't worry it isn't happening anytime soon in the form of singularity. Just like other people in other jobs, we sometimes come up with more efficient ways of doing things than our bosses thought possible, and then we have some extra free time to do what we like.
- astrofinch 11y ago
  "But don't worry it isn't happening anytime soon in the form of singularity." Probably not soon. Worth noting that the Go victory happened a decade or two before it was predicted to though. (To clarify, I was not trying to establish that any intelligence explosion is on the immediate horizon. Rather, I was trying to establish that it's a pretty sensible concept when you think about it, and has a solid chance of happening at some point.) >Just like other people in other jobs, we sometimes come up with more efficient ways of doing things than our bosses thought possible, and then we have some extra free time to do what we like. Yes, I'm a programmer (UC Berkeley computer science)... I know. But don't listen to me. Listen to Stuart Russell: https://www.cs.berkeley.edu/~russell/research/future/ https://www.cs.berkeley.edu/~russell/research/future/
  - studentrob 11y ago
    > (To clarify, I was not trying to establish that any intelligence explosion is on the immediate horizon. Rather, I was trying to establish that it's a pretty sensible concept when you think about it, and has a solid chance of happening at some point.) Well first you were saying programmers may hesitate to create an AI singularity because it may cost them their job. I said we would love to but probably won't anytime soon. Now you're saying that it's likely that it will happen some day. I'm not sure these points follow a single train of thought.
    - astrofinch 11y ago