6 ms·

Bill Joy: Why the future doesn't need us (2004)

42 points by zengr 16y ago

zengr 16y ago
Looking for a summary? Read this wiki entry: http://en.wikipedia.org/wiki/Why_the_future_doesn%27t_need_us http://en.wikipedia.org/wiki/Why_the_future_doesn%27t_need_u...
- FrojoS 16y ago
  Thanks! Very interesting. From the WP article: "Martin Ford author of The Lights in the Tunnel: Automation, Accelerating Technology and the Economy of the Future [6] makes the case that the risk posed by accelerating technology may be primarily economic in nature. Ford argues that before technology reaches the point where it represents a physical existential threat, it will become possible to automate nearly all routine and repetitive jobs in the economy." I find this, for short or medium time scales, quite likely, too. There will always be a demand for high skilled humans. Maybe even after the machines are the new bosses. But if we don't make groundbreaking progress in human learning techniques, most people will have trouble to learn these skills fast enough. Then he writes further: "In the absence of a major reform to the capitalist system, this could result in massive unemployment, plunging consumer spending and confidence, and an economic crisis potentially even more severe than the Great Depression. If such a crisis were to occur, subsequent technological progress would dramatically slow because there would be insufficient incentive to invest in innovation." This sounds somewhat plausible but I don't believe it. The trend goes towards highly profitable mega corporations. Governments, and with it most people, become less powerful in economic terms. I think we can already see this effect very well. So there won't necessary be a recession. As long as the rich find ways to spend their money - like flying to Mars.
bermanoid 16y ago
Here's the problem with the "Let's just agree not to do this research!" plan that everyone seems to suggest when they start thinking about existential risks: when we're sitting around in 2030 with a million times more computing power at our fingertips than we have today, constructing a workable AI just isn't going to be that difficult of an engineering problem. We already know the equations that we'd need to use do general intelligence, it's just that they're not computable with finite computer power, so we'd have to do some approximations, and at present it's not realistic because the approximation schemes we know of would work too slowly. Pump up our computer power a million times and these schemes start to become a lot more realistic, especially with some halfway decent pruning heuristics. It's bad enough that (IMO) by 2040 or so, any reasonably smart asshole in his basement could probably do it on his laptop with access to only the reference materials available today; I have no idea how you avoid that risk by making some political agreement. Hell, ban the research altogether on pain of death, and there's still going to be some terrorist team working on it somewhere (and that's even if all the governments actually stop work on it, which they won't). The only positive way out of this is to go to great pains to figure out how to design safe (friendly) AI, and to do so while it's still too difficult for random-dude-with-a-botnet to achieve (and preferably we should do it before the governments of the world see it as feasible enough to throw military research dollars at). We need to tackle the problem while it's still a difficult software problem, not a brute-force one that can be cracked by better hardware.
- anonymousguru 16y ago
  "when we're sitting around in 2030 with a million times more computing power at our fingertips than we have today, constructing a workable AI just isn't going to be that difficult of an engineering problem." Sorry, but bruteforcing "creativity" is a p==np problem. If you have a proof of p==np then please submit it right away and collect your turing award. "We already know the equations that we'd need to use do general intelligence, it's just that they're not computable with finite computer power, so we'd have to do some approximations, and at present it's not realistic because the approximation schemes we know of would work too slowly." Unless there have been some MAJOR discovery within the past couple of days, this sentence is completely wrong. We don't have equations to build an AGI because if we did, we would already have a working AGI. If we had these so called equations, we would have a more fundamental understanding of humans, how they work, and how they "learn". I will now stop thinking about this ridiculous sentence before I get an aneurysm. P.S. I would love it if you could prove me wrong with links to any academic papers.
  - elehack 16y ago
    P?=NP doesn't have anything to do with possibility - it has to do with feasibility. The point is, with sufficient hardware, we might be able to just do the NP problem. Also, brute-forcing creativity is not P==NP. Brute-forcing is NP. If P=NP, then there's a way to do it without brute-forcing.
- alextp 16y ago
  > We already know the equations that we'd need to use do general intelligence, it's just that they're not computable with finite computer power, so we'd have to do some approximations, and at present it's not realistic because the approximation schemes we know of would work too slowly. If you do know the equations, please tell me, as all models that I know of have fundamental failures that prevent them from generalizing to arbitrary problems even without considering computation.
- arethuza 16y ago
  "We already know the equations that we'd need to use do general intelligence" Not only am I pretty sure we don't know how to build a general intelligence I'm pretty sure that nobody really knows what kind of approach would be most likely to succeed. Having said that, I would love to be proved wrong on this one - so as you specifically say that the necessary techniques have already been published perhaps you could give the relevant references?
tybris 16y ago
Humanity may be doomed if it keeps innovating, but it's most certainly doomed if it stops.
- FrojoS 16y ago
  I second this!
jcfrei 16y ago
I think a relative simple solution to this is written in the first paragraphs. We will more or less 'merge' our minds with computers, just to a further extent than we already do now. Nowadays a computer is merely a tool, which helps us keeping in touch with relatives, visualizing ideas, calculating stuff. but this bond will probably become much more intense in the future, where whole subroutines of our thinking will rely on artificial machines. This might only seem like a threat considering our 21st century morality - but I think this will become widely accept in the next century.
- gmaslov 16y ago
  The future must be now. At least one of my subroutines of thinking already relies on an artificial machine. I call it my Google neuron; it's wired up directly to everything I don't know off the top of my head and fires whenever I feel unsure about something. Well, the latency is still a bit high, but I'm sure someone is working on that problem. ;)
atlei 16y ago
Even the most trivial computer programs have lots of bugs (with VERY few exceptions [1]), and we're worrying about creating a super-brain that is actually smarter than we are ourselves ? And let's not forget the debugging, which is TWICE as hard as the coding ;-) We may be able to simulate the hardware of the brain (using "biologic hardware"), but programming the AI software is probably greatly under-estimated.... [1] Some of the NASA software is probably as close to bug-free as we get, and check the required amount of planning, documentation and testing compared to the amount of actual code produced - http://www.fastcompany.com/magazine/06/writestuff.html http://www.fastcompany.com/magazine/06/writestuff.html
- robertk 16y ago
  The human brain has much more bugs than any current piece of software. The actual program will be small. The brain is clearly a learning agent in a task environment. The sensors and actuators are implementable, the only real question is what algorithm should process the input stimuli.
RandyHelzerman 16y ago
I remember we talked about this on the second date I had with my wife. I'll tell you about that, but first let me tell you about our first date: I asked her what she at work. She said she was in P.R. and had just got done pitching stories to appear the following week. She said she told the Wall Street Journal to say X on monday, Newsweek to say Y on wednesday, and Investor's Business daily to say Z on Friday. Sure enough, the following week, the Wall Street Journal said X on Monday, Newsweek said Y on Wednesday, and on Friday, Investor's Business daily said Z. I was beyond words impressed, and had to see this woman again. At our second date, I mentioned how impressed I was, and she told me how most of the news we read is actually manufactured by people like her at large corporations. She then told me that Sun was doomed. She already knew this because of the whack-tastic article which Bill Joy had published in Wired Magazine. She said that any company which would let their executives say such loonie things was already dead, the actual death was only a matter of time. I had to marry this woman, and we've been happily married ever since.
FrojoS 16y ago
ATTENTION: Book spoiler below! SF author Vernor Vinge who introduced the term "singularity" tried to come up with an idea to prevent this and other "out of the kids basements" lethal threads to humanity, in his latest book Rainbow End. The "solution" in this book though, is to put all of humanity under mind control.

42 points by zengr 16y ago

zengr 16y ago

Looking for a summary? Read this wiki entry: http://en.wikipedia.org/wiki/Why_the_future_doesn%27t_need_us http://en.wikipedia.org/wiki/Why_the_future_doesn%27t_need_u...

FrojoS 16y ago
Thanks! Very interesting. From the WP article: "Martin Ford author of The Lights in the Tunnel: Automation, Accelerating Technology and the Economy of the Future [6] makes the case that the risk posed by accelerating technology may be primarily economic in nature. Ford argues that before technology reaches the point where it represents a physical existential threat, it will become possible to automate nearly all routine and repetitive jobs in the economy." I find this, for short or medium time scales, quite likely, too. There will always be a demand for high skilled humans. Maybe even after the machines are the new bosses. But if we don't make groundbreaking progress in human learning techniques, most people will have trouble to learn these skills fast enough. Then he writes further: "In the absence of a major reform to the capitalist system, this could result in massive unemployment, plunging consumer spending and confidence, and an economic crisis potentially even more severe than the Great Depression. If such a crisis were to occur, subsequent technological progress would dramatically slow because there would be insufficient incentive to invest in innovation." This sounds somewhat plausible but I don't believe it. The trend goes towards highly profitable mega corporations. Governments, and with it most people, become less powerful in economic terms. I think we can already see this effect very well. So there won't necessary be a recession. As long as the rich find ways to spend their money - like flying to Mars.

bermanoid 16y ago

Here's the problem with the "Let's just agree not to do this research!" plan that everyone seems to suggest when they start thinking about existential risks: when we're sitting around in 2030 with a million times more computing power at our fingertips than we have today, constructing a workable AI just isn't going to be that difficult of an engineering problem. We already know the equations that we'd need to use do general intelligence, it's just that they're not computable with finite computer power, so we'd have to do some approximations, and at present it's not realistic because the approximation schemes we know of would work too slowly. Pump up our computer power a million times and these schemes start to become a lot more realistic, especially with some halfway decent pruning heuristics. It's bad enough that (IMO) by 2040 or so, any reasonably smart asshole in his basement could probably do it on his laptop with access to only the reference materials available today; I have no idea how you avoid that risk by making some political agreement. Hell, ban the research altogether on pain of death, and there's still going to be some terrorist team working on it somewhere (and that's even if all the governments actually stop work on it, which they won't). The only positive way out of this is to go to great pains to figure out how to design safe (friendly) AI, and to do so while it's still too difficult for random-dude-with-a-botnet to achieve (and preferably we should do it before the governments of the world see it as feasible enough to throw military research dollars at). We need to tackle the problem while it's still a difficult software problem, not a brute-force one that can be cracked by better hardware.

anonymousguru 16y ago
"when we're sitting around in 2030 with a million times more computing power at our fingertips than we have today, constructing a workable AI just isn't going to be that difficult of an engineering problem." Sorry, but bruteforcing "creativity" is a p==np problem. If you have a proof of p==np then please submit it right away and collect your turing award. "We already know the equations that we'd need to use do general intelligence, it's just that they're not computable with finite computer power, so we'd have to do some approximations, and at present it's not realistic because the approximation schemes we know of would work too slowly." Unless there have been some MAJOR discovery within the past couple of days, this sentence is completely wrong. We don't have equations to build an AGI because if we did, we would already have a working AGI. If we had these so called equations, we would have a more fundamental understanding of humans, how they work, and how they "learn". I will now stop thinking about this ridiculous sentence before I get an aneurysm. P.S. I would love it if you could prove me wrong with links to any academic papers.
- elehack 16y ago
  P?=NP doesn't have anything to do with possibility - it has to do with feasibility. The point is, with sufficient hardware, we might be able to just do the NP problem. Also, brute-forcing creativity is not P==NP. Brute-forcing is NP. If P=NP, then there's a way to do it without brute-forcing.
alextp 16y ago
> We already know the equations that we'd need to use do general intelligence, it's just that they're not computable with finite computer power, so we'd have to do some approximations, and at present it's not realistic because the approximation schemes we know of would work too slowly. If you do know the equations, please tell me, as all models that I know of have fundamental failures that prevent them from generalizing to arbitrary problems even without considering computation.
arethuza 16y ago
"We already know the equations that we'd need to use do general intelligence" Not only am I pretty sure we don't know how to build a general intelligence I'm pretty sure that nobody really knows what kind of approach would be most likely to succeed. Having said that, I would love to be proved wrong on this one - so as you specifically say that the necessary techniques have already been published perhaps you could give the relevant references?

tybris 16y ago

Humanity may be doomed if it keeps innovating, but it's most certainly doomed if it stops.

FrojoS 16y ago
I second this!

jcfrei 16y ago

I think a relative simple solution to this is written in the first paragraphs. We will more or less 'merge' our minds with computers, just to a further extent than we already do now. Nowadays a computer is merely a tool, which helps us keeping in touch with relatives, visualizing ideas, calculating stuff. but this bond will probably become much more intense in the future, where whole subroutines of our thinking will rely on artificial machines. This might only seem like a threat considering our 21st century morality - but I think this will become widely accept in the next century.

gmaslov 16y ago
The future must be now. At least one of my subroutines of thinking already relies on an artificial machine. I call it my Google neuron; it's wired up directly to everything I don't know off the top of my head and fires whenever I feel unsure about something. Well, the latency is still a bit high, but I'm sure someone is working on that problem. ;)

atlei 16y ago

Even the most trivial computer programs have lots of bugs (with VERY few exceptions [1]), and we're worrying about creating a super-brain that is actually smarter than we are ourselves ? And let's not forget the debugging, which is TWICE as hard as the coding ;-) We may be able to simulate the hardware of the brain (using "biologic hardware"), but programming the AI software is probably greatly under-estimated.... [1] Some of the NASA software is probably as close to bug-free as we get, and check the required amount of planning, documentation and testing compared to the amount of actual code produced - http://www.fastcompany.com/magazine/06/writestuff.html http://www.fastcompany.com/magazine/06/writestuff.html

robertk 16y ago
The human brain has much more bugs than any current piece of software. The actual program will be small. The brain is clearly a learning agent in a task environment. The sensors and actuators are implementable, the only real question is what algorithm should process the input stimuli.

RandyHelzerman 16y ago

I remember we talked about this on the second date I had with my wife. I'll tell you about that, but first let me tell you about our first date: I asked her what she at work. She said she was in P.R. and had just got done pitching stories to appear the following week. She said she told the Wall Street Journal to say X on monday, Newsweek to say Y on wednesday, and Investor's Business daily to say Z on Friday. Sure enough, the following week, the Wall Street Journal said X on Monday, Newsweek said Y on Wednesday, and on Friday, Investor's Business daily said Z. I was beyond words impressed, and had to see this woman again. At our second date, I mentioned how impressed I was, and she told me how most of the news we read is actually manufactured by people like her at large corporations. She then told me that Sun was doomed. She already knew this because of the whack-tastic article which Bill Joy had published in Wired Magazine. She said that any company which would let their executives say such loonie things was already dead, the actual death was only a matter of time. I had to marry this woman, and we've been happily married ever since.

FrojoS 16y ago

ATTENTION: Book spoiler below! SF author Vernor Vinge who introduced the term "singularity" tried to come up with an idea to prevent this and other "out of the kids basements" lethal threads to humanity, in his latest book Rainbow End. The "solution" in this book though, is to put all of humanity under mind control.