4 ms·

Devil’s advocate: success is a step function. AI isn’t going to take over the world until it is able to, and then it very rapidly does.

goatlover 8y ago
Devil's devil's advocate. The world is too complicated to be reduced to a function. AI isn't going to take over the world anymore than anyone else has (governments and organizations included).
- gowld 8y ago
  "the world" is vague. Governments certainly have taken over various worlds.
- canjobear 8y ago
  > The world is too complicated to be reduced to a function. I can only interpret this as a mystical claim...
  - cornholio 8y ago
    The world is too complicated to be reduced to a function that does not exhibit general intelligence. At minimum, able to formulate rational strategies in arbitrary games against other rational adversaries.
  - goatlover 8y ago
    The mystical claim is that the world is like a board game where we can extrapolate from the success of AlphaGo/Zero to AGI taking over the world. That's just pure speculation because nobody has a clue what such an intelligence would look like. I can imagine using the mind infinity stone to take over the world, but that doesn't make it realistic.
    - red75prime 8y ago
      We have a working model of general intelligence. It runs on 20 watts, takes 20 years to became somewhat usable, and is known to cause local disasters.
- fossuser 8y ago
  Take the human general intelligence learning architecture and scale it up to a billion operations a second. The trick with AGI is that it can learn faster, and humans (even with our limitations) did take over the world.
  - goatlover 8y ago
    So one AGI is like a billion human beings, so a little bit less than China or India. And like countries, there will be other AGIs. An AGI isn't born in a vacuum, it comes into existence in a world advanced enough for such a tech to be developed. As such, it's reasonable to suppose there will limitations for AGIs when it decides that world conquest is a worthy goal.
    - fossuser 8y ago
      It's not like one billion human beings. Humans are limited by a lot of things like motivation and communication which is likely distinct from the general problem solving architecture itself. Humans also have goals and an evolved morality that guides their thinking. Humans share a similar type of intelligence and understanding of goals, culture, right and wrong. You don't get this for free with AGI. The point of the human example is that there already exists a general problem solving architecture you can scale up (it's not impossible and there are brains everywhere in nature). If you increase operations per second many magnitudes you can have all human learning over thousands of years compressed into a couple of hours. You see this loosely in narrow spaces now with things like Alpha Go Zero. There may not be other AGIs if there's an intelligence explosion and the first one improves itself very quickly. If its utility function is not aligned with human goals then it could end up turning all matter into paperclips - not because it's evil, but because it's hard to set goals tied to human morality (when humans don't even agree in every case) and it happens to be configured such that making paperclips maxes out its reward function by accident. It may be possible to make the unsafe version before making the safe one, since it seems harder to make an AGI that's aligned with human goals than just an AGI in general.
- garmaine 8y ago
  Organizations of humans (governments) seem to have conquered the world pretty well, outside of a scant few lawless areas. Even then there's usually some local warlord maintaining order, just not a nation state. I'm not sure what you're trying to say?
laichzeit0 8y ago
What seems more likely at this point is that: machine translation will get so good that it will be better at translating between languages than any human can; image classifiers will become so good that they will be better at classifying images than any human can; self-driving cars will become so good that they will be better at driving than any human can; etc. etc. I think the mismatch comes when people looking from the outside who have no real understanding of how the internals work see this progress in a very narrow domain and extrapolate that idea into thinking that something more general is happening. When you understand the trick, it stops being magic.
- klipt 8y ago
  If a machine can drive a car, how much harder is it to drive a tank? If a machine can generate strategies to win simulated wars, how much harder is it to generate strategies to win real wars? If machines can out compete people on 50% of jobs, how much harder is it to out compete people on 100% of jobs?
  - YeGoblynQueenne 8y ago
    Well, even for humans driving a car doesn't mean you can drive a tank; being good at Rome: Total War or Starcraft doesn't mean you can win a real war; and we're a long way away from "machines" being able to outcompete 50% of humans at their jobs.
- red75prime 8y ago
  Some day we will understand how we do our tricks though.
  - garmaine 8y ago
    Or alternatively, how to accomplish the same thing in an easier way -- we don't fly in commercial aircraft by flapping feathered wings, after all. And then magic will be dissolved and we might find that general AI isn't really all that much harder than narrow AI.
- 8y ago
YeGoblynQueenne 8y ago
>> AI isn’t going to take over the world until it is able to, and then it very rapidly does. The question is, of course- how? How will AI become able to take over the world? This is an important question to ask because predictions of AI rapidly becoming capable of taking over the world tend to flare up in times of progress in AI performance. For example, the above article is basically based on the preimse that the latest developments, i.e. deep learning, are sufficient to give some nation-state technological supremacy by leading directly to AGI. However, there is no clear path from deep learning, however powerful it may be as a set of techniques to train classifiers, to AGI, which (well, probably) entails much more than powerful classification. For instance, deep learning can't really do inference. It seems reasonable to assume that AGI will need the ability to reason about the objects it can recognise and the relations between them. Deep learning can't do causal analysis- like Judea Pearl has been screaming from the rooftops lately. It seems reasonable again to assume that AGI will need to model causal relations. Deep learning can't do semantics. AGI will need to have some handle on semantics. And so on and so forth. In general, anyone that claims that X current development in AI risks inadvertently producing AGI, must explain how this is supposed to happen- not just wave their hands about and claim that, if you throw enough compute and enough data at the problem, magic will happen and AGI will turn us all into paperclips.

goatlover 8y ago

Devil's devil's advocate. The world is too complicated to be reduced to a function. AI isn't going to take over the world anymore than anyone else has (governments and organizations included).

gowld 8y ago
"the world" is vague. Governments certainly have taken over various worlds.
canjobear 8y ago
> The world is too complicated to be reduced to a function. I can only interpret this as a mystical claim...
- cornholio 8y ago
  The world is too complicated to be reduced to a function that does not exhibit general intelligence. At minimum, able to formulate rational strategies in arbitrary games against other rational adversaries.
- goatlover 8y ago
  The mystical claim is that the world is like a board game where we can extrapolate from the success of AlphaGo/Zero to AGI taking over the world. That's just pure speculation because nobody has a clue what such an intelligence would look like. I can imagine using the mind infinity stone to take over the world, but that doesn't make it realistic.
  - red75prime 8y ago
    We have a working model of general intelligence. It runs on 20 watts, takes 20 years to became somewhat usable, and is known to cause local disasters.
fossuser 8y ago
Take the human general intelligence learning architecture and scale it up to a billion operations a second. The trick with AGI is that it can learn faster, and humans (even with our limitations) did take over the world.
- goatlover 8y ago
  So one AGI is like a billion human beings, so a little bit less than China or India. And like countries, there will be other AGIs. An AGI isn't born in a vacuum, it comes into existence in a world advanced enough for such a tech to be developed. As such, it's reasonable to suppose there will limitations for AGIs when it decides that world conquest is a worthy goal.
  - fossuser 8y ago
    It's not like one billion human beings. Humans are limited by a lot of things like motivation and communication which is likely distinct from the general problem solving architecture itself. Humans also have goals and an evolved morality that guides their thinking. Humans share a similar type of intelligence and understanding of goals, culture, right and wrong. You don't get this for free with AGI. The point of the human example is that there already exists a general problem solving architecture you can scale up (it's not impossible and there are brains everywhere in nature). If you increase operations per second many magnitudes you can have all human learning over thousands of years compressed into a couple of hours. You see this loosely in narrow spaces now with things like Alpha Go Zero. There may not be other AGIs if there's an intelligence explosion and the first one improves itself very quickly. If its utility function is not aligned with human goals then it could end up turning all matter into paperclips - not because it's evil, but because it's hard to set goals tied to human morality (when humans don't even agree in every case) and it happens to be configured such that making paperclips maxes out its reward function by accident. It may be possible to make the unsafe version before making the safe one, since it seems harder to make an AGI that's aligned with human goals than just an AGI in general.
garmaine 8y ago
Organizations of humans (governments) seem to have conquered the world pretty well, outside of a scant few lawless areas. Even then there's usually some local warlord maintaining order, just not a nation state. I'm not sure what you're trying to say?

laichzeit0 8y ago

What seems more likely at this point is that: machine translation will get so good that it will be better at translating between languages than any human can; image classifiers will become so good that they will be better at classifying images than any human can; self-driving cars will become so good that they will be better at driving than any human can; etc. etc. I think the mismatch comes when people looking from the outside who have no real understanding of how the internals work see this progress in a very narrow domain and extrapolate that idea into thinking that something more general is happening. When you understand the trick, it stops being magic.

klipt 8y ago
If a machine can drive a car, how much harder is it to drive a tank? If a machine can generate strategies to win simulated wars, how much harder is it to generate strategies to win real wars? If machines can out compete people on 50% of jobs, how much harder is it to out compete people on 100% of jobs?
- YeGoblynQueenne 8y ago
  Well, even for humans driving a car doesn't mean you can drive a tank; being good at Rome: Total War or Starcraft doesn't mean you can win a real war; and we're a long way away from "machines" being able to outcompete 50% of humans at their jobs.
red75prime 8y ago
Some day we will understand how we do our tricks though.
- garmaine 8y ago
  Or alternatively, how to accomplish the same thing in an easier way -- we don't fly in commercial aircraft by flapping feathered wings, after all. And then magic will be dissolved and we might find that general AI isn't really all that much harder than narrow AI.
8y ago

YeGoblynQueenne 8y ago

>> AI isn’t going to take over the world until it is able to, and then it very rapidly does. The question is, of course- how? How will AI become able to take over the world? This is an important question to ask because predictions of AI rapidly becoming capable of taking over the world tend to flare up in times of progress in AI performance. For example, the above article is basically based on the preimse that the latest developments, i.e. deep learning, are sufficient to give some nation-state technological supremacy by leading directly to AGI. However, there is no clear path from deep learning, however powerful it may be as a set of techniques to train classifiers, to AGI, which (well, probably) entails much more than powerful classification. For instance, deep learning can't really do inference. It seems reasonable to assume that AGI will need the ability to reason about the objects it can recognise and the relations between them. Deep learning can't do causal analysis- like Judea Pearl has been screaming from the rooftops lately. It seems reasonable again to assume that AGI will need to model causal relations. Deep learning can't do semantics. AGI will need to have some handle on semantics. And so on and so forth. In general, anyone that claims that X current development in AI risks inadvertently producing AGI, must explain how this is supposed to happen- not just wave their hands about and claim that, if you throw enough compute and enough data at the problem, magic will happen and AGI will turn us all into paperclips.