4 ms·

What we've to understand is that a machine that has intelligence doesn't necessarily have emotion, as you so rightly pointed out with your friendly comment. How

by chunky1994 15y ago

What we've to understand is that a machine that has intelligence doesn't necessarily have emotion, as you so rightly pointed out with your friendly comment. However this doesn't mean it will take over the world, it just means it will do what it was meant to, and become better at what it was meant to do that's intelligence. It may even start to work on things it hasn't been told to work on, but that doesn't mean it's going to abolish governance and force humans into submission.

loup-vaillant 15y ago
By "taking over the world", I meant establishing a Bostromian Singleton[1]. I didn't want to predict anything more, and certainly not anything resembling a human dictatorship. I'm also not sure you could really limit your AI's reach. If for instance you give it a limited goal, like solving the Riemann Hypothesis, it could transform the solar system into a giant computer, killing off humanity along the way, just to solve the freaking mathematical problem. Assuming you could limit the reach of your AI, you'd probably need to have a working Friendliness Theory to do that. At that point, I suggest you save the world instead. http://en.wikipedia.org/wiki/Singleton_%28global_governance%29 http://en.wikipedia.org/wiki/Singleton_%28global_governance%...
- chunky1994 15y ago
  I'm not sure that having a singleton is such a bad idea. However, in your second scenario I highly doubt that an AI with transhuman reasoning would be unable to reason that (bear with me here) the reason it's solving said problem only exists because humans exist, and if they were killed off, it's reason for solving the problem would be redundant. This doesn't mean that we can't limit the AI though, give it sufficient hardware and let it solve the problem (and don't converse with it or contact it in any matter unless it has solved the problem or requires more hardware), although this is like slavery and I don't like the concept, that would be one way of limiting it.
  - loup-vaillant 15y ago
    Oh I thought about your reasoning. It just need to keep the one that asked the question, then. Killing everyone else won't prevent it to tell the answer to the researcher. Even with limited hardware, the AI still can have a non-trivial influence over the world, provided it can self-improve. And if you limit the hardware to present-day computers, and you ensure the format of the answer is highly constrained, you may be safe. The AI will be much more limited, though.

loup-vaillant 15y ago

By "taking over the world", I meant establishing a Bostromian Singleton[1]. I didn't want to predict anything more, and certainly not anything resembling a human dictatorship. I'm also not sure you could really limit your AI's reach. If for instance you give it a limited goal, like solving the Riemann Hypothesis, it could transform the solar system into a giant computer, killing off humanity along the way, just to solve the freaking mathematical problem. Assuming you could limit the reach of your AI, you'd probably need to have a working Friendliness Theory to do that. At that point, I suggest you save the world instead. http://en.wikipedia.org/wiki/Singleton_%28global_governance%29 http://en.wikipedia.org/wiki/Singleton_%28global_governance%...

chunky1994 15y ago
I'm not sure that having a singleton is such a bad idea. However, in your second scenario I highly doubt that an AI with transhuman reasoning would be unable to reason that (bear with me here) the reason it's solving said problem only exists because humans exist, and if they were killed off, it's reason for solving the problem would be redundant. This doesn't mean that we can't limit the AI though, give it sufficient hardware and let it solve the problem (and don't converse with it or contact it in any matter unless it has solved the problem or requires more hardware), although this is like slavery and I don't like the concept, that would be one way of limiting it.
- loup-vaillant 15y ago
  Oh I thought about your reasoning. It just need to keep the one that asked the question, then. Killing everyone else won't prevent it to tell the answer to the researcher. Even with limited hardware, the AI still can have a non-trivial influence over the world, provided it can self-improve. And if you limit the hardware to present-day computers, and you ensure the format of the answer is highly constrained, you may be safe. The AI will be much more limited, though.