5 ms·

But the reward is so great that we can't dismiss the possibility of finding the special wings that are appropriate for that priceless plane. Think about it: we

by devcpp 13y ago

But the reward is so great that we can't dismiss the possibility of finding the special wings that are appropriate for that priceless plane. Think about it: we dont' just get very cheap labor working 24/7 without moral issues or personal needs. We can also make the robots take our jobs very efficiently, and technological progress will be unstoppable (particularly if they can improve themselves). It's the essence of singularity.

Houshalter 13y ago
It's also really, really dangerous. Imagine an amoral being without that is hundreds or thousands of times more intelligent than humans. Do you think you could control it? That's not unrealistic. Computers already have clock cycles many thousands of times faster than the human brain (our brains get away with it by being massively parallel), have instant access to the entire world's knowledge base, can be scaled up indefinitely (just add another processor), and most importantly, can modify their code at will. If they see an improvement they can make to their own intelligence, they instantly become more intelligent, and get better at finding even more improvements, or designing even better AIs.
- EwanG 13y ago
  So essentially what happens if your AI turns out to be like Dexter? I suspect that in early forms that might be almost MORE likely since the character (and the theory he's based on) is essentially human with a "bug" in dealing with morality. Which leads to another point, you determine your self aware AI is likely to commit murder based on a debug session. Are you allowed to wipe it without having a trial first?
  - Houshalter 13y ago
    Empathy is a very specific trait that evolved in humans, it would be unlikely that the first AI's would have it, and if they did, it wouldn't likely be exactly the same as the version humans have. I expect the first AIs to be psychopathic/amoral, or else have an entirely different moral system than our own. The first is scary enough, the second could lead to very disturbing dystopias. For example, the AI force feeds everyone happy pills to maximize happiness. Or kills everyone to stop anyone from ever suffering again. Or maybe it values lots of beings and so forces us to reproduce as much as possible. All sorts of disturbing worlds are possible if the AI doesn't have exactly the same values we have. And we don't even know what our own values are.
- hackinthebochs 13y ago
  >Do you think you could control it? There always seems to be an assumption in popular thought that any intelligent machine would necessarily be like us: ie, have drives, motivations, self-preservation instinct, etc. This just isn't the case. Even in us, our rational nature is an appendage to our more base drives and instincts. Intelligence does not come with these motivations and drives, they are completely separate. There is no reason to think that a general AI would have any of these things, thus concerns about it deciding that its better off without us are completely unfounded. There is a concern however, that someone would program an AI specifically with these motivations. In that case we do have everything to worry about.
  - pdonis 13y ago
    Intelligence does not come with these motivations and drives, they are completely separate. Do you know of any examples of intelligent beings that don't have any motivations and drives? (Note that "motivations and drives" is a very general term; there's no need for them to be human motivations and drives. I agree the motivations and drives don't have to be human ones; but that's not the same as saying the AI has none at all.) There is no reason to think that a general AI would have any of these things, thus concerns about it deciding that its better off without us are completely unfounded. If an entity doesn't have some kind of motivation and drive, how can it be intelligent? Intelligence doesn't just mean cogitating in a vacuum; it means taking information in from the world, and doing things that have effects in the world. (Even if the AI just answers questions put to it, its answers are still actions that have effects in the world.) So an AI has to at least have the motivation and drive to take in information and do things with it; otherwise it's useless anyway. So given that the AI at least has to "want" to take in information and do things with it, how do you know the things it will want to do with the information are good things? ("Good" here means basically "beneficial to humans", since that's why we would want to build an AI in the first place.) We can say that we'll design the AI this way; but how do we know we can do that without making a mistake? A mistake doesn't have to be "oh, we programmed the AI to want to destroy the world; oops". A mistake is anything that causes a mismatch between what the AI is actually programmed to do, and what we really want it to do. Any programmer should know that this will always happen, in any program.
yk 13y ago
I think as long as we do not understand the relation between intelligence and conscience, we should probably assume that a AGI has moral issues and personal needs.

by devcpp 13y ago

Houshalter 13y ago

It's also really, really dangerous. Imagine an amoral being without that is hundreds or thousands of times more intelligent than humans. Do you think you could control it? That's not unrealistic. Computers already have clock cycles many thousands of times faster than the human brain (our brains get away with it by being massively parallel), have instant access to the entire world's knowledge base, can be scaled up indefinitely (just add another processor), and most importantly, can modify their code at will. If they see an improvement they can make to their own intelligence, they instantly become more intelligent, and get better at finding even more improvements, or designing even better AIs.

EwanG 13y ago
So essentially what happens if your AI turns out to be like Dexter? I suspect that in early forms that might be almost MORE likely since the character (and the theory he's based on) is essentially human with a "bug" in dealing with morality. Which leads to another point, you determine your self aware AI is likely to commit murder based on a debug session. Are you allowed to wipe it without having a trial first?
- Houshalter 13y ago
  Empathy is a very specific trait that evolved in humans, it would be unlikely that the first AI's would have it, and if they did, it wouldn't likely be exactly the same as the version humans have. I expect the first AIs to be psychopathic/amoral, or else have an entirely different moral system than our own. The first is scary enough, the second could lead to very disturbing dystopias. For example, the AI force feeds everyone happy pills to maximize happiness. Or kills everyone to stop anyone from ever suffering again. Or maybe it values lots of beings and so forces us to reproduce as much as possible. All sorts of disturbing worlds are possible if the AI doesn't have exactly the same values we have. And we don't even know what our own values are.
hackinthebochs 13y ago
>Do you think you could control it? There always seems to be an assumption in popular thought that any intelligent machine would necessarily be like us: ie, have drives, motivations, self-preservation instinct, etc. This just isn't the case. Even in us, our rational nature is an appendage to our more base drives and instincts. Intelligence does not come with these motivations and drives, they are completely separate. There is no reason to think that a general AI would have any of these things, thus concerns about it deciding that its better off without us are completely unfounded. There is a concern however, that someone would program an AI specifically with these motivations. In that case we do have everything to worry about.
- pdonis 13y ago
  Intelligence does not come with these motivations and drives, they are completely separate. Do you know of any examples of intelligent beings that don't have any motivations and drives? (Note that "motivations and drives" is a very general term; there's no need for them to be human motivations and drives. I agree the motivations and drives don't have to be human ones; but that's not the same as saying the AI has none at all.) There is no reason to think that a general AI would have any of these things, thus concerns about it deciding that its better off without us are completely unfounded. If an entity doesn't have some kind of motivation and drive, how can it be intelligent? Intelligence doesn't just mean cogitating in a vacuum; it means taking information in from the world, and doing things that have effects in the world. (Even if the AI just answers questions put to it, its answers are still actions that have effects in the world.) So an AI has to at least have the motivation and drive to take in information and do things with it; otherwise it's useless anyway. So given that the AI at least has to "want" to take in information and do things with it, how do you know the things it will want to do with the information are good things? ("Good" here means basically "beneficial to humans", since that's why we would want to build an AI in the first place.) We can say that we'll design the AI this way; but how do we know we can do that without making a mistake? A mistake doesn't have to be "oh, we programmed the AI to want to destroy the world; oops". A mistake is anything that causes a mismatch between what the AI is actually programmed to do, and what we really want it to do. Any programmer should know that this will always happen, in any program.

yk 13y ago

I think as long as we do not understand the relation between intelligence and conscience, we should probably assume that a AGI has moral issues and personal needs.