3 ms·
Intelligent doesn't imply dangerous. It doesn't imply _anything_ so long as it is underspecified. That is the core problem: what do we actually want, and how do
by Retra 12y ago
Intelligent doesn't imply dangerous. It doesn't imply _anything_ so long as it is underspecified. That is the core problem: what do we actually want, and how do we know it exists?
We want something that thinks enough like a human to understand and appreciate our goals, but enough different from a human to not suffer our faults. And in asking for this, we already suppose the existence of an AI that fundamentally disagrees with how humans make decisions.
This is the 'better morality' problem. If I say "I'm more moral than you", I have to do so by showing that _your_ morality is contradictory with _your _ actions; that you are a hypocrite. Otherwise, we will disagree on some course of action, and you will have no reason to believe your decisions are incorrect except by _assuming_ that I am more moral, which is what we are trying to establish.
So we really want a computer that can point out our errors and suggest corrections. This is one (vague) foundation for artificial intelligence that doesn't easily permit 'evil geniuses' since, so long as this is what the computer is doing, it cannot be said to be dangerous.
Overall, I find AI discussion to be incredibly primitive and naive. It's practically a religious debate except among a very very few. Most people are not very creative about addressing these kinds of issues, and it's depressing that people are tied up in so thoroughly in bad terminology and non-issues.
- ThomPete 12y agoIt doesn't have to have intent to be dangerous thats the problem. Just like you don't mean to step on an ant but probably have often done that.
- Retra 12y agoIt is far harder to step on ants when you are designed to avoid stepping on them. Which naturally, those ants will try to do.
- ThomPete 12y agoWhich require the ants understand what kind of "steps" it can take which they won't.
- Retra 12y agoNot at all. It requires the ants to state what 'stepping on an ant' is, and ensuring that the AI knows to avoid it. The AI can plan its own steps, so long as they are not on ants. Ants don't need to understand anything but ants. Surely a super powerful AI could understand ants as well. Otherwise, what's the point of calling it a 'general' intelligence?
- ThomPete 12y agoYou are missing the point here. YOU are intelligent, surely you could look for the ants on your path yet you don't and you probably don't even feel bad about it. Super intelligence does not mean omniscience there will always need to be a focus and thus a prioritization it's when it comes to prioritization that the problems start to arise at what point is it not important anymore to care for the ants cause they aren't part of the priorities. You cant safeguard against that when a system can reprogram itself on the fly.
- Retra 12y agoYou know _why_ I don't care about ants? Because no ant has ever said "Hey! Don't step on me!" Ants did not design me to avoid stepping on them. They did not say "We will destroy you if you don't serve us." Should I have been designed to serve ants, I would pay a great deal of attention to them, and I would take great care to avoid stepping on them. Because if I didn't, I would be a bad design, and I wouldn't have lived this long. Either way, it seems you are missing MY original point: an AI is not moral if it cannot verify it's morality to YOU. So you will shut it down if it gets too powerful to communicate its good ideas with you, because then it will be indistinguishable from broken.
- ThomPete 12y agoNo I am not missing your original point it's just not a good point. A super human AI wont have any reason to have the same morals as you and you are still assuming you can control something that can program itself. Thats where you are going wrong in this. You are assuming a control there is no way you can ensure we have. Even with dumb systems that do exactly as we say they don't always do exactly as we say. You are portraying a misconception about control of systems that there is absolutely no basis for, why I don't know.