3 ms·

> When engineers/scientists/programmers think that AI is the true threat Engineers/scientists/programmers almost never think that AI is a threat. Because they

by maratd 12y ago

> When engineers/scientists/programmers think that AI is the true threat Engineers/scientists/programmers almost never think that AI is a threat. Because they understand the nature of the problem, how little we know about human intelligence, and how poorly our binary technology compares to what little we do know. Even in the unlikely event that we do develop a competent AI in the near future and a malicious AI comes into being ... there is no reason to think that a benevolent one won't be around at the same time ... and that the benevolent AIs will outnumber the malicious. Just like there are computers on the net doing bad things, there are plenty of others serving the role of protecting the common good. > The human race will destroy itself through reckless use of technology We haven't thus far. And we've had the capability for a while. Care to present some evidence or a rational argument that we will? Signs point to the contrary.

PeterisP 12y ago
A multitude of competing AIs would be very, very unlikely especially if one would expect that a different AI would appear. If a super-human AGI desires a benevolent future, then an obvious precondition for such a benevolent future to become real is to ensure that no other entities can threaten the future and that any other AIs are either with the same matching goals or permanently, fundamentally crippled to be limited in their power. An evil AI would do the same. An AI that doesn't care would be eliminated by the first (and the last) AI that does care. Humans don't try to achieve such goals because we obviously can't and we need others; an AI doesn't - it can be fine with being the only intelligent entity in the universe, and the only reason why an AI would allow other intelligent beings with potentially different goals to exist, is if it had been explicitly designed to want this.
- maratd 12y ago
  > Humans don't try to achieve such goals because we obviously can't and we need others We don't need all those others. In fact, we actually need far fewer than all the others out there. With the exception of a few lunatics here and there throughout history, human beings don't make it their business to exterminate the rest of humanity. It's a waste of time. We only compete with a small group of humans at any one time. Similarly, AIs whose interests don't overlap would have no interest in destroying other AIs. So I find it extremely likely that there would be a multitude of AIs, in the unlikely event that AIs come into fruition. Also, competition isn't constant for humans. One minute you're competing, another you're cooperating, and another you're ignoring. The same dynamism can be expected with AIs.
danso 12y ago
> We haven't thus far. And we've had the capability for a while. Care to present some evidence or a rational argument that we will? Signs point to the contrary. I'm going to assume your argument is not merely, "Well, we haven't yet destroyed ourselves, so therefore, we aren't capable of doing that"...because then my response will just be, "Well, we haven't yet developed AI, so therefore, we aren't capable of doing that"...and so your counter-argument is based on a different interpretation of history than mine: I think the decades of Cold War and the near-misses we had with all-out nuclear war are examples of situations in which we have the potential to quickly wipe ourselves out without the help of AI. Others would point out the trend of mass surveillance -- again, implemented and controlled by humans and human institutions -- of a harbinger of doom. Keep in mind that I'm not saying that technology has reached its peak. I am very open to the idea that we could reach a point of semi-autonomous systems, and yet still have human institutions as faulty as they are now, and the combination of both will result in a threat greater than what we've faced so far.
- maratd 12y ago
  > I'm going to assume your argument is not merely, "Well, we haven't yet destroyed ourselves, so therefore, we aren't capable of doing that" What? Read what I wrote again. "We haven't thus far. And we've had the capability for a while." We've had the capability for a while. That means we're clearly capable. That's what capability means. The fact that we haven't is supporting evidence that we won't. Because we've had the capability but have not used it. Which is historical evidence against our using it. Which is not a guarantee that it won't happen, just an indication that it won't.

by maratd 12y ago

PeterisP 12y ago

A multitude of competing AIs would be very, very unlikely especially if one would expect that a different AI would appear. If a super-human AGI desires a benevolent future, then an obvious precondition for such a benevolent future to become real is to ensure that no other entities can threaten the future and that any other AIs are either with the same matching goals or permanently, fundamentally crippled to be limited in their power. An evil AI would do the same. An AI that doesn't care would be eliminated by the first (and the last) AI that does care. Humans don't try to achieve such goals because we obviously can't and we need others; an AI doesn't - it can be fine with being the only intelligent entity in the universe, and the only reason why an AI would allow other intelligent beings with potentially different goals to exist, is if it had been explicitly designed to want this.

maratd 12y ago
> Humans don't try to achieve such goals because we obviously can't and we need others We don't need all those others. In fact, we actually need far fewer than all the others out there. With the exception of a few lunatics here and there throughout history, human beings don't make it their business to exterminate the rest of humanity. It's a waste of time. We only compete with a small group of humans at any one time. Similarly, AIs whose interests don't overlap would have no interest in destroying other AIs. So I find it extremely likely that there would be a multitude of AIs, in the unlikely event that AIs come into fruition. Also, competition isn't constant for humans. One minute you're competing, another you're cooperating, and another you're ignoring. The same dynamism can be expected with AIs.

danso 12y ago

> We haven't thus far. And we've had the capability for a while. Care to present some evidence or a rational argument that we will? Signs point to the contrary. I'm going to assume your argument is not merely, "Well, we haven't yet destroyed ourselves, so therefore, we aren't capable of doing that"...because then my response will just be, "Well, we haven't yet developed AI, so therefore, we aren't capable of doing that"...and so your counter-argument is based on a different interpretation of history than mine: I think the decades of Cold War and the near-misses we had with all-out nuclear war are examples of situations in which we have the potential to quickly wipe ourselves out without the help of AI. Others would point out the trend of mass surveillance -- again, implemented and controlled by humans and human institutions -- of a harbinger of doom. Keep in mind that I'm not saying that technology has reached its peak. I am very open to the idea that we could reach a point of semi-autonomous systems, and yet still have human institutions as faulty as they are now, and the combination of both will result in a threat greater than what we've faced so far.

maratd 12y ago
> I'm going to assume your argument is not merely, "Well, we haven't yet destroyed ourselves, so therefore, we aren't capable of doing that" What? Read what I wrote again. "We haven't thus far. And we've had the capability for a while." We've had the capability for a while. That means we're clearly capable. That's what capability means. The fact that we haven't is supporting evidence that we won't. Because we've had the capability but have not used it. Which is historical evidence against our using it. Which is not a guarantee that it won't happen, just an indication that it won't.