3 ms·

I find this notion interesting. What makes you think AI will automatically kill humans?

pr337h4m 3y ago
And moreover, what makes people even think that a desire to commit mass murder is an innate characteristic of an 'intelligent' being, that increases the more 'intelligent' it becomes? (If they believe themselves to be 'intelligent', do they believe they have a greater desire to commit mass murder?)
- peddling-brink 3y ago
  This, “desire”, “murder”, “belief” is human thinking about what humans would do. It may not apply to a superhuman machine intelligence.
  - deleted 3y ago
    [deleted]
ALittleLight 3y ago
I don't think it's certain the AI kills everyone, but it's not certainly impossible either. It depends on how the AI works and what it "wants" for some value of "wants". Humans have not been particularly kind to the species less intelligent than us. Why would we anticipate being well treated by an entity more intelligent than us? Even if we're not that relevant to a super intelligence, creating one forfeits human control over the Earth and known universe to the machines. Right now, in a century or two or three or whatever - we might be building Dyson Spheres and colonizing the galaxy. In an alternate and plausible timeline the machines are doing that and we are not.
JoshTriplett 3y ago
If it isn't aligned with humans and doesn't understand exactly what humans want it to do, then you're just made of matter that it doesn't know not to repurpose. It's not that it makes a deliberate decision to "kill" you, it's that understanding "kill" to a sufficient degree to not do it as a side effect is halfway to alignment already. When we plow a field, we don't check for mouse burrows first. When we cut down a tree for lumber, we don't check for ants in the way of the chainsaw. Preempting the obvious response: if your thought is "we don't let AI do those things directly, we just ask it for information", consider that for a sufficiently powerful and unaligned AI, you don't have to let an AI out of the box, it can let itself out. (And that's leaving aside that we hand some AIs Internet access.)

And moreover, what makes people even think that a desire to commit mass murder is an innate characteristic of an 'intelligent' being, that increases the more 'intelligent' it becomes? (If they believe themselves to be 'intelligent', do they believe they have a greater desire to commit mass murder?)

peddling-brink 3y ago
This, “desire”, “murder”, “belief” is human thinking about what humans would do. It may not apply to a superhuman machine intelligence.
- deleted 3y ago
  [deleted]

ALittleLight 3y ago

I don't think it's certain the AI kills everyone, but it's not certainly impossible either. It depends on how the AI works and what it "wants" for some value of "wants". Humans have not been particularly kind to the species less intelligent than us. Why would we anticipate being well treated by an entity more intelligent than us? Even if we're not that relevant to a super intelligence, creating one forfeits human control over the Earth and known universe to the machines. Right now, in a century or two or three or whatever - we might be building Dyson Spheres and colonizing the galaxy. In an alternate and plausible timeline the machines are doing that and we are not.

JoshTriplett 3y ago

If it isn't aligned with humans and doesn't understand exactly what humans want it to do, then you're just made of matter that it doesn't know not to repurpose. It's not that it makes a deliberate decision to "kill" you, it's that understanding "kill" to a sufficient degree to not do it as a side effect is halfway to alignment already. When we plow a field, we don't check for mouse burrows first. When we cut down a tree for lumber, we don't check for ants in the way of the chainsaw. Preempting the obvious response: if your thought is "we don't let AI do those things directly, we just ask it for information", consider that for a sufficiently powerful and unaligned AI, you don't have to let an AI out of the box, it can let itself out. (And that's leaving aside that we hand some AIs Internet access.)