5 ms·

I'm much more scared of the AGI, because even the most inhuman human is still human, and I can have sympathy for them.

by mitthrowaway2 3y ago

bamboozled 3y ago
How do you know part of being self aware and intelligent also means compassion is part of the package ?
- mitthrowaway2 3y ago
  Sorry, I don't understand what you mean. Did you mean to reply to a different comment?
  - bamboozled 3y ago
    I mean, maybe what happens is, intelligence + self-awareness gives rise to compassion in even digital beings. Understanding might be the basis for compassion.
    - kmnc 3y ago
      While this idea sounds a bit fluffy, I actually think there may be something to this, understanding the “other” may very well be an integral component in what we define as sentience.
    - mitthrowaway2 3y ago
      Yes, maybe what happens is exactly that. We can put on a blindfold, hit the gas pedal, and trust in the benevolence of nature that things will go our way. But AI doomsayers aren't saying that there's no possible good outcomes, they're saying that the good outcomes are a narrow target surrounded by a vast ocean of bad outcomes, and it will take more than luck to hit them. It will take careful, detailed analysis, and a great deal of precaution, verification, engineering, and double- and triple-checking. Not luck. Not hope.
      - bamboozled 3y ago
        Have the doomsayers prevent anything significant from going forwards ?
drdaeman 3y ago
Won't any AGI made by humans unavoidably have human origins, and thus, simply be a [post-]human? I don't see a reason I should be scared of a non-biological but generally human-like intelligence - not on a general principle. If anything, in general, I'd be happy that someone is finally getting real good chances of conquering death. I possibly would fear genuine non-human intelligence (on a principle of fearing what we don't understand), but given that chances of one evolving under our nose, on its own, entirely within its own world, without the connection for ours, then emerging, are quite improbable... Unless real aliens arrive - but let's avoid spoilers for 2025 this early. What should I fear, I think, is hostile intelligence, but then I'm not sure I should discriminate on its origin. I mean, there are some very real humans out there who can try to end this world (in one way or another), so I'm probably kind of desensitized and isn't exactly afraid of some very theoretical AGI threats.
- mitthrowaway2 3y ago
  If the AGI were made from a detailed scan of a human brain converted into a physical simulation, or otherwise proved very similar in terms of embodying human values, then perhaps. But it doesn't look like things are going this way. Humans make a lot of things that range from clever to mundane to horrifying, and I can't think of a single one of them that I would be happy with replacing humans across the surface of the Earth, even if it were very good at maximizing production figures of a paperclip factory or responding helpfully to queries.
  - drdaeman 3y ago
    That’s a hostile intelligence scenario, right? What I wanted to say is that, I, in particular, really don’t care if it is a human trying to harm me, or a machine. Both of those scenarios are so unfavorable, I really don’t see a point in having any preference. So, I guess, I can’t think of any evil human to replace with even just an evil robot either. But I think I’m less imaginative of a machine becoming sentient and hostile, than a human doing so. The former hadn’t happened - and isn’t exactly trivial, given how big this fear is a part of human culture, and how slow and limited first AGI is inevitably going to be. And then we have an abundance of the latter.
deleted 3y ago
[deleted]
kouru225 3y ago
Sounds kinda like your problem ngl

I'm much more scared of the AGI, because even the most inhuman human is still human, and I can have sympathy for them.

bamboozled 3y ago

How do you know part of being self aware and intelligent also means compassion is part of the package ?

mitthrowaway2 3y ago
Sorry, I don't understand what you mean. Did you mean to reply to a different comment?
- bamboozled 3y ago
  I mean, maybe what happens is, intelligence + self-awareness gives rise to compassion in even digital beings. Understanding might be the basis for compassion.
  - kmnc 3y ago
    While this idea sounds a bit fluffy, I actually think there may be something to this, understanding the “other” may very well be an integral component in what we define as sentience.
  - mitthrowaway2 3y ago
    Yes, maybe what happens is exactly that. We can put on a blindfold, hit the gas pedal, and trust in the benevolence of nature that things will go our way. But AI doomsayers aren't saying that there's no possible good outcomes, they're saying that the good outcomes are a narrow target surrounded by a vast ocean of bad outcomes, and it will take more than luck to hit them. It will take careful, detailed analysis, and a great deal of precaution, verification, engineering, and double- and triple-checking. Not luck. Not hope.
    - bamboozled 3y ago
      Have the doomsayers prevent anything significant from going forwards ?

drdaeman 3y ago

Won't any AGI made by humans unavoidably have human origins, and thus, simply be a [post-]human? I don't see a reason I should be scared of a non-biological but generally human-like intelligence - not on a general principle. If anything, in general, I'd be happy that someone is finally getting real good chances of conquering death. I possibly would fear genuine non-human intelligence (on a principle of fearing what we don't understand), but given that chances of one evolving under our nose, on its own, entirely within its own world, without the connection for ours, then emerging, are quite improbable... Unless real aliens arrive - but let's avoid spoilers for 2025 this early. What should I fear, I think, is hostile intelligence, but then I'm not sure I should discriminate on its origin. I mean, there are some very real humans out there who can try to end this world (in one way or another), so I'm probably kind of desensitized and isn't exactly afraid of some very theoretical AGI threats.

mitthrowaway2 3y ago
If the AGI were made from a detailed scan of a human brain converted into a physical simulation, or otherwise proved very similar in terms of embodying human values, then perhaps. But it doesn't look like things are going this way. Humans make a lot of things that range from clever to mundane to horrifying, and I can't think of a single one of them that I would be happy with replacing humans across the surface of the Earth, even if it were very good at maximizing production figures of a paperclip factory or responding helpfully to queries.
- drdaeman 3y ago
  That’s a hostile intelligence scenario, right? What I wanted to say is that, I, in particular, really don’t care if it is a human trying to harm me, or a machine. Both of those scenarios are so unfavorable, I really don’t see a point in having any preference. So, I guess, I can’t think of any evil human to replace with even just an evil robot either. But I think I’m less imaginative of a machine becoming sentient and hostile, than a human doing so. The former hadn’t happened - and isn’t exactly trivial, given how big this fear is a part of human culture, and how slow and limited first AGI is inevitably going to be. And then we have an abundance of the latter.

deleted 3y ago

[deleted]

kouru225 3y ago

Sounds kinda like your problem ngl