3 ms·

A Creepy Question We'll All Have to Answer Soon

14 points by jger15 3y ago

greenbit 3y ago
Was half expecting the robot child dialog to swerve into "so if there were no functioning society, what I did would not be wrong? One moment .."
dimmke 3y ago
Read the title as “A Creepy Question We’ll All Have the Answer to Soon” and was immediately extremely intrigued. Then I read the title again and lost interest in even clicking through. Amazing what difference a word can make in a title.
dwroberts 3y ago
> Do we need to create machines that can suffer? No. Thanks for coming to my TED talk
- ChatGTP 3y ago
  But actually I think if the AI systems were to survive they would need to learn to cooperate and probably understand each others direction or have empathy. Imagine 5 super intelligences that hated each other and just wanted to wipe each other out? They’d get nothing done… If they had conflicting goals, they’d have to first stop one another from screwing up their plans…
  - jareklupinski 3y ago
    i'd rather have 5 superintelligences fighting each other than having them collude against us but that's just my biobrain thinking :(
q87b 3y ago
Anti-clickbait: The question is in the subheading "Do we need to create machines that can suffer?"
choudharism 3y ago
I like John and his body of work, but this and the general slew of posts with very little actual knowledge or expertise making grandiose / prophetic assertions are the soothsayings of our age - people convinced that they've cracked "it", but without any factual reasoning as to why. They raise philosophical assertions disguised as logical / scientific assertions. > “Better.” > That is a concept that cannot be made via cold calculation. A universe boiled down to particles and numbers cannot care whether atoms arranged as a suffering plague victim are “better” than those arranged as a healthy child, or a plant, or a cloud of gas. It can, John. The engineers teach machines what's good and bad. That's the first step in creating silicon intelligence. > It is impossible to understand pain without having felt it yourself; it does not convey as an abstract value. I think the machine needs to tangibly interact with human suffering in the same way the Roomba’s wheels need to physically touch the floor. Pain is nature's deterrence mechanism, bred into us through evolution. Deterrence mechanisms can take many shapes. > This leads us to the ultimate cosmic joke: This task requires answering for this machine questions that we ourselves have never been able to answer. Let’s say I’m right and this system won’t exist until we can teach it emotion, and the capacity to suffer. How long until it asks the simple question, “What is our goal, long-term?” Like, as a civilization, what are we trying to do? What are hoping the superintelligence will assist us in achieving? I hate being disingenuous and make a flippant argument here, but reading this from start to finish makes me ask a bad faith question - your assertion is that we won't have self driving cars because we can't tell the robots what to do with the whole civilisation right away? There's a whole spectrum of amazing (or terrible) possibilities between the extremes, my guy.
- ChatGTP 3y ago
  But what is really good and bad? We’re imprinting this into the machines but we don’t always have a grasp on this ourselves. Was the use nuclear weapons at the end of WW2 good or bad ? For example …
  - choudharism 3y ago
    Does any intelligent life exist on earth which doesn't hold a formative black-and-white answer to that question?
ben_w 3y ago
An interesting thought experiment. I aver the negative, on the grounds that, even for humans, recognising emotional states well enough to engage with other humans is independent of empathising with them. Narcissistic personality disorder[0] appears to be an example of the former without the latter, while ASD[0] appears to be an example of the latter without the former. [0] judging by the popular descriptions, I'm not qualified in this field
- ChatGTP 3y ago
  What about a world full of incompatible, narcissistic, competing AIs?
  - ben_w 3y ago
    I think that premise requires the AI to have qualia, but other than that I can't really say anything; I'm not even sure what you're asking me to what-about.
    - ChatGTP 3y ago
      There definitely seems to be a lot of assumptions being made the AI won't have qualia, let's hope the assumptions remain true, but I don't imagine it will be the case forever?
      - ben_w 3y ago
        As we don't yet, so far as I know, have a testable hypotheses for the necessary and sufficient conditions for qualia, I think we can neither confirm that any given AI (or animal, including both cattle and other humans) does or does not have it.
js8 3y ago
I think whether something is painful or not depends on the perspective. Humans can voluntarily suffer lots of pain for a good cause. And vice versa, small things can be extremely painful if we don't understand them. (Edit: In particular, to some people, even prospect of loss of their own property is more painful than death of other people.) So it comes down to your expectations. In reinforcement learning, this is very clear - bad predictions of your future actions lead to negative reward, i.e. pain. We could almost say pain depends on your expectations of reality (however the expectations are not always consciously expressed). So you can torture a human, because our nerves have learned (have the expectation) of not being in pain as a normal state. If they could learn more (like RL agent), they would eventually learn that the pain is just nature of the world, and ignore it. Similarly, Data can ignore pain of his missing hand because he trusts the Enterprise crew to put it back. If they didn't and betray him, or if he hurt his arm so badly it couldn't be replaced, then I think, on some level, he would feel pain. (Edit: Data mourned death of Tasha Yar, this can be interpreted as pain.)
trabant00 3y ago
I'm having a hard time understanding these ethical questions about things that we haven't proven we can build and I'm very skeptical about the "Soon" part. It all seems like trying to build an air traffic control model in the 1800s back when we only had dreams about flying. Even in Asimov's SF novels humanity iterates through several models by ways of trial and error over hundreds/thousands of years before arriving at somewhat stable laws.
hug 3y ago
The author fails in imagination, expecting that future computer intelligence needs to follow the path of human intelligence, where it can’t be immediately snapshotted, reset, rebirths, or fed new information. Is it cruelty to delete an AI without it knowing that might happen? Is it cruelty to inject it with feelings of euphoria for its pending deletion? Two myopic thoughts that reach further than the article’s premise.
DemocracyFTW2 3y ago
> if [a Roomba] runs into a wall, a switch is depressed See, machines do have emotions after all! Case closed I say!
deleted 3y ago
[deleted]
ukuuru 3y ago
Human level AI is unlikely but near human level will do just like every other prodcut engineered to do its thing around us. In theory a programmable entity that feels would have infinite applications but an entity that can quantize feelings can cover lets say 90% of those applications.
zuuuuuh 3y ago
[dead]

14 points by jger15 3y ago

Was half expecting the robot child dialog to swerve into "so if there were no functioning society, what I did would not be wrong? One moment .."

dimmke 3y ago

Read the title as “A Creepy Question We’ll All Have the Answer to Soon” and was immediately extremely intrigued. Then I read the title again and lost interest in even clicking through. Amazing what difference a word can make in a title.

dwroberts 3y ago

> Do we need to create machines that can suffer? No. Thanks for coming to my TED talk

ChatGTP 3y ago
But actually I think if the AI systems were to survive they would need to learn to cooperate and probably understand each others direction or have empathy. Imagine 5 super intelligences that hated each other and just wanted to wipe each other out? They’d get nothing done… If they had conflicting goals, they’d have to first stop one another from screwing up their plans…
- jareklupinski 3y ago
  i'd rather have 5 superintelligences fighting each other than having them collude against us but that's just my biobrain thinking :(

q87b 3y ago

Anti-clickbait: The question is in the subheading "Do we need to create machines that can suffer?"

choudharism 3y ago

I like John and his body of work, but this and the general slew of posts with very little actual knowledge or expertise making grandiose / prophetic assertions are the soothsayings of our age - people convinced that they've cracked "it", but without any factual reasoning as to why. They raise philosophical assertions disguised as logical / scientific assertions. > “Better.” > That is a concept that cannot be made via cold calculation. A universe boiled down to particles and numbers cannot care whether atoms arranged as a suffering plague victim are “better” than those arranged as a healthy child, or a plant, or a cloud of gas. It can, John. The engineers teach machines what's good and bad. That's the first step in creating silicon intelligence. > It is impossible to understand pain without having felt it yourself; it does not convey as an abstract value. I think the machine needs to tangibly interact with human suffering in the same way the Roomba’s wheels need to physically touch the floor. Pain is nature's deterrence mechanism, bred into us through evolution. Deterrence mechanisms can take many shapes. > This leads us to the ultimate cosmic joke: This task requires answering for this machine questions that we ourselves have never been able to answer. Let’s say I’m right and this system won’t exist until we can teach it emotion, and the capacity to suffer. How long until it asks the simple question, “What is our goal, long-term?” Like, as a civilization, what are we trying to do? What are hoping the superintelligence will assist us in achieving? I hate being disingenuous and make a flippant argument here, but reading this from start to finish makes me ask a bad faith question - your assertion is that we won't have self driving cars because we can't tell the robots what to do with the whole civilisation right away? There's a whole spectrum of amazing (or terrible) possibilities between the extremes, my guy.

ChatGTP 3y ago
But what is really good and bad? We’re imprinting this into the machines but we don’t always have a grasp on this ourselves. Was the use nuclear weapons at the end of WW2 good or bad ? For example …
- choudharism 3y ago
  Does any intelligent life exist on earth which doesn't hold a formative black-and-white answer to that question?

ben_w 3y ago

An interesting thought experiment. I aver the negative, on the grounds that, even for humans, recognising emotional states well enough to engage with other humans is independent of empathising with them. Narcissistic personality disorder[0] appears to be an example of the former without the latter, while ASD[0] appears to be an example of the latter without the former. [0] judging by the popular descriptions, I'm not qualified in this field

ChatGTP 3y ago
What about a world full of incompatible, narcissistic, competing AIs?
- ben_w 3y ago
  I think that premise requires the AI to have qualia, but other than that I can't really say anything; I'm not even sure what you're asking me to what-about.
  - ChatGTP 3y ago
    There definitely seems to be a lot of assumptions being made the AI won't have qualia, let's hope the assumptions remain true, but I don't imagine it will be the case forever?
    - ben_w 3y ago
      As we don't yet, so far as I know, have a testable hypotheses for the necessary and sufficient conditions for qualia, I think we can neither confirm that any given AI (or animal, including both cattle and other humans) does or does not have it.

js8 3y ago

I think whether something is painful or not depends on the perspective. Humans can voluntarily suffer lots of pain for a good cause. And vice versa, small things can be extremely painful if we don't understand them. (Edit: In particular, to some people, even prospect of loss of their own property is more painful than death of other people.) So it comes down to your expectations. In reinforcement learning, this is very clear - bad predictions of your future actions lead to negative reward, i.e. pain. We could almost say pain depends on your expectations of reality (however the expectations are not always consciously expressed). So you can torture a human, because our nerves have learned (have the expectation) of not being in pain as a normal state. If they could learn more (like RL agent), they would eventually learn that the pain is just nature of the world, and ignore it. Similarly, Data can ignore pain of his missing hand because he trusts the Enterprise crew to put it back. If they didn't and betray him, or if he hurt his arm so badly it couldn't be replaced, then I think, on some level, he would feel pain. (Edit: Data mourned death of Tasha Yar, this can be interpreted as pain.)

trabant00 3y ago

I'm having a hard time understanding these ethical questions about things that we haven't proven we can build and I'm very skeptical about the "Soon" part. It all seems like trying to build an air traffic control model in the 1800s back when we only had dreams about flying. Even in Asimov's SF novels humanity iterates through several models by ways of trial and error over hundreds/thousands of years before arriving at somewhat stable laws.

hug 3y ago

The author fails in imagination, expecting that future computer intelligence needs to follow the path of human intelligence, where it can’t be immediately snapshotted, reset, rebirths, or fed new information. Is it cruelty to delete an AI without it knowing that might happen? Is it cruelty to inject it with feelings of euphoria for its pending deletion? Two myopic thoughts that reach further than the article’s premise.

DemocracyFTW2 3y ago

> if [a Roomba] runs into a wall, a switch is depressed See, machines do have emotions after all! Case closed I say!

deleted 3y ago

[deleted]

ukuuru 3y ago

Human level AI is unlikely but near human level will do just like every other prodcut engineered to do its thing around us. In theory a programmable entity that feels would have infinite applications but an entity that can quantize feelings can cover lets say 90% of those applications.

zuuuuuh 3y ago

[dead]