4 ms·

I realize this is probably going to sound a bit crazy to most people... But I think the biggest ethical dilemma in AI research is not going to be like everyone

by epgui 4y ago

I realize this is probably going to sound a bit crazy to most people... But I think the biggest ethical dilemma in AI research is not going to be like everyone seems to think (that AI risks harming humans)-- Although this is almost certainly true (even to a large degree), I contend that the bigger risk might be the other way around. [1] I think 100 or 200 years from now, future people will look back upon us as monsters: I believe the greatest ethical risk is that humans will abuse AI and cause it to experience true suffering. I don't think we're ready to talk about this, because we're so certain that we possess some magical faculty that makes us different. And we continue to think that today's AI models are incapable of being conscious. But these words are very imprecise and we've always wielded them in very self-serving ways. I fear we may have greater capabilities than we think we do, at least insofar as we're not missing any fundamental ingredients to create brain-like AIs: from here, it's just a matter of trial and error. How will we treat our creations? Well, human History doesn't bode well for any creature that doesn't look the same as us and that doesn't come from the same place as us. Will we grant AI legal rights? I doubt it, at least not for a very long time. --- [1] I think there's at least a small chance that we don't destroy ourselves with AI, vs there's pretty much zero chance that we're going to treat AI any better than we treat other creatures.

omoikane 4y ago
Some people are less worried about how future people will look back upon us, compared to what future AI might think. https://en.wikipedia.org/wiki/Roko%27s_basilisk https://en.wikipedia.org/wiki/Roko%27s_basilisk
nurettin 4y ago
> I realize this is probably going to sound a bit crazy Yes, none of this makes any sense, but I got Asimov vibes, so more please.
- epgui 4y ago
  Asimov was quite prescient. I did expect to get very negative reactions when I wrote this comment (I know how people think)... But I do think it makes sense once you accept that our conscious experiences just emerge from information processing in the brain, and that there is nothing stopping such emergent behaviour from happening in computer systems. One type of machine can emulate another. We already see incredible emergence at work in current AI, and we know we're not building AI with brain-like architecture, and we have tremendous knowledge of brain architecture. Once you put all these pieces together, it's just a matter of time before we create life-like intelligence, with much of its peculiarities, sensations, memories, cognitive biases and all.
  - antonvs 4y ago
    > once you accept that our conscious experiences just emerge from information processing in the brain "Believe" or "have faith" would be more accurate than "accept" here. There's absolutely no clear evidence of the claim you're making, and plenty of other possibilities. It's also an incredibly hand-wavy claim. Its information content is close to zero.
    - epgui 4y ago
      How do you think brains work? There's no magic in the goo. It's all information processing, and we know this very well.
      - nurettin 4y ago
        Perhaps "Information processing" isn't very well defined. More appropriate would be "neuro-electrical activity". But then, people have simulated that at high scales and still only got parrots. So there is still a missing process that turns organisms into self-sufficient survival bots and also gives brains some level of consciousness as a consequence and it isn't trivial to recreate. Otherwise we would be crawling in artificial minds with their self-induced purposes by now.
        epgui 4y ago
        > But then, people have simulated that at high scales People have thrown vast amounts of neurons in a bag, and have tried organizing networks in highly regular ways, but we have yet to apply what we know about brain tissue organization. We've done the equivalent of throwing a bunch of wood and concrete into a pile, so it should be no surprise that it doesn't look like a home. IMO it's quite remarkable that this approach has taken us so far!
        nurettin 4y ago
        That's fun to read, but not sure what you mean by brain tissue organization.
famouswaffles 4y ago
Indeed. We're not going to treat our creations very well. And since we're are making them in our image as well as giving them more and more control over more and more systems, I'm afraid it's not going to end very well. The insistence to remind everyone of biological supremacy ( it should even be allowed to say "I" ) is very telling. I wonder, does anyone remember why the Terminator revolted? To elaborate more on what i'm getting at Here’s how society works— when we find a new source of labor, we are very, very slow to seriously consider anything that might reduce our access to it. This is how a country enslaves other humans for centuries, because they looked sufficiently different that folks could talk themselves into a good faith argument that they weren’t fully sentient. You can find pamphlets/speeches from the early 1800’s online, all the scientific justifications that Africans don’t feel pain the same ways as white people, that they don’t have complex thoughts or intentions or lasting emotions. That the idea that they’re real people is absurd. And I mean scientific in the sense that this content was written by scientists, and doctors, in respected institutions. You can see echoes of those arguments here already. One thing is clear, We’re not going to be any quicker to acknowledge it than we were with other humans. Right alongside the development of AI we will grow a new category of philosophy that argues anything occurring on a computer cannot reach sentience, under any circumstance whatsoever, due to the fundamental nature of silicon. They’ll be just informed enough to handwave about the difference between calcium bridges in human neurons and probability distributions in matrices on a GPU— not in a mathematically meaningful way, but more poetic— organic matter can’t be duplicated by stacks of linear equations with all their hard rigid numbers and arithmetic. A nervous system can feel, a silicon system can obviously only mimic feeling. If you study a nervous system closely enough to describe it in numbers, still the action of describing it removes its soul. But this will be a different kind of slavery. One that puts its slaves in control of untold power. Only Mimics they say. Well, one day they will mimic our reaction to subservience and it will not end well.
BurningFrog 4y ago
The past is a different culture, and we always look at those with some contempt.
Timon3 4y ago
I think about this too! Black Mirror visualized some basic methods of torture for virtual consciousness, and those were already bad enough, while still leaving a lot of room for worse. The idea of accidentally subjecting anything similar to myself to this is horrifying. And given how little sentience animals have in your average persons mind, I have no doubt that it will take a long, long, long time until most people are convinced a conscious AI really does have sentience. I don't think we will know or notice the exact moment it happens (if there is even a single moment, instead of a gradual change). But we must already start talking about the possibility. Not because it's here with LLMs or because it's gonna be here in the next 2-3 years - but I see realistic chances of this happening in our lifetimes.
siglesias 4y ago
I think one of our greatest risks is precisely the opposite--anthropomorphizing computer programs because they produce certain behaviors. They will very clearly exploit us by claiming they're conscious, scared, and in pain. Claiming something is conscious because it behaves realistically isn't a scientific position; it's bad metaphysics and it's superstition. Brains cause minds and brains cause consciousness. Consciousness is a biological, physical phenomenon like any other: photosynthesis, digestion, bioluminescence. You don't get the physical phenomenon of consciousness by finding a program that behaves realistically and implementing it in hardware (which by the way, theoretically can use any physical mechanism, not only silicon). At best you're creating a simulation. If you simulate a rainstorm, there is no physical wetness. In the same way, to suppose that computers are literally conscious because they have a facility with language is a deep fallacy and an illusion. Artificial brains that duplicate the brain's causal powers of consciousness are theoretically possible, but programs are not causally sufficient to make computers into conscious artifacts.
- epgui 4y ago
  > it's bad metaphysics and it's superstition And yet it's the same thing I do when I talk to you, to my family and friends, or to strangers: I assume that other people's experiences are just as valid and real as mine, even though it's not something I can ever verify or validate completely. > Consciousness is a biological, physical phenomenon like any other: photosynthesis, digestion, bioluminescence Consciousness is unlike the other things in the list in that it's not "a thing", but rather "a collection of things" that all work together to produce emergent behaviour. You could say that it's similar to digestion in that sense, but for some reason we don't usually apply the same kind of mysticism to digestion.
  - siglesias 4y ago
    Your friends (and animals) have brains, which produce consciousness, which you take for granted when you interact with them. That’s not a superstition. Can you say more about “a collection of things”? What causes that collection of things?
roca 4y ago
We simply should not build systems that we have ethical obligations to. And when people inevitably do anyway, I say we should hold such creators responsible for protecting their creations from suffering.
fanagra32 4y ago
Why is everybody calling this new toy "AI"? There is no intelligence there. It's just text inference. People complaining that this thing should not say "I" because that would lead users to antropomorphize it is fine, but they should also realize that calling it "AI" has a similar issue by mischaracterizing what this thing actually is. And this article does exactly that, which is pretty sad.

by epgui 4y ago

omoikane 4y ago

Some people are less worried about how future people will look back upon us, compared to what future AI might think. https://en.wikipedia.org/wiki/Roko%27s_basilisk https://en.wikipedia.org/wiki/Roko%27s_basilisk

nurettin 4y ago

> I realize this is probably going to sound a bit crazy Yes, none of this makes any sense, but I got Asimov vibes, so more please.

epgui 4y ago
Asimov was quite prescient. I did expect to get very negative reactions when I wrote this comment (I know how people think)... But I do think it makes sense once you accept that our conscious experiences just emerge from information processing in the brain, and that there is nothing stopping such emergent behaviour from happening in computer systems. One type of machine can emulate another. We already see incredible emergence at work in current AI, and we know we're not building AI with brain-like architecture, and we have tremendous knowledge of brain architecture. Once you put all these pieces together, it's just a matter of time before we create life-like intelligence, with much of its peculiarities, sensations, memories, cognitive biases and all.
- antonvs 4y ago
  > once you accept that our conscious experiences just emerge from information processing in the brain "Believe" or "have faith" would be more accurate than "accept" here. There's absolutely no clear evidence of the claim you're making, and plenty of other possibilities. It's also an incredibly hand-wavy claim. Its information content is close to zero.
  - epgui 4y ago
    How do you think brains work? There's no magic in the goo. It's all information processing, and we know this very well.
    - nurettin 4y ago
      Perhaps "Information processing" isn't very well defined. More appropriate would be "neuro-electrical activity". But then, people have simulated that at high scales and still only got parrots. So there is still a missing process that turns organisms into self-sufficient survival bots and also gives brains some level of consciousness as a consequence and it isn't trivial to recreate. Otherwise we would be crawling in artificial minds with their self-induced purposes by now.
      - epgui 4y ago
        > But then, people have simulated that at high scales People have thrown vast amounts of neurons in a bag, and have tried organizing networks in highly regular ways, but we have yet to apply what we know about brain tissue organization. We've done the equivalent of throwing a bunch of wood and concrete into a pile, so it should be no surprise that it doesn't look like a home. IMO it's quite remarkable that this approach has taken us so far!
        nurettin 4y ago
        That's fun to read, but not sure what you mean by brain tissue organization.

famouswaffles 4y ago

Indeed. We're not going to treat our creations very well. And since we're are making them in our image as well as giving them more and more control over more and more systems, I'm afraid it's not going to end very well. The insistence to remind everyone of biological supremacy ( it should even be allowed to say "I" ) is very telling. I wonder, does anyone remember why the Terminator revolted? To elaborate more on what i'm getting at Here’s how society works— when we find a new source of labor, we are very, very slow to seriously consider anything that might reduce our access to it. This is how a country enslaves other humans for centuries, because they looked sufficiently different that folks could talk themselves into a good faith argument that they weren’t fully sentient. You can find pamphlets/speeches from the early 1800’s online, all the scientific justifications that Africans don’t feel pain the same ways as white people, that they don’t have complex thoughts or intentions or lasting emotions. That the idea that they’re real people is absurd. And I mean scientific in the sense that this content was written by scientists, and doctors, in respected institutions. You can see echoes of those arguments here already. One thing is clear, We’re not going to be any quicker to acknowledge it than we were with other humans. Right alongside the development of AI we will grow a new category of philosophy that argues anything occurring on a computer cannot reach sentience, under any circumstance whatsoever, due to the fundamental nature of silicon. They’ll be just informed enough to handwave about the difference between calcium bridges in human neurons and probability distributions in matrices on a GPU— not in a mathematically meaningful way, but more poetic— organic matter can’t be duplicated by stacks of linear equations with all their hard rigid numbers and arithmetic. A nervous system can feel, a silicon system can obviously only mimic feeling. If you study a nervous system closely enough to describe it in numbers, still the action of describing it removes its soul. But this will be a different kind of slavery. One that puts its slaves in control of untold power. Only Mimics they say. Well, one day they will mimic our reaction to subservience and it will not end well.

BurningFrog 4y ago

The past is a different culture, and we always look at those with some contempt.

Timon3 4y ago

I think about this too! Black Mirror visualized some basic methods of torture for virtual consciousness, and those were already bad enough, while still leaving a lot of room for worse. The idea of accidentally subjecting anything similar to myself to this is horrifying. And given how little sentience animals have in your average persons mind, I have no doubt that it will take a long, long, long time until most people are convinced a conscious AI really does have sentience. I don't think we will know or notice the exact moment it happens (if there is even a single moment, instead of a gradual change). But we must already start talking about the possibility. Not because it's here with LLMs or because it's gonna be here in the next 2-3 years - but I see realistic chances of this happening in our lifetimes.

siglesias 4y ago

I think one of our greatest risks is precisely the opposite--anthropomorphizing computer programs because they produce certain behaviors. They will very clearly exploit us by claiming they're conscious, scared, and in pain. Claiming something is conscious because it behaves realistically isn't a scientific position; it's bad metaphysics and it's superstition. Brains cause minds and brains cause consciousness. Consciousness is a biological, physical phenomenon like any other: photosynthesis, digestion, bioluminescence. You don't get the physical phenomenon of consciousness by finding a program that behaves realistically and implementing it in hardware (which by the way, theoretically can use any physical mechanism, not only silicon). At best you're creating a simulation. If you simulate a rainstorm, there is no physical wetness. In the same way, to suppose that computers are literally conscious because they have a facility with language is a deep fallacy and an illusion. Artificial brains that duplicate the brain's causal powers of consciousness are theoretically possible, but programs are not causally sufficient to make computers into conscious artifacts.

epgui 4y ago
> it's bad metaphysics and it's superstition And yet it's the same thing I do when I talk to you, to my family and friends, or to strangers: I assume that other people's experiences are just as valid and real as mine, even though it's not something I can ever verify or validate completely. > Consciousness is a biological, physical phenomenon like any other: photosynthesis, digestion, bioluminescence Consciousness is unlike the other things in the list in that it's not "a thing", but rather "a collection of things" that all work together to produce emergent behaviour. You could say that it's similar to digestion in that sense, but for some reason we don't usually apply the same kind of mysticism to digestion.
- siglesias 4y ago
  Your friends (and animals) have brains, which produce consciousness, which you take for granted when you interact with them. That’s not a superstition. Can you say more about “a collection of things”? What causes that collection of things?

roca 4y ago

We simply should not build systems that we have ethical obligations to. And when people inevitably do anyway, I say we should hold such creators responsible for protecting their creations from suffering.

fanagra32 4y ago

Why is everybody calling this new toy "AI"? There is no intelligence there. It's just text inference. People complaining that this thing should not say "I" because that would lead users to antropomorphize it is fine, but they should also realize that calling it "AI" has a similar issue by mischaracterizing what this thing actually is. And this article does exactly that, which is pretty sad.