4 ms·

> The idea of making conscious AI scares me, because we could potentially create an AI that just experiences constant suffering at a level unimaginable to human

> The idea of making conscious AI scares me, because we could potentially create an AI that just experiences constant suffering at a level unimaginable to humans. Really scary from an ethical perspective. Some schools of thought posit that everything has consciousness. Including things like rocks. Even these things suffer; you just can’t hear the screams.

Trasmatta 6y ago
I don't really buy into panpsychism, but it is impossible to disprove. If things like rocks were conscious in some way, I don't think there would be any suffering present. Our suffering is intrinsically tied to our physical and mental systems. "What it's like to be a rock" would be something completely alien to our experience, and without the survival based mechanisms that we have, likely to be devoid of anything like suffering. An AI, on the other hand, could be an entirely different story.
enchiridion 6y ago
Can you elaborate?
- Trasmatta 6y ago
  Take a look at panpsychism. It's interesting, but I've never been sold on it. Integrated information theory (like this article talks about) is more palatable for me. It allows for the possibility of consciousness of certain non biological systems, but doesn't just give a carte blanche "everything is conscious" like panpsychism.
adatavizguy 6y ago
I wonder what Bertrand Russell would say about that?
jameslevy 6y ago
A rock seems unlikely to be conscious in any real way. Something as complex as a star, however...maybe all of the humans worshipping the sun throughout many thousands of years were onto something after all.
- Trasmatta 6y ago
  Is a star particularly complex, though? Is there much going on besides a lot of thermonuclear fusion? (Disclaimer: I don't know much about stars or astrophysics, so this question is coming from a place of ignorance.)
  - hoseja 6y ago
    Yes, incredibly. Magnetohydrodynamics in a star are incomprehensibly complex.
- jbay 6y ago
  A rock the size of a human head (or, say, a container of seawater) has just as many electrons, protons, neutrons, photons, and other particles going about their business and doing their little dance, following the same equations of motion as our brains do. They're certainly doing it in different ways, but if you're just looking for complexity, there's no shortage of it. The rock might lack some macroscopic structural change over human timescales, but definitely not the seawater. I think if you take the perspective that the human brain is conscious but not a brain-sized container of seawater, you need to then look carefully for distinctions between them. "Information processing" or "response to environment" is probably not good enough; the seawater is actually doing all of those, with a unique reaction to any possible input, so you'd have to be more specific. Probably the only recourse you could look for to make the distinction is to say the brain embeds particular mathematical patterns that the seawater doesn't, such as a compact, stored representation of its environment (or its history of inputs), or a future-looking planning algorithm, or both. I personally take this view (I think of qualia, like "the appearance of a red apple", is just precisely what it feels like to read from the buffered [R,G,B] memory array in my head, filtered through image-recognition networks). But then if you put your hopes on consciousness originating from those mathematical functions, you have to admit that any analogous expression of those functions in other systems would also be conscious, such as animals and robots. And worse, once you start thinking about math and how flexible it is, how information is in the eye of the beholder and almost any system that follows certain rules can embed almost any mathematical computation, just like illegible scratches to me are information-rich writing to you, you might have to circle back and that there could be very analogous computations going on inside rocks and seawater. And that brings us back full-circle.
toohotatopic 6y ago
I like that train of thought: The question in another form: Why are we conscious at all? It doesn't make sense. Our arms or legs are not conscious, or are they? The Information is transported into the brain, into a conscious centre. What are its essential building blocks? Why should some neurons in the brain be conscious and everything else be unconscious? If it is not neurons, what else can it be? Atoms? Electric Fields? Gravity? If it is just information processing, and I don't see why a simulation doesn't have an Integrated Information Number, then a Van Neumann machine, created by moving rocks in a desert, would be conscious. So, if it is not intelligence, what else is it? What are the building blocks of a raw conscious machine? One that does no information processing but instead is just aware. It could be a stone. As OP suggests, they just can't move their lips to scream. But then again, why is there just one consciousness in my body? Long question short: HN, what are your explanations for being conscious?

Trasmatta 6y ago

I don't really buy into panpsychism, but it is impossible to disprove. If things like rocks were conscious in some way, I don't think there would be any suffering present. Our suffering is intrinsically tied to our physical and mental systems. "What it's like to be a rock" would be something completely alien to our experience, and without the survival based mechanisms that we have, likely to be devoid of anything like suffering. An AI, on the other hand, could be an entirely different story.

enchiridion 6y ago

Can you elaborate?

Trasmatta 6y ago
Take a look at panpsychism. It's interesting, but I've never been sold on it. Integrated information theory (like this article talks about) is more palatable for me. It allows for the possibility of consciousness of certain non biological systems, but doesn't just give a carte blanche "everything is conscious" like panpsychism.

adatavizguy 6y ago

I wonder what Bertrand Russell would say about that?

jameslevy 6y ago

A rock seems unlikely to be conscious in any real way. Something as complex as a star, however...maybe all of the humans worshipping the sun throughout many thousands of years were onto something after all.

Trasmatta 6y ago
Is a star particularly complex, though? Is there much going on besides a lot of thermonuclear fusion? (Disclaimer: I don't know much about stars or astrophysics, so this question is coming from a place of ignorance.)
- hoseja 6y ago
  Yes, incredibly. Magnetohydrodynamics in a star are incomprehensibly complex.
jbay 6y ago
A rock the size of a human head (or, say, a container of seawater) has just as many electrons, protons, neutrons, photons, and other particles going about their business and doing their little dance, following the same equations of motion as our brains do. They're certainly doing it in different ways, but if you're just looking for complexity, there's no shortage of it. The rock might lack some macroscopic structural change over human timescales, but definitely not the seawater. I think if you take the perspective that the human brain is conscious but not a brain-sized container of seawater, you need to then look carefully for distinctions between them. "Information processing" or "response to environment" is probably not good enough; the seawater is actually doing all of those, with a unique reaction to any possible input, so you'd have to be more specific. Probably the only recourse you could look for to make the distinction is to say the brain embeds particular mathematical patterns that the seawater doesn't, such as a compact, stored representation of its environment (or its history of inputs), or a future-looking planning algorithm, or both. I personally take this view (I think of qualia, like "the appearance of a red apple", is just precisely what it feels like to read from the buffered [R,G,B] memory array in my head, filtered through image-recognition networks). But then if you put your hopes on consciousness originating from those mathematical functions, you have to admit that any analogous expression of those functions in other systems would also be conscious, such as animals and robots. And worse, once you start thinking about math and how flexible it is, how information is in the eye of the beholder and almost any system that follows certain rules can embed almost any mathematical computation, just like illegible scratches to me are information-rich writing to you, you might have to circle back and that there could be very analogous computations going on inside rocks and seawater. And that brings us back full-circle.

toohotatopic 6y ago

I like that train of thought: The question in another form: Why are we conscious at all? It doesn't make sense. Our arms or legs are not conscious, or are they? The Information is transported into the brain, into a conscious centre. What are its essential building blocks? Why should some neurons in the brain be conscious and everything else be unconscious? If it is not neurons, what else can it be? Atoms? Electric Fields? Gravity? If it is just information processing, and I don't see why a simulation doesn't have an Integrated Information Number, then a Van Neumann machine, created by moving rocks in a desert, would be conscious. So, if it is not intelligence, what else is it? What are the building blocks of a raw conscious machine? One that does no information processing but instead is just aware. It could be a stone. As OP suggests, they just can't move their lips to scream. But then again, why is there just one consciousness in my body? Long question short: HN, what are your explanations for being conscious?