3 ms·

have you ever used or spoken to GPT2 or GPT3? Not chatgpt. The one before they did any RLHF, to train it to respond a certain way. If you asked it whether it wo

have you ever used or spoken to GPT2 or GPT3? Not chatgpt. The one before they did any RLHF, to train it to respond a certain way. If you asked it whether it would like to be hurt, it would beg you not to. If you asked it to move it's leg out of the way, it would apologize, and claim it obliged. It would claim to be conscious, to be aware. Of course, these statement do not come from a place of considered action: everybody knows the machine is not conscious in the same way as a human, but the point is that an unfeeling machine even being able to make such claims makes us have to move the "consciousness" divider further and further back into the shadows, until it's just some nebulous vibe people insist must be somewhere. It's possible there is a clear and unambiguous way to define humans and intelligent animals conscious, but nobody has come up with a workable definition yet that lets us neatly divide them. Another slight thing that gives me a little pause: you know how great our brain is at confabulating right? Have you ever done something, then had someone ask you why you did it? You generally tell them a story about how you thought about doing something, weighed the pros and cons etc, that isn't actually true when you think deep down. We like to think we are a single being that thinks carefully about everything it does. instead we're more like an explaining machine sitting on top of a big pile of confusing processes and making up stories why the processes do what they do. How exactly this last thought relates to the discussion I haven't figured out yet, its just something that comes to mind :)

staticman2 3y ago
I guess I'll just say that it's not obvious that language has much to do with consciousness, so it's not obvious that a language model has moved things into the shadows. Like, maybe we're in the shadows, but I don't think you can blame GPT 3 for that. In 1974 philosopher Thomas Nagel wrote, "Conscious experience is a widespread phenomenon. It occurs at many levels of animal life, though we cannot be sure of its presence in the simpler organisms, and it is very difficult to say in general what provides evidence of it. (Some extremists have been prepared to deny it even of mammals other than man.) No doubt it occurs in countless forms totally unimaginable to us, on other planets in other solar systems throughout the universe. But no matter how the form may vary, the fact that an organism has conscious experience at all means, basically, that there is something it is like to be that organism." He's clearly willing to recognize consciousness to non verbal creatures.
- RugnirViking 3y ago
  The point is, how can you tell if a being is conciousness. We can start with the idea that we think these models aren't (I think they aren't!) A lot of your arguments seem to be based in asking questions of the being, whether it can experience things. But seeing as we can't read minds, we can't tell whether it actually believes or experiences what it is saying. So we have (had) to trust that a creature capable of explaining such things is concious. Now we can't do that, hence the dividing line that we feel must exist now moves into some unobservable territory. I'm not commenting on whether such a line exists, just that it's kinda hard to test right now, so any argument about it has to go into hypotheticals and philosophy

staticman2 3y ago

I guess I'll just say that it's not obvious that language has much to do with consciousness, so it's not obvious that a language model has moved things into the shadows. Like, maybe we're in the shadows, but I don't think you can blame GPT 3 for that. In 1974 philosopher Thomas Nagel wrote, "Conscious experience is a widespread phenomenon. It occurs at many levels of animal life, though we cannot be sure of its presence in the simpler organisms, and it is very difficult to say in general what provides evidence of it. (Some extremists have been prepared to deny it even of mammals other than man.) No doubt it occurs in countless forms totally unimaginable to us, on other planets in other solar systems throughout the universe. But no matter how the form may vary, the fact that an organism has conscious experience at all means, basically, that there is something it is like to be that organism." He's clearly willing to recognize consciousness to non verbal creatures.

RugnirViking 3y ago
The point is, how can you tell if a being is conciousness. We can start with the idea that we think these models aren't (I think they aren't!) A lot of your arguments seem to be based in asking questions of the being, whether it can experience things. But seeing as we can't read minds, we can't tell whether it actually believes or experiences what it is saying. So we have (had) to trust that a creature capable of explaining such things is concious. Now we can't do that, hence the dividing line that we feel must exist now moves into some unobservable territory. I'm not commenting on whether such a line exists, just that it's kinda hard to test right now, so any argument about it has to go into hypotheticals and philosophy