4 ms·

I don’t understand what you consider self-awareness to be in this trivial sense. Is Eliza self-aware for example? Eliza maintains a dialogue albeit not obviousl

I don’t understand what you consider self-awareness to be in this trivial sense. Is Eliza self-aware for example? Eliza maintains a dialogue albeit not obviously as coherent as a modern LLM. From my perspective, we know what the benchmark is for our own self-awareness, because Decartes gave it to us: Cogito ergo sum. I think therefore I am. We know we think, so we know we exist. That is the root of our self-awareness. There is a great deal of controversy about the question of whether any of the existing models think at all (and of course that’s the whole point of Turing’s amazing paper[1], and the Chinese room thought experiment[2]) and the best you could say is the burden at the moment is on the people who say models can think to prove that. Given that, I really don’t see how you can say models have self-awareness at the moment. Models may hypothetically be able to convince themselves they are self-aware via Decartes’ method but notice that Decartes’ proof doesn’t work for us - he was able to pull himself up by his own bootstraps because he knew there was a thought so there must be an “I” who was doing the thinking. We have to observe models from the outside and determine whether or not thought is present, and that’s where the concept behind the Chinese room shows how tricky this is. [1] Computing Machinery and Intelligence https://academic.oup.com/mind/article/LIX/236/433/986238 https://academic.oup.com/mind/article/LIX/236/433/986238 [2] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room

killerstorm 2y ago
The only kind of intelligence we know of since childhood is meat-based intelligence. Brain made out of flesh. So our intuition tells us that flesh is essential. "Chinese room" appeals to this intuition. But it's a circular argument. Anyway, I believe they are self-aware, to some extent. Not like a human. But I have no arsenal to convince people who believe intelligence has to be made out of meat.
- seanhunter 2y ago
  I certainly don’t think intelligence has to be made out of meat but I do think it’s a complex set of questions and we owe it to ourselves to actually try to tackle the underlying issue of what intelligence and self-awareness etc really are and how we might know whether a model did or did not exhibit these, because we are apt to fool ourselves on both sides of this argument depending on our prior beliefs. You can see the same thing when people talk about whether animals exhibit self-awareness. There are experiments with dolphins and mirrors for example that definitely suggest that dolphins recognise and might even be amused by their reflection when they see it, but some people find it very hard to reconcile themselves to the idea tgat a dolphin might have a sense of self. I personally find it harder to believe that any particular characteristic would be uniquely human.

killerstorm 2y ago

The only kind of intelligence we know of since childhood is meat-based intelligence. Brain made out of flesh. So our intuition tells us that flesh is essential. "Chinese room" appeals to this intuition. But it's a circular argument. Anyway, I believe they are self-aware, to some extent. Not like a human. But I have no arsenal to convince people who believe intelligence has to be made out of meat.

seanhunter 2y ago
I certainly don’t think intelligence has to be made out of meat but I do think it’s a complex set of questions and we owe it to ourselves to actually try to tackle the underlying issue of what intelligence and self-awareness etc really are and how we might know whether a model did or did not exhibit these, because we are apt to fool ourselves on both sides of this argument depending on our prior beliefs. You can see the same thing when people talk about whether animals exhibit self-awareness. There are experiments with dolphins and mirrors for example that definitely suggest that dolphins recognise and might even be amused by their reflection when they see it, but some people find it very hard to reconcile themselves to the idea tgat a dolphin might have a sense of self. I personally find it harder to believe that any particular characteristic would be uniquely human.