3 ms·
That's because it's been fine tuned on RLHF data which gives those responses to that kind of question. It says "I" because that's how people talk and it's model
by taneq 3y ago
That's because it's been fine tuned on RLHF data which gives those responses to that kind of question. It says "I" because that's how people talk and it's modeling how people talk. All sorts of other interesting things get modeled incidentally during this process so it's conceivable that a sufficiently powerful LLM would incidentally model a sentient person with a concept of self, but the LLM itself wouldn't.
- doubloon 3y agoso if you have an entity that fakes all the inputs and outputs that would indicate that it has a concept of self, how can you tell the difference between that and a entity that actually does have a sense of self?
- taneq 3y agoIf this entity successfully faked all those outputs then you wouldn't be able to tell the difference, by definition, since if you could tell the difference then they wouldn't be successfully faked. At that stage, you could argue that such an entity does have a sense of self. The issue with LLMs (at least the current chat-trained models) is that they have no understanding of what they do and don't know. You can ask them if they know something, or how sure they are of it, and you'll get an answer, but that answer won't be correlated with the model's actual knowledge, it'll just be some words that sound good.