3 ms·
The LLM doesn't "know" anything, can't reason about its own knowledge, and has no self-awareness. It has training data, and it can use your prompts to synthesiz
by danaris 2mo ago
The LLM doesn't "know" anything, can't reason about its own knowledge, and has no self-awareness. It has training data, and it can use your prompts to synthesize that training data into probable continuations or responses.
If the training data doesn't include lots of text of people being asked questions and saying "I don't know", then it's unlikely to respond "I don't know" when prompted, regardless of whether anything in its training data that might actually answer your question.
- WarmWash 2mo agoDid you just make up in your head how LLMs work and then just run with it? There are a zillion explainers out there, at least put in some time before talking about it. Humans run on training data too in the same broad sense.
- red75prime 2mo ago> If the training data doesn't include lots of text of people being asked questions and saying "I don't know", then it's unlikely to respond "I don't know" There's a problem with your reasoning. The above applies to a foundational model (that is to an autoregressively pretrained model). If the training data doesn't contain "I don't know" in a dialogue context, then, indeed, the model is extremely unlikely to output "I don't know" when asked. That's the nature of autoregressive training. But we are dealing with fine-tuned, RLHF-, intruction-, RL-trained models. If the model has "grasped" a concept of knowledge, this concept can be elicited during the mentioned training.