2 ms·
But can it say “I don’t know” if ya know, it doesn’t
by throwfaraway4 1y ago
But can it say “I don’t know” if ya know, it doesn’t
- asadm 1y agothere needs to be a benchmark for this actually.
- tmvphil 1y agoKind of have one with the missing image benchmark: https://openai.com/index/introducing-gpt-5/#more-honest-responses https://openai.com/index/introducing-gpt-5/#more-honest-resp...
- dcchambers 1y agoI agree with the sentiment, but the problem with this question is that LLMs don't "know" *anything*, and they don't actually "know" how to answer a question like this. It's just statistical text generation. There is *no actual knowledge*.
- AnimalMuppet 1y agoTrue, but I still think it could be done, within the LLM model. It's just generating the next token for what's within the context window. There are various options with various probabilities. If none of the probabilities are above a threshold, say "I don't know", because there's nothing in the training data that tells you what to say there. Is that good enough? "I don't know." I suspect the answer is, "No, but it's closer than what we're doing now."
- m4nu3l 1y agoIt still got it wrong in the very first answer, as I mentioned in my top-level comment.