> I wonder in LLMs either need to be rethought/retrained to say "I don't know" instead of guessing (I have no clue what would go into this) You can already do

> I wonder in LLMs either need to be rethought/retrained to say "I don't know" instead of guessing (I have no clue what would go into this) You can already do this yourself to some extent by providing examples in the prompt. Ask your question, explicitly add that you're allowing the "don't know" answer and add that for this similar question the answer is X, for that the answer is don't know, for that the answer is Y. It's a bit of work, but it works in practice. Check out the concept of priming. Yeah, it would be nice if this was pre-trained - maybe in the future.