4 ms·
> Because they were trained on internet arguments, and nobody on the internet has ever admitted they don't know something Or people’s questions were left unans
by josteink 2y ago
> Because they were trained on internet arguments, and nobody on the internet has ever admitted they don't know something
Or people’s questions were left unanswered (because nobody knew), and there was nothing for the LLM to learn there.
So I guess LLMs can’t learn the absence of answers?
- bdhcuidbebe 2y agoOr, the ai is not thinking at all, just predicting. RTFA
- josteink 2y agoThat’s my point. In a unanswered question, there is nothing to predict (ie «learn» or «think» which I never said), so the AI/LLM will predict using a different source which has responded something, correctness or relevancy be damned. What’s with your tone btw? RTFA? To me that feels quite unwarranted and in violation of the site guidelines. Chill out, eh?
- dongecko 2y agoYes, they can. Meta described a clever way in their paper on training Llama3 [1] (in section about factuality). The idea is to sample several answers to a question that you know the answer to. Let an LLM decide if the given answers are different from your known truth. If so, you found a question, that you can train your LLM in the next post-training round to answer with "I don't know". Do that a couple hundred of times and your LLM will identify neurons that indicate doubt and from here on have the ability to answer with "I don't know". [Edit] The article here also mentions a paper [2] that comes up with the idea of an uncertainty token. So here the incorporation of uncertainty is already baked in at pre-training.[/Edit] [1] https://arxiv.org/pdf/2407.21783 https://arxiv.org/pdf/2407.21783 [2] https://openreview.net/pdf?id=Wc0vlQuoLb https://openreview.net/pdf?id=Wc0vlQuoLb