3 ms·
https://archive.ph/XKG86 https://archive.ph/XKG86 Because they were trained on internet arguments, and nobody on the internet has ever admitted they don't know
by do_not_redeem 2y ago
https://archive.ph/XKG86 https://archive.ph/XKG86
Because they were trained on internet arguments, and nobody on the internet has ever admitted they don't know something
- tsumnia 2y agoDon't forget we made fake Internet points to tell people we don't like when someone says "I don't know" as well
- Lalabadie 2y agoAnd human-refined training sets are also selected for the accuracy of their knowledge. Book authors mostly default to assuming they know. There's just comparatively not much reference material for someone correctly self-assessing that they don't know.
- josteink 2y ago> Because they were trained on internet arguments, and nobody on the internet has ever admitted they don't know something Or people’s questions were left unanswered (because nobody knew), and there was nothing for the LLM to learn there. So I guess LLMs can’t learn the absence of answers?
- bdhcuidbebe 2y agoOr, the ai is not thinking at all, just predicting. RTFA
- josteink 2y agoThat’s my point. In a unanswered question, there is nothing to predict (ie «learn» or «think» which I never said), so the AI/LLM will predict using a different source which has responded something, correctness or relevancy be damned. What’s with your tone btw? RTFA? To me that feels quite unwarranted and in violation of the site guidelines. Chill out, eh?
- dongecko 2y agoYes, they can. Meta described a clever way in their paper on training Llama3 [1] (in section about factuality). The idea is to sample several answers to a question that you know the answer to. Let an LLM decide if the given answers are different from your known truth. If so, you found a question, that you can train your LLM in the next post-training round to answer with "I don't know". Do that a couple hundred of times and your LLM will identify neurons that indicate doubt and from here on have the ability to answer with "I don't know". [Edit] The article here also mentions a paper [2] that comes up with the idea of an uncertainty token. So here the incorporation of uncertainty is already baked in at pre-training.[/Edit] [1] https://arxiv.org/pdf/2407.21783 https://arxiv.org/pdf/2407.21783 [2] https://openreview.net/pdf?id=Wc0vlQuoLb https://openreview.net/pdf?id=Wc0vlQuoLb
- CommieBobDole 2y agoAnd on a lot of Reddit and Stack Overflow-type posts. Nobody much replies to a question to say "I don't know".
- Arnt 2y agoI tried Claude now, which gave me a perfectly clear "don't know". "While I aim to be accurate in my responses [...] I think it's best for me to acknowledge that I don't have reliable information about [...] marital status rather than make a potentially incorrect statement." Difficult to have an exciting discussion about such an answer, so I assume most participants on this internet forum will focus on other LLMs ;)
- tim333 2y agoWell, if you don't know the normal thing is just not to post a reply.
- hoc 2y agoThey were trained on forum posts and this one also misses tbe opportunity and instead explains and guesses that... Ah, I don't know.
- thomastjeffery 2y agoThat's not the whole story, though. If it did say "I don't know", it still wouldn't know when it should. Instead, we would get confident expressions of uncertainty; and we would label them hallucinations, too. This actually happens, just not very often.