3 ms·
It seems plausible that stressing the importance of the system prompt instructions might do something, but I don't see how telling the model not to hallucinate
by jsheard 2y ago
It seems plausible that stressing the importance of the system prompt instructions might do something, but I don't see how telling the model not to hallucinate would work. How could the model know that its most likely prediction has gone off the rails, without any external point of reference?
- jshmrsn 2y agoSome of the text that the LLM is trained on is fictional, some of the text that its trained on is factual. Telling it to not make things up can tell it to generate text that’s more like the factual text. Not saying it does work, but this is a reason how it might work.
- viraptor 2y agoThe model can be trained to interpret "don't hallucinate" as "refer only to the provided context and known facts, do not guess or extrapolate new information", which wouldn't get rid of the issue completely, but likely would improve the quality if that's what you're after and if there's enough training data for "I don't know" responses. (But it all depends on the fine-tuning they did, so who knows, maybe it's just an Easter egg)
- famouswaffles 2y agoInternally, LLMs know a whole lot more about the truth and uncertainty of their prediction than the say. Pushing that to words is difficult but not impossible. https://news.ycombinator.com/item?id=41504226 https://news.ycombinator.com/item?id=41504226
- potatoman22 2y agoI think it's more likely that it's included for liability reasons.