9 ms·
"Why would a language model do anything other than "hallucinate" (i.e. generate words without any care about truthiness) ?" That's exactly the question the pap
by not2b 2y ago
"Why would a language model do anything other than "hallucinate" (i.e. generate words without any care about truthiness) ?"
That's exactly the question the paper attempts to answer: why do LLMs ever get it right? The answer is that on topics where there's a lot of data and a general consensus on what the right answer is, the statistical model will find that answer, and otherwise you get junk. That's why they work so well for people trying to write Python or Javascript, for example.
But I already knew this, you might say. Sure, but the authors produced evidence to back it up.
- jrm4 2y agoThere's really no "right or wrong" per se -- the question that's really being asked is "to what extent does it resonate with a person?"
- danielbln 2y agoAs context sizes grow, it's easier to add lots of information outside of training data via in-context learning, which should offset that issue quite a bit.