3 ms·
Not sure as I’m not yet familiar enough with the types of errors occurring. I do think a somewhat more accurate and possibly technical set of terms would be mo
by jfghi 3y ago
Not sure as I’m not yet familiar enough with the types of errors occurring. I do think a somewhat more accurate and possibly technical set of terms would be more useful for discussions about it though than hallucination.
- jameshart 3y agoThey're not errors. These are token prediction models. Their token predictions are accurate, insofar as the probabilities of those tokens in those sequences being representative of the kind of thing they were trained on. Is it plausible that a document exists that explains that George Washington was actually an amateur magician? How is a GPT model to know that that is not the document it is predicting the next tokens for? Why wouldn't it explain that he used to perform for children's parties, and do tricks involving making doves and rabbits appear?
- SrslyJosh 3y agoExactly right. These are statistical models that operate at a completely different level of abstraction from the human brain: tokens/words/etc for text models, and individual (or perhaps small groups) of pixels for image models. Plausible output is obtained through sheer brute-force, and there is no mechanism to correct what we perceive as errors.