3 ms·
I don't pretend to have a deep understanding of inner workings of LLMs, but this is a "great" illustration that LLMs are not "truth models" but "statistical mod
by zvmaz 3y ago
I don't pretend to have a deep understanding of inner workings of LLMs, but this is a "great" illustration that LLMs are not "truth models" but "statistical models".
- dkarras 3y agoyes, but is there truth without statistics? what is a "truth model" to begin with? can you be convinced of any truth without having a statistical basis? some argue that we all act due to what we experience (which forms the statistical basis of our beliefs) - but proper stats is very expensive to compute (for the human brain) so we take shortcuts with heuristics. those shortcuts are where all the logical fallacies, reasoning errors etc. come from. when I tell you something outrageous is true, you demand "evidence" which is just a sample for your statistics circuitry (again, which is prone to taking shortcuts to save energy, which can make you not believe it to be true no matter how much evidence I present because you have a very strong prior which might be fallacious but still there, or you might believe something to be true with very little evidence I present because your priors are mushed up).
- fl7305 3y ago> LLMs are not "truth models" You could write a piece of software that is a truth model when it operates correctly. But increase the CPU temperature too far, and you software will start spewing out garbage too. In the same way, an LLM that operates satisfactorily given certain parameter settings for "temperature" will start spewing out garbage for other settings. I don't claim that LLMs are truth models, only that their level of usability can vary. The glitch here doesn't mean that they are inherently unusable.
- astrange 3y agoPeople also behave this way; the high temperature hallucinations are called fever dreams.