3 ms·
> LLMs seem to operate on the principle of "will this pass for a correct answer" rather than "this is how this thing works, so here's a reasonable opinion answe
by 3cats-in-a-coat 3y ago
> LLMs seem to operate on the principle of "will this pass for a correct answer" rather than "this is how this thing works, so here's a reasonable opinion answering your question".
That's NOT how LLMs operate. They're trained on what is supposed to be selected representative human text of some at least minimum quality, and ideally truthful, factual and relevant.
And then their prediction is judged on how close to that ground truth it is. Not on whether it's "plausible" or "will this pass for a correct answer". There's no math function for pretending to be right. You have to be right.
One reason why LLMs make up things sometimes is because they're incredibly tightly optimized. Like you have no idea how tight. GPT-4 has the comparable complexity of a mouse brain. A mouse brain that has to fit the world's knowledge.
Obviously, something has to give, and you'll not be accurate sometimes when you have so limited space. You'll store some things approximately, generally, and then extrapolate answers from them, maximizing for correct output.
- lazy_moderator1 3y agothis is an interesting statement, are you involved in LLM research?
- ethanbond 3y ago> And then their prediction is judged on how close to that ground truth it is When/where/how?
- SAI_Peregrinus 3y agoExtant LLMs are trained on whatever data the trainers can get their hands on. There's no magical "check for truth" step, either at the input or at the output.
- 3cats-in-a-coat 3y agoNo, data is significantly filtered in order to bias it to accuracy. Of course if you feed an AI scientific papers and books of some quality, information will be of high quality. THAT... is how you "filter for truth". It's not supposed or required to be perfect. But of course you can filter the input. The fact you also add conversational input from various sources like Reddit and Twitter doesn't change this balance significantly. Also even those sources are filtered to exclude vulgarities, certain topics and so on. Also you're not thinking this through. If a question is on average answered incorrectly 'round the world, it means you're over 50% likely to also give the incorrect answer when asked. Who equipped you to know the truth?
- bsenftner 3y agoDespite the group think down vote, you are correct. However, a problem arises from using generalized web content as the ground truth training data. That's a problem because the generalized web is chock full of political propaganda, marketing exaggerations, and real people parroting that with partial and complete misunderstandings of it all, with others of the same incorrect understanding. Perhaps in the future, the training data will be more carefully curated to eliminate as many 'wishful truths that do not exist' and 'misinformed conversations between parties with no correct voice' as possible.
- 3cats-in-a-coat 3y agoThanks for your comment of support. Ironically the "groupthink" is hallucinating explanations about how LLMs work, irony. Anyway, yes, the "web" in general is full of garbage and propaganda. But then again we're exposed to the same web, and without the pre-processing which OpenAI etc. apply to filter out certain words, phrases, topics (including lots of manual human labor). So it's funny how we're living on that same input, but we're so confident we know what's true and what's wrong. But LLMs aren't just trained on literally anything on the web. For example consider Wikipedia. That's not one's average drivel for sure. And next to the general web, you need higher quality sources of data in the mix to bias the AI for truth. Scientific papers, good books, transcripts from conferences and presentations. Wikipedia to mention again. So on. Of course, sometimes LLM make technical mistakes which you can clearly rule as incorrect. Say, plumbus.com's API never had a method getDescription(). But the AI saw many APIs of services like this, which have API getDescription() and given its small network, it's forced to fill-in gaps by inferring the answer based on a vast assortment of contextual clues. Which we call hallucinations (incorrectly). But even the correct answers are inferred. It's all inference. And we all think by inference. We just have 50 times more capacity than the biggest LLM at the moment, so our inference tends to be much more sophisticated, and therefore more likely to be correct. I mean in theory. After all, it's humans and their big brains who produced the garbage that's the web, so.
- bsenftner 3y agoI agree with your points, but would also like to point out humans are not exposed to the general web as infants, and only a curated exposure until they become smart enough get through the various age restrictions placed weakly here and there. Children's initial years, being basically lied to about the nature of the adult world, is where their sophisticated inference comes from. The internal intellectual conflict when one realizes it is actually good to tell children the world is a nice, fair, happy place - once figuring out it is not. That right there, I theorize, is where our sophisticated inference is created: we realize our own dynamic minds. We realize the necessity to seat concepts of "a fair world" into people before allowing them to mentally mature and learn it is not.