2 ms·
The key difference is that while an encyclopedia holds facts themselves, LLMs trained on that source material encode something more like a highly probable facsi
by sethhochberg 7d ago
The key difference is that while an encyclopedia holds facts themselves, LLMs trained on that source material encode something more like a highly probable facsimile of those facts - the original fact was lost, LLMs are lossy, but can often be generated again with a decent level of accuracy by churning through stats about words, concepts, and relationships between them.
The whole catch is that they can often be regenerated. But LLMs (on their own, in their parametric memory - which is the result of training) don't have any conception of whether what they've generated is a real reproduction of some training material or whether they've invented something false that seemed probable based on their encoded stats. When the probability produces something contrary to what was in the training material, you get hallucinations.
They're very, very good predictive text models and can be very, very powerful when hooked up to other tools or outside databases. But its fundamentally lossy technology and all the books having been fed in doesn't guarantee all of the knowledge from those books can be spat back out.
- zmgsabst 7d agoYour criticism of LLMs also applies to humans and so implies humans don’t possess knowledge.
- JanNash 7d agoHumans can learn texts by heart, even lots of text (I have some expertise on that, having studied opera singing). They can reproduce these texts accurately, deterministically and repeatably. An LLM is a statistical machine. It does not know any text by heart and it is by pure chance that it sometimes reproduces existing texts verbatim.
- copperx 7d agoAnt tips for doing so? I've never been able to do that, even as a kid. I remember the meaning but not the exact words.
- zdragnar 7d agoBreak it into small chunks and practice. Practice more. Put the chunks together and practice even more. Like any skill, it isn't something people are magically good at beyond minor proficiency. Every performance- singing a song, playing an instrument, performing a stand-up routine, giving a speech, performing a theatrical role are all things that are best done from memory but require practice. There are pneumonic tricks you can use- I've seen some people do it for tricks like memorizing the order of a deck of cards- but it's less useful for long term recital because it helps with order but not comprehension or fast indexing.
- JanNash 6d agoWhat zdragnar says :)
- kjshsh123 7d agoHumans sometimes don't remember things 100%. LLMs can produce certain text and be run deterministically. I don't really understand this side of the debate other than as a gotcha tbh. If LLM use atrophies your brain and skills that's bad. If it has a repulsive writing style that's bad. I'm not sure what the debate about whether an AI is a statistical parrot unlike humans accomplishes. Is relying 100% on a bad human speechwriter somehow better?
- zdragnar 7d agoIt comes up frequently when discussing whether LLMs actually have intelligence or qualify as AI, especially in discussions on the path to achieving AGI. The primary complaint seems to be that LLMs are held to a higher standard than humans, though I don't particularly buy that line of reasoning.