4 ms·
Absolutely false, at the core of every LLM is a highly compressed text corpus from an Internet search engine. (The wonder here isn't that an LLM succeeds at te
by otabdeveloper4 3y ago
Absolutely false, at the core of every LLM is a highly compressed text corpus from an Internet search engine.
(The wonder here isn't that an LLM succeeds at text retrieval tasks, the wonder is how highly compressed the index turns out to be. But maybe we just severely overestimate our own information complexity.)
- deleted 3y ago[deleted]
- jzombie 3y agoSo, you're saying an LLM is a just a database that does text retrieval?
- otabdeveloper4 3y agoYes, using a statistical model which is in effect a very lossy compressor.
- ProllyInfamous 3y agoSo, like a human, then?
- jzombie 3y agoSo, what you're telling me is that every thing they say has already been said before, completely verbatim? Like, if I asked it to write a story about a dog named Jebediah surfing to planet Xbajahabvash, it would basically just find a link to someone else's story that wrote about the same dog surfing to the same planet? That sounds like an infinitely large amount of combinations. Perhaps the internet is just infinitely large, squared (or even circled).