37 ms·
> [...] I can't help but feel it's an a really old take [...] To be fair the article is from two years ago, which when talking about LLMs in this age arguably
by lxgr 1y ago
> [...] I can't help but feel it's an a really old take [...]
To be fair the article is from two years ago, which when talking about LLMs in this age arguably does count as "old", maybe even "really old".
- lostmsu 1y agoI think GPT-2 (2019) was already strong enough argument for possibility of modeling knowledge and language that Chomsky rejected.
- gf000 1y agoThough given that LLMs fundamentally can't know whether they know something or not (without a later pass of fine-tuning on what they should know) is a pretty good argument against them being good knowledge bases.
- lostmsu 1y agoNo, it is not. In mathematical limit this applies to literally everything. In practice you are not going to store video compressed with a lossless codec, for example.
- gf000 1y agoMe forgetting/never having "recorded" what necklace the other person had during an important event is not at all similar to a statistical text-generation. If they ask me the previous question I can retrospect/query my memory and tell 100% whether I know it or not - lossy compression aside. An LLM will just reply based on how likely a yes answer is with no regards to having that knowledge or not.
- lostmsu 1y agoYou obviously forgot you previously heard about false memories and/or never thought that happens to you (would be v. ironic).