2 ms·
This seems at least plausible, and it agrees with my preconceived notions that a good chunk of LLM capability is driven by memorization and not computation[0].
by kmeisthax 3y ago
This seems at least plausible, and it agrees with my preconceived notions that a good chunk of LLM capability is driven by memorization and not computation[0]. Is there any substantive critique of the underlying idea from the other reviewers, and not just the (evidently terrible) presentation of it?
[0] For a good idea as to why I think this way, see https://not-just-memorization.github.io/extracting-training-data-from-chatgpt.html https://not-just-memorization.github.io/extracting-training-...