2 ms·
> Can you memorize “War and Peace in one sitting - neither can an LLM, unless you weaken it to "well I meant input that fits in context", not memorize > precis
by refulgentis 2y ago
> Can you memorize “War and Peace in one sitting - neither can an LLM, unless you weaken it to "well I meant input that fits in context", not memorize
> precisely recall things from it on demand - "precisely" is load-bearing here, and the question at hand. (no, LLMs don't have perfect or precise recall, even from in-context material, and it's not particularly close)
> translate Farsi into Hawaiian and then speak it - 0 models, is this supposed to be a gpt-4o reference? What does this have to do with "perfect recall?" Let's pretend it does. Are you claiming that gpt-4o has a 100% success rate translating Farsi to Hawaiian?
> Can you give instant, and nearly always correct to almost any textbook question - sobs in just spent 4 days benchmarking. For the record, best non-RAG I have is gpt-4o at 86% on MedQA and LegalBench. With RAG, 93%. RAG gpt-4o just barely scrapes by my cofounders USMLE scores. It's excellent. But it's not superhuman.
> All of that is recall - no, translation is not recall
- imtringued 2y agoYou're quoting yourself. It's hard to understand which quotes are yours or not.
- ein0p 2y agoIronically, a failed recall issue. They forgot they needed to add newlines in between. :-)
- ein0p 2y agoBut you see, while some of those things aren’t _just_ recall, they are recall, too. Just different types of it, rather than point recall. And a modern SOTA LLM can accurately recall _vastly more_ data than any human or even any group of humans. Transformers are quite literally very fancy associative memories.