3 ms·
but nobody is using LLMs all by themselves. Long-term memory is stored outside the model. In fact, Andrej Karpathy recently talked about the idea that it would
by danielmarkbruce 11mo ago
but nobody is using LLMs all by themselves.
Long-term memory is stored outside the model. In fact, Andrej Karpathy recently talked about the idea that it would be great if we could get LLMs to not know any facts, and that humans poor memory might be a feature which helps with generalization rather than a bug.
- n0w 11mo agoThis is an interesting idea. I wonder if it's more that we have different "levels" of memory instead of generally "poor" memory though. I'm reminded of an article on the front page recently about the use of bloom filters for search. Would something like a bloom filter per-topic make it easier to link seemingly unrelated ideas?
- danielmarkbruce 11mo agoThe details of the search are just details. The way current "LLM Apps" like chatgpt work are just fine. They access material from various places, put it in the context, the LLM reacts.