3 ms·
Sure it's just an analogy. We clearly don't have 3 distinct separate levels, it's more like one memory where recalling memories strengthens them (but can also c
by mike_hearn 3y ago
Sure it's just an analogy. We clearly don't have 3 distinct separate levels, it's more like one memory where recalling memories strengthens them (but can also change them). But whatever we have, it's clearly very different to an LLM.
We don't even want to replicate the human mind exactly. If we did that what'd be the point? AI is useful exactly because it's not the same as how people work.
I don't think RAG is a good analogy for how the brain works because RAG is about finding original concrete sources (like books), not about matching and recalling digested and abstracted thoughts inside the "mind" itself. The article does talk about this a bit at the beginning.
- naasking 3y ago> But whatever we have, it's clearly very different to an LLM. My only real quibble with your post is, again, that it seems different to an LLM. Current ML models are still multiple orders of magnitude away from being on the same order of complexity as the human brain, and scaling doesn't yet seem to have hit a limit (more a data limit). I just don't think we can draw such conclusions at this time.
- mike_hearn 3y agoSure, but what I'm getting at is that even if you scale a GPT-4 style transformer up 1000x, it won't suddenly develop a human-like memory. It will still be read only, whereas there's no such thing as a read only human unless they are unfortunate enough to have severe brain damage.