3 ms·
> RAG is much closer to how the human brain works than fine tuning a model is. Arguably, RAG points to the existence or the beginnings of AGI. I see these two
by janalsncm 3y ago
> RAG is much closer to how the human brain works than fine tuning a model is. Arguably, RAG points to the existence or the beginnings of AGI.
I see these two claims as orthogonal. AGI need not work like the human brain, and I’m not sure people’s brains have any lossless representation of memories like a RAG system would. People are infamously unreliable when it comes to recalling information during crimes for example.
However, it does seem like a RAG system is much more versatile than continuously training. Facts change all the time and it’s infeasible to retrain every time that happens.
- ldjkfkdsjnv 3y agoIsn't RAG analogous to how humans get information during a conversation or searching the internet? Its contextual world information, in the future RAG will also include vision, hearing, sight, the rest of the senses etc. But those are just lower level inputs on what the current language inputs are
- janalsncm 3y agoAt a high level yes? There is some representation of facts in the brain, and the same is true in a language model that uses a knowledge base. But if you look any closer the differences are pretty clear. We don’t store facts as a serialized list in our brains. We have a knowledge graph, and a lossy one at that.