It’s a neat approach. VRAM is obviously the major concern here since it sounds like the parameter count grows as you insert more facts. I’m also curious how we

It’s a neat approach. VRAM is obviously the major concern here since it sounds like the parameter count grows as you insert more facts. I’m also curious how well the facts communicate with each other. A major problem with RAG is that the model can’t draw novel inferences between documents. You have a model with a broad knowledge base but a very shallow understanding of the connections between datapoints. Let’s say I insert two facts separately using this method: “the cookie is under cup A” and “cup A is moved to position 2”. If I ask where the cookie is can this new model tell me? This is trivial for in-context learning and non-trivial for RAG. Curious where this solution falls.