> However, if this is even partially correct I'd like to understand how the "formed memory" and the base model cohabitate during inference, because this would c

> However, if this is even partially correct I'd like to understand how the "formed memory" and the base model cohabitate during inference, because this would create memory pressure on the GPU. Not really. RAG loads selected data into the neural network which changes the state of the existing "neurons" (aka parameters), so the memory usage on GPU is only the size of the neural network. You will hear about "context size" a lot. This means the amount of tokens a particular model can have loaded without becoming saturated and starting to lose things that were previously loaded.