Speaking as software-developers we have to see RAG as kind of a caching-mechanism. Instead of computing every token every time on the whole context, we can gra

Speaking as software-developers we have to see RAG as kind of a caching-mechanism. Instead of computing every token every time on the whole context, we can grab a cache to make some shortcut. We do the same in software development all the time. Of course it's a performance issue.