This is a subjective and semi-informed opinion, but I don't really see that RAG is "a hack". The value of LLMs isn't their ability to regurgitate information.

This is a subjective and semi-informed opinion, but I don't really see that RAG is "a hack". The value of LLMs isn't their ability to regurgitate information. Computers could do that already, with databases, or even flat files. As I see it, the value of LLMs is their ability to interact with human language and abstract concepts. There are situations where you have some limited amount of relatively static data you want to add to your model. In those situations you'll get value from fine tuning the model. But if you have a large set of data to add, or data that updates frequently, then a model that understands human language and basic logical reasoning hooked into a RAG makes much more sense. That strikes me as an elegant solution. I don't see that as "a hack" at all.