I spent a bit of time playing with h2ogpt, which is a popular RAG framework. I gave it all our architectural documentation for our software and then tried askin

I spent a bit of time playing with h2ogpt, which is a popular RAG framework. I gave it all our architectural documentation for our software and then tried asking it questions that transcended basic search (so the answer is not directly in there, but you could formulate it if you put disparate parts together). It started hallucinating pretty fast and told me all kinds of BS about how our software works. I think RAG can be used in a way that eliminates or drastically reduces hallucinations, but to do that you have to do quite a lot of work to constrain the context and structure the prompting to address very specific questions. When you apply these more general frameworks they pump in large amounts of context in an unstructured manner and you just end right back at hallucinations again because the context isn't constrained enough. So RAG is useful to me but not a silver bullet. It doesn't solve the original problem of wanting all the features of an LLM but without the hallucinations. It gives you some targeted way to use the LLM that doesn't hallucinate but misses a lot of the functionality people want.