Your two options are RAG or fine-tuning. RAG (with vector db) I've found to be a little finicky, and requires a decent amount of preprocessing of your data in

Your two options are RAG or fine-tuning. RAG (with vector db) I've found to be a little finicky, and requires a decent amount of preprocessing of your data in order to split it up into reasonable chunks that vector encode well. I also think that due to the nature of how RAG works, it might not be able to do certain types of queries unless you allow for a more complicated back-and-forth of querying. But I've only toyed/built some proof-of-concepts with it, nothing production ready. I found llama-index to be useful here; it lets you spin up a barebones RAG system with sensible defaults in like 20 lines of code. But ofc for any real world application you'll have to start making a bunch of mods. Would love feedback from folks who have used RAG in production systems -- was it difficult to split your documents up? Did you have trouble with it using irrelevant chunks from your vector db? Etc. Fine-tuning is modifying an existing llm to have knowledge about your data. It's been on my to try list for a while, but it's got a higher barrier to entry. There are good video tutorials on YouTube though. It seems like this would allow for a more in depth understanding of your documents, making it more likely to answer complicated prompts. But would love feedback on that hypothesis from folks with experience using it!