4 ms·
All the podcasts I've been listening to recommend RAG over fine-tuning. My intuition is that having the relevant knowledge in the context rather than the weight
by firejake308 3y ago
All the podcasts I've been listening to recommend RAG over fine-tuning. My intuition is that having the relevant knowledge in the context rather than the weights brings it closer to the outputs, thereby making it much more likely to provide accurate information and avoid hallucinations/confabulations.
- benjaminwootton 3y agoDo you have any podcasts you would reccomend with this type of content?
- firejake308 3y agoI listen to Data Skeptic and Practical AI. It's interesting content, and I find Practical AI has guests who are particularly relevant to what I want to learn, but you should keep in mind that the guests are usually trying to promote their own products, so everything they say should be taken with a grain of salt
- zmmmmm 3y ago> All the podcasts I've been listening to recommend RAG over fine-tuning I'm always suspicious that is just because RAG is so much more accessible (both compute wise and in terms of expertise required). There's far more profit in selling something accessible to the masses to a lot of people than something only a niche group of users can do. I think most people who do actual fine tuning would still probably then use RAG afterwards ...
- zainhoda 3y agoOne added benefit of RAG is that it's more "pluggable." It's a lot easier to plug into newer LLMs that come out. If and when GPT-5 comes out, it'll be a one character change in your code to start using it and still maintain the same reference corpus.