4 ms·
What is the general thought process on when it makes sense to use RAG vs fine tuning? How does segmenting fine tuning models make sense? Do I need a terraform
by SOLAR_FIELDS 3y ago
What is the general thought process on when it makes sense to use RAG vs fine tuning?
How does segmenting fine tuning models make sense? Do I need a terraform LLM, a SQL LLM, and a python LLM, or can I just use a “code” LLM?
- DebtDeflation 3y agoFine tuning for training the model to perform a new task, RAG for adding knowledge. In your example, you would fine tune the model to train it to code in a language it hasn't seen before, RAG will not really help with that.
- solaxun 3y agoI've read that SFT is good for "leveraging existing knowledge" gained during initial pretraining, and helpful in changing the way that the model responds, but not useful for teaching it new knowledge. In your experience is that true? For example, changing the way in which it responds could be: - debate me - brainstorm - be sarcastic Which also seems like something that could be accomplished with a system prompt or few shot examples, so I'm not sure when SFT is the more appropriate approach or what the tradeoffs are. Alternatively, gaining new knowledge would be training it on a dataset of e.g. sports trivia to make it highly effective at answering those types of questions. P.S. nice username... Irving Fisher would approve.
- jawerty 3y agoI have an RAG video (my "make a ChatGPT with podcasts" video) you might be interested in. Semantic search is increddible and you might be surprised how good a Q/A solution is by just extracting passages that answer the question. Overall it depends on whether or not you can turn your data into a fine-tuning data and if you can find a low parameter (enough) model that can use your found contexts as input to host yourself of use inference endpoints. Hosting an LLM is actually not easy and I'm finding in the field working an information retrieval business OpenAI isn't terrible compared to costs of having a GPUs for your users across the world.
- zby 3y agoThere is an article at the original site about that: https://www.anyscale.com/blog/fine-tuning-is-for-form-not-facts https://www.anyscale.com/blog/fine-tuning-is-for-form-not-fa... Everybody new to this field thinks that he needs finetuning to teach the LLM of new facts. I made the same mistake initially, later I published a slightly ranty post on that: https://zzbbyy.substack.com/p/why-you-need-rag-not-finetuning https://zzbbyy.substack.com/p/why-you-need-rag-not-finetunin...
- sandGorgon 3y agoQuick question - Gorilla paper talks about finetuning for RAG. Do you see this in practice ? can you do finetuning that specifically affects RAG ?
- zby 3y agoSorry - myself I don't have much experience yet, I am at the research phase, but from what I read it makes sense to finetune the model to better understand the format used for calling the external tools including a search engine.