3 ms·
There is an article at the original site about that: https://www.anyscale.com/blog/fine-tuning-is-for-form-not-facts https://www.anyscale.com/blog/fine-tuning-i
by zby 3y ago
There is an article at the original site about that: https://www.anyscale.com/blog/fine-tuning-is-for-form-not-facts https://www.anyscale.com/blog/fine-tuning-is-for-form-not-fa...
Everybody new to this field thinks that he needs finetuning to teach the LLM of new facts. I made the same mistake initially, later I published a slightly ranty post on that: https://zzbbyy.substack.com/p/why-you-need-rag-not-finetuning https://zzbbyy.substack.com/p/why-you-need-rag-not-finetunin...
- sandGorgon 3y agoQuick question - Gorilla paper talks about finetuning for RAG. Do you see this in practice ? can you do finetuning that specifically affects RAG ?
- zby 3y agoSorry - myself I don't have much experience yet, I am at the research phase, but from what I read it makes sense to finetune the model to better understand the format used for calling the external tools including a search engine.