3 ms·
I really don’t get this sentiment - why not do both? Retrieval allows looking up facts - eg in a Google search Finetuning allows reasoning using new knowledge
by gdiamos 3y ago
I really don’t get this sentiment - why not do both?
Retrieval allows looking up facts - eg in a Google search
Finetuning allows reasoning using new knowledge.
Humans do both.
- swalsh 3y agoI think fine tuning makes sense when you need some domain specific knowledge to properly read, analyze, and interpret the information you're passing to it. But its not an information store itself. The most valuable skill an LLM can have is good reasoning skills and a broad enough knowledge base to understand. From there you can pass it the important bits it needs.
- gdiamos 3y agoI think we are saying the same thing. The key ingredients are: reasoning(skills) + knowledge + important bits/facts The best systems have all of these