3 ms·
You've a point. Doesn't look like RAG (w/ in-context learning) or fine-tuning add domain knowledge to the LLM, so some claim they're equivalent: https://twitter
by ignoramous 3y ago
You've a point. Doesn't look like RAG (w/ in-context learning) or fine-tuning add domain knowledge to the LLM, so some claim they're equivalent: https://twitter.com/Shahules786/status/1748059074556760421 https://twitter.com/Shahules786/status/1748059074556760421 / https://archive.is/iRN5j https://archive.is/iRN5j
Proxy-tuning (https://twitter.com/rasbt/status/1748021765790376385 https://twitter.com/rasbt/status/1748021765790376385 / https://archive.is/oQs0m https://archive.is/oQs0m) and other such merged models(https://twitter.com/osanseviero/status/1745121420353454219 https://twitter.com/osanseviero/status/1745121420353454219 / https://archive.is/hFYbh https://archive.is/hFYbh) are an interesting area of study, too