2 ms·

How is LLM+RAG any different from a fine-tuned LLM without RAG? They're both trained on data, and have the capability to hallucinate.

by imiric 3y ago

michaelt 3y ago
Theoretically some people think RAG sounds more feasible to make factually accurate. After all, if you've trained an LLM on a masses of unchecked data you've scraped from the internet, your training data probably includes "Joe Biden is the president" and "Donald Trump is the president" and "Barrack Obama is the president" and "Emmanuel Macron est le président" and so on. It would be understandable if an LLM was confused about who the president was. These people think handing an LLM the contents of https://en.wikipedia.org/wiki/President_of_the_United_States https://en.wikipedia.org/wiki/President_of_the_United_States then asking who the president is sounds a lot more feasible. Personally I'm not so sure - I've never seen a RAG implementation that impressed me.
- yunwal 3y ago
  The point is that it should be able to point you to the source article. The synthesis isn’t necessarily more accurate, but you can check its work easily.
throwaway4233 3y ago
I believe what the parent comment is trying to imply is that the search results are fetched/retrieved from Bing's own internal ranking vector(?) database and then passed to the LLM, which then converts the received documents into a more human readable format and fills in any missing gaps in the information with it's own data. So the gaps are the only areas where the LLM can hallucinate on and if your search query is easily available information on the internet, then hallucinations will be less or none. Edit: I have used RAG with a project that I am working on and it's quite hard to ascertain if the LLM used the information provided as part of the RAG documents or just made up information on it's own, since even without RAG, we were getting similar responses 7 times out of 10.

How is LLM+RAG any different from a fine-tuned LLM without RAG? They're both trained on data, and have the capability to hallucinate.

by imiric 3y ago

How is LLM+RAG any different from a fine-tuned LLM without RAG? They're both trained on data, and have the capability to hallucinate.

michaelt 3y ago

Theoretically some people think RAG sounds more feasible to make factually accurate. After all, if you've trained an LLM on a masses of unchecked data you've scraped from the internet, your training data probably includes "Joe Biden is the president" and "Donald Trump is the president" and "Barrack Obama is the president" and "Emmanuel Macron est le président" and so on. It would be understandable if an LLM was confused about who the president was. These people think handing an LLM the contents of https://en.wikipedia.org/wiki/President_of_the_United_States https://en.wikipedia.org/wiki/President_of_the_United_States then asking who the president is sounds a lot more feasible. Personally I'm not so sure - I've never seen a RAG implementation that impressed me.

yunwal 3y ago
The point is that it should be able to point you to the source article. The synthesis isn’t necessarily more accurate, but you can check its work easily.

throwaway4233 3y ago

I believe what the parent comment is trying to imply is that the search results are fetched/retrieved from Bing's own internal ranking vector(?) database and then passed to the LLM, which then converts the received documents into a more human readable format and fills in any missing gaps in the information with it's own data. So the gaps are the only areas where the LLM can hallucinate on and if your search query is easily available information on the internet, then hallucinations will be less or none. Edit: I have used RAG with a project that I am working on and it's quite hard to ascertain if the LLM used the information provided as part of the RAG documents or just made up information on it's own, since even without RAG, we were getting similar responses 7 times out of 10.