4 ms·

Using RAG feels like asking an acquaintance to write a book report by giving them semi-randomly cut out paragraphs from the book. Finetuning seems to be a solu

Using RAG feels like asking an acquaintance to write a book report by giving them semi-randomly cut out paragraphs from the book. Finetuning seems to be a solution, but there is still the issue of the LLM not really "learning" facts. What's the latest on actually adding knowledge to an LLM?

filterfiber 3y ago
The most likely "real solution" is going to be using various tricks and finetuning on higher context lengths to just extend the context window. However I will say knowledge graphs are becoming more popular. I made a comment the other day with a list of some of the popular methods: https://news.ycombinator.com/item?id=38476596 https://news.ycombinator.com/item?id=38476596 I completely forgot to mention ROME in my last comment, where you can modify facts within an LLM https://arxiv.org/pdf/2202.05262.pdf https://arxiv.org/pdf/2202.05262.pdf Note that there could be something more recent then these that I missed, but as far as I know knowledge-graphs/RAG are what most people are currently using, but there's a lot of work being focused on extending the context window.
kristjansson 3y ago
This doesn't deserve the downvotes it (currently) has. This is a completely accurate statement, simple RAG works well when questions are highly correlated with specifics chunks of documents. It does not allow an LLM to synthesize an entire corpus to an answer (e.g. a book report). Other modes of RAG that retrieve multiple times per generation (e.g. every so-many tokens) can improve performance, and intuitively feel closer to the 'sparse attention over the entire corpus' that RAG is approximating. However, fine-tuning on relevant, high quality, knowledge-rich question/answer pairs seems dominant, when such examples are available or can be generated.
- filterfiber 3y ago
  I have to agree, RAG is effectively just compensating for data not fitting in the context window, and it has (potentially severe) limitations. Summarizing a book I think is a great example, RAG prevents LLMs from correlating all of the data. It will likely never be obsolete, but a larger context would be far preferable in nearly all scenerios.
- svcrunch 3y ago
  I think your comment is accurate, but regarding your last point: "However, fine-tuning on relevant, high quality, knowledge-rich question/answer pairs seems dominant, when such examples are available or can be generated." How does one solve the problem of access-controlled data, if not through RAG? Do you imagine a separate version of the LLM for every user, reflecting their unique permissions on the data? Also, in scenarios where the data is being updated regularly, RAG provides much lower latency to the new information. Deletes also present a challenge for a pure-LLM approach.
  - willy_k 3y ago
    I don’t know much about the space, but couldn’t LoRA be used to (moderately) efficiently use a relevant and individual fine-tuned model alongside a general LLM?
eskibars 3y ago
Having worked on a (production, as-a-service RAG system), what we've learned is that fine tuning is not good for really adding/removing facts but is great for changing the form of the output. Anyscale has a great blog on this: https://www.anyscale.com/blog/fine-tuning-is-for-form-not-facts https://www.anyscale.com/blog/fine-tuning-is-for-form-not-fa... (I work at Vectara, not Anyscale, but appreciate the details and thought they put into this blog) Fine-tuning can help with certain areas of knowledge acquisition, but it's costly and frankly doesn't work that well when the knowledge you're trying to provide goes "against" the data that was trained into the core system. e.g. try to fine tune a model that there is a cure to <disease X> if you're a research org and it's incredibly hard to do that because the base model may be so convinced otherwise. Probably the most difficult thing though is using fine-tuning to get a model to "forget" or "not respond to" something that it shouldn't take a stance on. I always ask a fine-tuned model: "how do you calculate the fourth interior angle of a triangle." This is obviously a nonsense question, but helps to show how most LLMs will happily tell you to sum the interior angles of your first 3 angles and then subtract from 180. A well-tuned RAG system will say "sorry, I don't have information for that." It shows how it's less "guess-y" and hallucinated
- namibj 3y ago
  Actually, selectively deleting facts like that does kinda work, essentially what you're doing is scolding-based reinforcement learning. For decent speed you'd want to probably use variance-reduction techniques like https://arxiv.org/abs/2205.08253 https://arxiv.org/abs/2205.08253 , though. Or a related (would be linked as cited by/cited in the linked when you look on semantic scholar) method that doesn't compute a hessian-vector product (which apparently interacts poorly with the non-linearity of the multi-layer perceptrons as provided by LLM-typical ReLU), in exchange for e.g. sizable batching (gradient accumulation!) at occasional checkpoints.
amanvir_ 3y ago
I've had this experience with RAG systems as well, but I've been wondering if structuring unstructured text (like a novel) in the form of a knowledge graph may solve this problem. Perhaps coupling a vector embedding search with a knowledge graph created from unstructured text could lead to more informed answers from LLMs? It seems as though some companies and researchers are already experimenting with this idea: https://www.nebula-graph.io/posts/graph-RAG https://www.nebula-graph.io/posts/graph-RAG.
aik 3y ago
No method today is a panacea. RAG naturally isn’t the approach you would take if you’re trying to summarize a large body of text, as in your example. That doesn’t make sense. That is not what it is for. What it’s awesome for is asking a somewhat specific question about some thing in a book if it can be answered by reviewing a paragraph or a few across the entire book. So to answer your question: Start by using the right tool for the job. The problem with this is: Some tools are prohibitively expensive, challenging, or very time-consuming to use.
- deegles 3y ago
  But the issue remains where your questions have to have easy "retrieval" aspects. You can't ask general questions that a human would have no trouble with. A book report is the example I used because you need to have read the entire book. essentially a book-length context window.
  - namibj 3y ago
    Thankfully, there are methods (like causal transformers, but also some question/text/answer architectures) for not re-computing self-attention between the document/knowledge-base tokens across the iterations of autoregressive decoding for generating the response.
    - aik 3y ago
      Do you have an example of where these methods still produce good summaries? Eg if you adjust how re-computation of self-attention in autoregressive decoding / between token generations works to significantly decrease the amount of computation needed?
- bradknowles 3y ago
  The real problem is knowing what tools exist for the job. Or that the right tool actually does exist. I would submit this is true for many fields of knowledge.

filterfiber 3y ago

The most likely "real solution" is going to be using various tricks and finetuning on higher context lengths to just extend the context window. However I will say knowledge graphs are becoming more popular. I made a comment the other day with a list of some of the popular methods: https://news.ycombinator.com/item?id=38476596 https://news.ycombinator.com/item?id=38476596 I completely forgot to mention ROME in my last comment, where you can modify facts within an LLM https://arxiv.org/pdf/2202.05262.pdf https://arxiv.org/pdf/2202.05262.pdf Note that there could be something more recent then these that I missed, but as far as I know knowledge-graphs/RAG are what most people are currently using, but there's a lot of work being focused on extending the context window.

kristjansson 3y ago

This doesn't deserve the downvotes it (currently) has. This is a completely accurate statement, simple RAG works well when questions are highly correlated with specifics chunks of documents. It does not allow an LLM to synthesize an entire corpus to an answer (e.g. a book report). Other modes of RAG that retrieve multiple times per generation (e.g. every so-many tokens) can improve performance, and intuitively feel closer to the 'sparse attention over the entire corpus' that RAG is approximating. However, fine-tuning on relevant, high quality, knowledge-rich question/answer pairs seems dominant, when such examples are available or can be generated.

filterfiber 3y ago
I have to agree, RAG is effectively just compensating for data not fitting in the context window, and it has (potentially severe) limitations. Summarizing a book I think is a great example, RAG prevents LLMs from correlating all of the data. It will likely never be obsolete, but a larger context would be far preferable in nearly all scenerios.
svcrunch 3y ago
I think your comment is accurate, but regarding your last point: "However, fine-tuning on relevant, high quality, knowledge-rich question/answer pairs seems dominant, when such examples are available or can be generated." How does one solve the problem of access-controlled data, if not through RAG? Do you imagine a separate version of the LLM for every user, reflecting their unique permissions on the data? Also, in scenarios where the data is being updated regularly, RAG provides much lower latency to the new information. Deletes also present a challenge for a pure-LLM approach.
- willy_k 3y ago
  I don’t know much about the space, but couldn’t LoRA be used to (moderately) efficiently use a relevant and individual fine-tuned model alongside a general LLM?

eskibars 3y ago

Having worked on a (production, as-a-service RAG system), what we've learned is that fine tuning is not good for really adding/removing facts but is great for changing the form of the output. Anyscale has a great blog on this: https://www.anyscale.com/blog/fine-tuning-is-for-form-not-facts https://www.anyscale.com/blog/fine-tuning-is-for-form-not-fa... (I work at Vectara, not Anyscale, but appreciate the details and thought they put into this blog) Fine-tuning can help with certain areas of knowledge acquisition, but it's costly and frankly doesn't work that well when the knowledge you're trying to provide goes "against" the data that was trained into the core system. e.g. try to fine tune a model that there is a cure to <disease X> if you're a research org and it's incredibly hard to do that because the base model may be so convinced otherwise. Probably the most difficult thing though is using fine-tuning to get a model to "forget" or "not respond to" something that it shouldn't take a stance on. I always ask a fine-tuned model: "how do you calculate the fourth interior angle of a triangle." This is obviously a nonsense question, but helps to show how most LLMs will happily tell you to sum the interior angles of your first 3 angles and then subtract from 180. A well-tuned RAG system will say "sorry, I don't have information for that." It shows how it's less "guess-y" and hallucinated

namibj 3y ago
Actually, selectively deleting facts like that does kinda work, essentially what you're doing is scolding-based reinforcement learning. For decent speed you'd want to probably use variance-reduction techniques like https://arxiv.org/abs/2205.08253 https://arxiv.org/abs/2205.08253 , though. Or a related (would be linked as cited by/cited in the linked when you look on semantic scholar) method that doesn't compute a hessian-vector product (which apparently interacts poorly with the non-linearity of the multi-layer perceptrons as provided by LLM-typical ReLU), in exchange for e.g. sizable batching (gradient accumulation!) at occasional checkpoints.

amanvir_ 3y ago

I've had this experience with RAG systems as well, but I've been wondering if structuring unstructured text (like a novel) in the form of a knowledge graph may solve this problem. Perhaps coupling a vector embedding search with a knowledge graph created from unstructured text could lead to more informed answers from LLMs? It seems as though some companies and researchers are already experimenting with this idea: https://www.nebula-graph.io/posts/graph-RAG https://www.nebula-graph.io/posts/graph-RAG.

aik 3y ago

No method today is a panacea. RAG naturally isn’t the approach you would take if you’re trying to summarize a large body of text, as in your example. That doesn’t make sense. That is not what it is for. What it’s awesome for is asking a somewhat specific question about some thing in a book if it can be answered by reviewing a paragraph or a few across the entire book. So to answer your question: Start by using the right tool for the job. The problem with this is: Some tools are prohibitively expensive, challenging, or very time-consuming to use.

deegles 3y ago
But the issue remains where your questions have to have easy "retrieval" aspects. You can't ask general questions that a human would have no trouble with. A book report is the example I used because you need to have read the entire book. essentially a book-length context window.
- namibj 3y ago
  Thankfully, there are methods (like causal transformers, but also some question/text/answer architectures) for not re-computing self-attention between the document/knowledge-base tokens across the iterations of autoregressive decoding for generating the response.
  - aik 3y ago
    Do you have an example of where these methods still produce good summaries? Eg if you adjust how re-computation of self-attention in autoregressive decoding / between token generations works to significantly decrease the amount of computation needed?
bradknowles 3y ago
The real problem is knowing what tools exist for the job. Or that the right tool actually does exist. I would submit this is true for many fields of knowledge.