4 ms·

I still don't understand why we have a different term (RAG) for something we've been doing for a while i.e. use a vector db/embeddings coupled with LLM to gener

I still don't understand why we have a different term (RAG) for something we've been doing for a while i.e. use a vector db/embeddings coupled with LLM to generate better answers. LLamaIndex (formerly GPTIndex) has been around for a while and the Pinecone boys have been talking about this for months. Is there a breakthrough in terms of the way the pipeline is set up here?

sprobertson 3y ago
Because "RAG" is shorter than "use a vector db/embeddings coupled with LLM to generate better answers" But better answer: "retrieval augmented" has been used in papers since at least 2018 [1], the full "RAG" acronym since at least 2020 [2], and it was recently repopularized when the GPT3+ hype wave realized they were doing the same thing. [1] https://arxiv.org/abs/1808.03430 https://arxiv.org/abs/1808.03430 [2] https://arxiv.org/abs/2005.11401 https://arxiv.org/abs/2005.11401
- jamesbriggs 3y ago
  Yeah RAG has been around for some time, paper [2] being where I first stumbled onto it — I remember a 2-3 years ago building a RAG pipeline using either the model they trained in paper 2, or that was trained based on the same idea, with Deepset AI's Haystack library. It has been a little surreal to see that terminology become so popular again
ilaksh 3y ago
I think the use of this term "RAG" is counterproductive and almost makes me suspicious. Because it is used ambiguously and imprecisely, usually with the main aim being to suggest a level of expertise. But all it really means is that you put some extra information in your prompt (from somewhere) that didn't come from the user. It's basically the first thing that anyone thinks of once they get the API call working. People don't use it to only mean vector search, despite often assuming that is what it means. What I have been leaning towards recently is not using vectors at all, or only using it when I have an extremely close vector match. Instead I give the API one or more functions to look up information that is like an index for documentation or retrieving some info from a database. But it's specific to the application. I think usually the non-vector stuff is more effective.
- sprobertson 3y ago
  > all it really means is that you put some extra information in your prompt (from somewhere) that didn't come from the user Yes, and if you are getting (retrieving) some extra information to add to (augment) your prompt for further responses (generation) then it's not very ambiguous.
  - ilaksh 3y ago
    Right, the name makes sense. People made it ambiguous because they often believe it means vector search. I don't think that's correct, but if you look in this discussion, that's the reality of usage. So that makes it ambiguous in practice.
    - sprobertson 3y ago
      I see what you mean - not sure about the usage of ambiguous but some people do have an overly narrow definition in mind

sprobertson 3y ago

Because "RAG" is shorter than "use a vector db/embeddings coupled with LLM to generate better answers" But better answer: "retrieval augmented" has been used in papers since at least 2018 [1], the full "RAG" acronym since at least 2020 [2], and it was recently repopularized when the GPT3+ hype wave realized they were doing the same thing. [1] https://arxiv.org/abs/1808.03430 https://arxiv.org/abs/1808.03430 [2] https://arxiv.org/abs/2005.11401 https://arxiv.org/abs/2005.11401

jamesbriggs 3y ago
Yeah RAG has been around for some time, paper [2] being where I first stumbled onto it — I remember a 2-3 years ago building a RAG pipeline using either the model they trained in paper 2, or that was trained based on the same idea, with Deepset AI's Haystack library. It has been a little surreal to see that terminology become so popular again

ilaksh 3y ago

I think the use of this term "RAG" is counterproductive and almost makes me suspicious. Because it is used ambiguously and imprecisely, usually with the main aim being to suggest a level of expertise. But all it really means is that you put some extra information in your prompt (from somewhere) that didn't come from the user. It's basically the first thing that anyone thinks of once they get the API call working. People don't use it to only mean vector search, despite often assuming that is what it means. What I have been leaning towards recently is not using vectors at all, or only using it when I have an extremely close vector match. Instead I give the API one or more functions to look up information that is like an index for documentation or retrieving some info from a database. But it's specific to the application. I think usually the non-vector stuff is more effective.

sprobertson 3y ago
> all it really means is that you put some extra information in your prompt (from somewhere) that didn't come from the user Yes, and if you are getting (retrieving) some extra information to add to (augment) your prompt for further responses (generation) then it's not very ambiguous.
- ilaksh 3y ago
  Right, the name makes sense. People made it ambiguous because they often believe it means vector search. I don't think that's correct, but if you look in this discussion, that's the reality of usage. So that makes it ambiguous in practice.
  - sprobertson 3y ago
    I see what you mean - not sure about the usage of ambiguous but some people do have an overly narrow definition in mind