3 ms·

This feels like a "No true Scotsman" answer. You say > RAG is not used to try to anchor to reality a general LLM by somehow making it come up with sources and

This feels like a "No true Scotsman" answer. You say > RAG is not used to try to anchor to reality a general LLM by somehow making it come up with sources and links. but that definitely is one particular use of RAG, i.e. to limit some potential hallucinations by grounding it in data provided in the prompt.

bambax 2y ago
Not really. RAG works very differently from a general (or generalist?) LLM. RAG is vector search first. It encodes the query, finds nearest vectors in the vector database, retrieves the fragments attached to those vectors, and then sends those vectors to the LLM for it to summarize them. A general LLM like Gemini or Claude or ChatGPT first produces an answer to a question, based on its training. This doesn't involve searching any external source at that point. Then after that answer is produced, the LLM can try to find sources that match what it has come up with.
- outofpaper 2y ago
  This is a generalization. These proprietary systems do different things at different times. With GPT4o you can see little icons when a RAG is in use or when code and tests are being used. People, we have to stop talking about what we know as though it's all there is. Don't confuse our knowledge for understanding. Understanding only comes from repeadly trying to prove our understandings wrong and learning how things truly are.
  - acka 2y ago
    This. There is no point in arguing amongst ourselves when we're all wearing blindfolds and it is the zookeeper who decides what parts of the proprietary elephant we're allowed to touch.
- simonw 2y ago
  You don't have to use vector search to implement RAG. You can use other search mechanisms instead or as well - whatever it takes to populate the context with the material most likely to help answer the user's question. Two common alternatives to vector search: 1. Ask the LLM to identify key terms in the user's question and use those terms with a regular full-text search engine 2. If the content you are answering questions about is short enough - an employee handbook for example - just jam the whole thing in the context. Claude 3 supports 200,000 tokens and Gemini Pro 1.5 supports a million so this can actually be pretty effective.
  - bambax 2y ago
    You're right and I was imprecise. 2. is how RAG is implemented in most cases. (I wouldn't call 1. 'RAG' exactly, though?)
    - bigfudge 2y ago
      HYDE is a related technique. Ask the model to generate a response with no context, then use this for semantic search agains actual data and respond by summarising these documents.
    - simonw 2y ago
      I'd definitely call 1 (the FTS version) RAG. It's how Bing and Google Gemini and ChatGPT Browse work - they don't have a full vector index of the Web to work with (at least as far as I know), they use the model's best guess at an appropriate FTS query instead.
- ozr 2y ago
  That's not what RAG is. RAG is the process of adding relevant information to a prompt in an LLM. It's a form of in-context learning.
  - mr_toad 2y ago
    It’s right there in the name - first you Retrieve relevant information (often a vector lookup) then you use it to Augment the prompt, then you Generate an answer. It’s bloody useful if you can’t cram your entire proprietary code base into a prompt.
  - kergonath 2y ago
    What makes it useful is precisely that it is not learning.
    - ozr 2y ago
      Yes it is; it's just not updating the weights. https://ai.stanford.edu/blog/understanding-incontext/ https://ai.stanford.edu/blog/understanding-incontext/
kergonath 2y ago
RAG really is not the right tool for that. It does not event prevent hallucinations. It is useful because it can retrieve information from documents the model never saw during its training phase and does not require a costly re-training. It is not a fine tuning either, and it cannot fundamentally change the model’s behaviour.
- bigfudge 2y ago
  In my experience, it really does reduce the risk of hallucination, especially if paired with a prompt which explicitly instruct the model to use only facts from the context window. Another strategy is to provide unique identifiers for the rag chunks dropped into the context and ask the model to cross reference in its response. You can then check that the response is exciting evidence from the context with Simple pattern match.
  - sahn44 2y ago
    Can you elaborate a little more on this? What's an example of how you do that in the prompt?
    - tkellogg 2y ago
      not OP, but for the "unique identifiers", you can think of it like the footnote style of markdown links. Most of these models are fine-tuned to do markdown well, and a short identifier is less likely to be hallucinated (my philosophy anyway), so it usually works pretty well. For the examples, something like this can work ID: 1 URL: https://example.com/test.html https://example.com/test.html Text: lksdjflkdsjlksjkl And then tell it to use the ID to link to the page.
      - throwthrowuknow 2y ago
        Instead of having the LLM generate the links couldn’t you use a combination of keyword matching and similarity on the model output and the results to automatically add citations? You could use a smaller NLP model or even a rule based system to extract entities or phrases to compare. I’m sure this is already being done by bing for example.
notjoemama 2y ago
This feels like a quantification fallacy. You say > but that definitely is one particular use of RAG One use of RAG doesn't imply that all uses of RAG are for grounding LLM hallucinations. Additionally, I disdain when fallacies come up in conversation when it doesn't appear someone is making an argument in bad faith. We all use logical fallacies by accident from time to time. A good use of calling them out is when they are used in poor taste. I find that more on Reddit than Hacker News. I have now also engaged in the Tu Quoque fallacy in this reply. See how annoying they are?

bambax 2y ago

Not really. RAG works very differently from a general (or generalist?) LLM. RAG is vector search first. It encodes the query, finds nearest vectors in the vector database, retrieves the fragments attached to those vectors, and then sends those vectors to the LLM for it to summarize them. A general LLM like Gemini or Claude or ChatGPT first produces an answer to a question, based on its training. This doesn't involve searching any external source at that point. Then after that answer is produced, the LLM can try to find sources that match what it has come up with.

outofpaper 2y ago
This is a generalization. These proprietary systems do different things at different times. With GPT4o you can see little icons when a RAG is in use or when code and tests are being used. People, we have to stop talking about what we know as though it's all there is. Don't confuse our knowledge for understanding. Understanding only comes from repeadly trying to prove our understandings wrong and learning how things truly are.
- acka 2y ago
  This. There is no point in arguing amongst ourselves when we're all wearing blindfolds and it is the zookeeper who decides what parts of the proprietary elephant we're allowed to touch.
simonw 2y ago
You don't have to use vector search to implement RAG. You can use other search mechanisms instead or as well - whatever it takes to populate the context with the material most likely to help answer the user's question. Two common alternatives to vector search: 1. Ask the LLM to identify key terms in the user's question and use those terms with a regular full-text search engine 2. If the content you are answering questions about is short enough - an employee handbook for example - just jam the whole thing in the context. Claude 3 supports 200,000 tokens and Gemini Pro 1.5 supports a million so this can actually be pretty effective.
- bambax 2y ago
  You're right and I was imprecise. 2. is how RAG is implemented in most cases. (I wouldn't call 1. 'RAG' exactly, though?)
  - bigfudge 2y ago
    HYDE is a related technique. Ask the model to generate a response with no context, then use this for semantic search agains actual data and respond by summarising these documents.
  - simonw 2y ago
    I'd definitely call 1 (the FTS version) RAG. It's how Bing and Google Gemini and ChatGPT Browse work - they don't have a full vector index of the Web to work with (at least as far as I know), they use the model's best guess at an appropriate FTS query instead.
ozr 2y ago
That's not what RAG is. RAG is the process of adding relevant information to a prompt in an LLM. It's a form of in-context learning.
- mr_toad 2y ago
  It’s right there in the name - first you Retrieve relevant information (often a vector lookup) then you use it to Augment the prompt, then you Generate an answer. It’s bloody useful if you can’t cram your entire proprietary code base into a prompt.
- kergonath 2y ago
  What makes it useful is precisely that it is not learning.
  - ozr 2y ago
    Yes it is; it's just not updating the weights. https://ai.stanford.edu/blog/understanding-incontext/ https://ai.stanford.edu/blog/understanding-incontext/

kergonath 2y ago

RAG really is not the right tool for that. It does not event prevent hallucinations. It is useful because it can retrieve information from documents the model never saw during its training phase and does not require a costly re-training. It is not a fine tuning either, and it cannot fundamentally change the model’s behaviour.

bigfudge 2y ago
In my experience, it really does reduce the risk of hallucination, especially if paired with a prompt which explicitly instruct the model to use only facts from the context window. Another strategy is to provide unique identifiers for the rag chunks dropped into the context and ask the model to cross reference in its response. You can then check that the response is exciting evidence from the context with Simple pattern match.
- sahn44 2y ago
  Can you elaborate a little more on this? What's an example of how you do that in the prompt?
  - tkellogg 2y ago
    not OP, but for the "unique identifiers", you can think of it like the footnote style of markdown links. Most of these models are fine-tuned to do markdown well, and a short identifier is less likely to be hallucinated (my philosophy anyway), so it usually works pretty well. For the examples, something like this can work ID: 1 URL: https://example.com/test.html https://example.com/test.html Text: lksdjflkdsjlksjkl And then tell it to use the ID to link to the page.
    - throwthrowuknow 2y ago
      Instead of having the LLM generate the links couldn’t you use a combination of keyword matching and similarity on the model output and the results to automatically add citations? You could use a smaller NLP model or even a rule based system to extract entities or phrases to compare. I’m sure this is already being done by bing for example.

notjoemama 2y ago

This feels like a quantification fallacy. You say > but that definitely is one particular use of RAG One use of RAG doesn't imply that all uses of RAG are for grounding LLM hallucinations. Additionally, I disdain when fallacies come up in conversation when it doesn't appear someone is making an argument in bad faith. We all use logical fallacies by accident from time to time. A good use of calling them out is when they are used in poor taste. I find that more on Reddit than Hacker News. I have now also engaged in the Tu Quoque fallacy in this reply. See how annoying they are?