3 ms·

There are already several RAG chat open source solutions available. Two that immediately come to mind are: Danswer https://github.com/danswer-ai/danswer https

There are already several RAG chat open source solutions available. Two that immediately come to mind are: Danswer https://github.com/danswer-ai/danswer https://github.com/danswer-ai/danswer Khoj https://github.com/khoj-ai/khoj https://github.com/khoj-ai/khoj

wkat4242 3y ago
Stupid question but what does RAG stand for?
- onehp 3y ago
  Retrieval augmented generation. In short you use an LLM to classify your documents (or chunks from them) up front. Then when you want to ask the LLM a question you pull the most relevant ones back to feed it as additional context.
  - wkat4242 3y ago
    Ahh ok I see. It's basically what MS CoPilot 365 does too, with its "Grounding" step.
    - brianjking 3y ago
      Yes.
  - danielovichdk 3y ago
    I dont get it. To my understanding it takes huge amounts of data to build any any form of RAG. Simply because it enlarges the statistical model you later prompt. If the model is not big enough how would you expect it to answer you in a non qualifying matter ? It simply can't. So I don't really buy it and I have yet to see it work better than any rdbms search index. Tell me I am wrong, I would like to see a local model based on my own docs being able to answer me quality answers based on quality prompts.
    - Grimblewald 3y ago
      You're misunderstanding. Imagine your query is matched against chunks of text from the database, where the relevance of information is evaluated for the window each time it slides. Then collecting the n most relevant chunks, these are included in the prompt so then the llm can provide answers from source documents verbatim. This is useful for cases where precise and exact answers are needed. For example, searching the docs for some package for the right api to call. You don't mant a name that's close to right it has to be correct to the character.
    - tveita 3y ago
      RAG doesn't require much data or involve any training, it is a fancy name for "automatically paste some relevant context into the prompt" Basically if you have a database of three emails and ask when Biff wanted to meet for lunch, a RAG system would select the most relevant email based on any kind of search - embeddings are most fashionable, and create a prompt like """Given this document: <your email>, answer the question "When does Biff want to meet for lunch?"""
      - machiaweliczny 3y ago
        Legit explanation, that's how it works AFAIK.
      - loudmax 3y ago
        That's not how RAG works. What you're describing is something closer to prompt optimization. Sibling comment from discordance has a more accurate description of RAG. There's a longer description from Nvidia here: https://blogs.nvidia.com/blog/what-is-retrieval-augmented-generation/ https://blogs.nvidia.com/blog/what-is-retrieval-augmented-ge...
        tveita 3y ago
        Right, you read something nebulous about how "the LLM combines the retrieved words and its own response to the query into a final answer it presents to the user", and you think there is some magic going on, and then you click one link deeper and read at https://ai.meta.com/blog/retrieval-augmented-generation-streamlining-the-creation-of-intelligent-natural-language-processing-models/ https://ai.meta.com/blog/retrieval-augmented-generation-stre... : > Given the prompt “When did the first mammal appear on Earth?” for instance, RAG might surface documents for “Mammal,” “History of Earth,” and “Evolution of Mammals.” These supporting documents are then concatenated as context with the original input and fed to the [...] model Finding the relevant context to put in the prompt is a search problem, nearest neighbour search on embeddings is one basic way to do it but the singular focus on "vector databases" is a bit of hype phenomenon IMO - a real world product should factor a lot more than just pure textual content into the relevancy score. Or is your personal AI assistant going to treat emails from yesterday as equally relevant as emails from a year ago?

wkat4242 3y ago

Stupid question but what does RAG stand for?

onehp 3y ago
Retrieval augmented generation. In short you use an LLM to classify your documents (or chunks from them) up front. Then when you want to ask the LLM a question you pull the most relevant ones back to feed it as additional context.
- wkat4242 3y ago
  Ahh ok I see. It's basically what MS CoPilot 365 does too, with its "Grounding" step.
  - brianjking 3y ago
    Yes.
- danielovichdk 3y ago
  I dont get it. To my understanding it takes huge amounts of data to build any any form of RAG. Simply because it enlarges the statistical model you later prompt. If the model is not big enough how would you expect it to answer you in a non qualifying matter ? It simply can't. So I don't really buy it and I have yet to see it work better than any rdbms search index. Tell me I am wrong, I would like to see a local model based on my own docs being able to answer me quality answers based on quality prompts.
  - Grimblewald 3y ago
    You're misunderstanding. Imagine your query is matched against chunks of text from the database, where the relevance of information is evaluated for the window each time it slides. Then collecting the n most relevant chunks, these are included in the prompt so then the llm can provide answers from source documents verbatim. This is useful for cases where precise and exact answers are needed. For example, searching the docs for some package for the right api to call. You don't mant a name that's close to right it has to be correct to the character.
  - tveita 3y ago
    RAG doesn't require much data or involve any training, it is a fancy name for "automatically paste some relevant context into the prompt" Basically if you have a database of three emails and ask when Biff wanted to meet for lunch, a RAG system would select the most relevant email based on any kind of search - embeddings are most fashionable, and create a prompt like """Given this document: <your email>, answer the question "When does Biff want to meet for lunch?"""
    - machiaweliczny 3y ago
      Legit explanation, that's how it works AFAIK.
    - loudmax 3y ago
      That's not how RAG works. What you're describing is something closer to prompt optimization. Sibling comment from discordance has a more accurate description of RAG. There's a longer description from Nvidia here: https://blogs.nvidia.com/blog/what-is-retrieval-augmented-generation/ https://blogs.nvidia.com/blog/what-is-retrieval-augmented-ge...
      - tveita 3y ago
        Right, you read something nebulous about how "the LLM combines the retrieved words and its own response to the query into a final answer it presents to the user", and you think there is some magic going on, and then you click one link deeper and read at https://ai.meta.com/blog/retrieval-augmented-generation-streamlining-the-creation-of-intelligent-natural-language-processing-models/ https://ai.meta.com/blog/retrieval-augmented-generation-stre... : > Given the prompt “When did the first mammal appear on Earth?” for instance, RAG might surface documents for “Mammal,” “History of Earth,” and “Evolution of Mammals.” These supporting documents are then concatenated as context with the original input and fed to the [...] model Finding the relevant context to put in the prompt is a search problem, nearest neighbour search on embeddings is one basic way to do it but the singular focus on "vector databases" is a bit of hype phenomenon IMO - a real world product should factor a lot more than just pure textual content into the relevancy score. Or is your personal AI assistant going to treat emails from yesterday as equally relevant as emails from a year ago?