Claude did a pretty good quality job explaining Retrieval Augmented Generation to me [1]: A choice quote: "The RAG model needs to retrieve memories that are r

Claude did a pretty good quality job explaining Retrieval Augmented Generation to me [1]: A choice quote: "The RAG model needs to retrieve memories that are relevant for the generation task, not just any random memories. It does this through two mechanisms: Memory indexing: The retrieved memories are indexed in a latent space, where similar memories are close together and dissimilar memories are far apart. The RAG model first encodes the prompt (e.g. a question) into the same latent space. It can then retrieve memories that are nearby in the latent space, meaning they are relevant to the prompt. Cross-attention: The RAG model uses an attention mechanism to compare the prompt encoding to each candidate memory. Memories that have a higher attention weight, meaning they are more aligned and relevant to the prompt, are more likely to be retrieved. This helps filter out irrelevant or dissimilar memories." [1] https://kemendo.com/blog/Claude.html https://kemendo.com/blog/Claude.html