3 ms·
Retrieval Augmented Generation. It's the trick where you take a question from a user, search for documents that match that question, stuff as many of the relev
by simonw 3y ago
Retrieval Augmented Generation.
It's the trick where you take a question from a user, search for documents that match that question, stuff as many of the relevant chunks of content from those documents as you can into the prompt (usually 4,000 or 8,000 tokens, but Claude can go up to 100,000) and then say to the LLM "Based on this context, answer this question: QUESTION".
I wrote about one way to implement that here: https://simonwillison.net/2023/Jan/13/semantic-search-answers/ https://simonwillison.net/2023/Jan/13/semantic-search-answer...