4 ms·
I’ll give you an analogy. We can imagine the content ingested is like a textbook, which would have a table of contents and an index. Now we lookup a topic, we f
by alchemist1e9 3y ago
I’ll give you an analogy. We can imagine the content ingested is like a textbook, which would have a table of contents and an index. Now we lookup a topic, we find it in the TOC then likely we should read that chapter in whole, we find it in the index, then we likely read all the chapters it’s mentioned.
I’d suggest RAG might perform better if it worked somewhat like that, the chunks for embeddings should be paragraph and sentence aware, and ideally should be tagged with any existing TOC or natural sections/headings that exist in the document. This approach would allow a retrieval logic that provides cohesive information, like an entire chapter or at least 3 paragraphs prior and 3 after the matched vector.
- esafak 3y agoAre you suggesting adding another step after retrieval, to add the surrounding context before submitting it to the LLM?
- alchemist1e9 3y agoYes absolutely. Often to understand the context of a topic it requires more than just the specific paragraph it is discussed and the LLMs are actually perfectly capable of understanding fairly complex concepts if provide the raw material properly.
- lmeyerov 3y agoThis seems similar to recursive summary indexing and then a choice of which linked chunks to load upon a match There are a lot of 'policy' choices here, so I've been curious how to automatically decide. The KG, GNN, and IR literature seems to have a lot of techniques relating to this, so been very non-obvious to me