2 ms·

Hybrid search is definitely worth exploring (e.g. adding in TF-IDF). I believe there is such an implementation out of the box with Weaviate. I have tried many

Hybrid search is definitely worth exploring (e.g. adding in TF-IDF). I believe there is such an implementation out of the box with Weaviate. I have tried many techniques and seen others try many different techniques. I think the hardest part is selecting the RIGHT technique. This is why it is somewhat easy to deploy a RAG pipeline but very hard to optimize one. It's hard to understand why it's failing and the global implications of design choices you make in your ingestion / embedding process.

alchemist1e9 3y ago
I’ll give you an analogy. We can imagine the content ingested is like a textbook, which would have a table of contents and an index. Now we lookup a topic, we find it in the TOC then likely we should read that chapter in whole, we find it in the index, then we likely read all the chapters it’s mentioned. I’d suggest RAG might perform better if it worked somewhat like that, the chunks for embeddings should be paragraph and sentence aware, and ideally should be tagged with any existing TOC or natural sections/headings that exist in the document. This approach would allow a retrieval logic that provides cohesive information, like an entire chapter or at least 3 paragraphs prior and 3 after the matched vector.
- esafak 3y ago
  Are you suggesting adding another step after retrieval, to add the surrounding context before submitting it to the LLM?
  - alchemist1e9 3y ago
    Yes absolutely. Often to understand the context of a topic it requires more than just the specific paragraph it is discussed and the LLMs are actually perfectly capable of understanding fairly complex concepts if provide the raw material properly.
    - lmeyerov 3y ago
      This seems similar to recursive summary indexing and then a choice of which linked chunks to load upon a match There are a lot of 'policy' choices here, so I've been curious how to automatically decide. The KG, GNN, and IR literature seems to have a lot of techniques relating to this, so been very non-obvious to me

alchemist1e9 3y ago

I’ll give you an analogy. We can imagine the content ingested is like a textbook, which would have a table of contents and an index. Now we lookup a topic, we find it in the TOC then likely we should read that chapter in whole, we find it in the index, then we likely read all the chapters it’s mentioned. I’d suggest RAG might perform better if it worked somewhat like that, the chunks for embeddings should be paragraph and sentence aware, and ideally should be tagged with any existing TOC or natural sections/headings that exist in the document. This approach would allow a retrieval logic that provides cohesive information, like an entire chapter or at least 3 paragraphs prior and 3 after the matched vector.

esafak 3y ago
Are you suggesting adding another step after retrieval, to add the surrounding context before submitting it to the LLM?
- alchemist1e9 3y ago
  Yes absolutely. Often to understand the context of a topic it requires more than just the specific paragraph it is discussed and the LLMs are actually perfectly capable of understanding fairly complex concepts if provide the raw material properly.
  - lmeyerov 3y ago
    This seems similar to recursive summary indexing and then a choice of which linked chunks to load upon a match There are a lot of 'policy' choices here, so I've been curious how to automatically decide. The KG, GNN, and IR literature seems to have a lot of techniques relating to this, so been very non-obvious to me