pchunduri6

6 ms·

1. 1.▲
   by pchunduri6 3y ago

   I just tried the demo, and it looks great! Congrats on the launch! I have a couple of questions: 1) How often do you find that the LLM fails to generate the correct question-answer pairs? The biggest challenge I'm facing with LLM-based
2. 2.▲
   by pchunduri6 3y ago

   This was exactly my experience trying to build a RAG pipeline that answers complex questions by generating sub-questions [1]. The hidden prompts in the complex libraries are quite difficult to get to and then start tweaking, whereas writing
3. 3.▲
   by pchunduri6 3y ago

   1) While building this system, I found that the LLM can sometimes generate unpredictable responses. For example, the LLM sometimes chooses to summarize the document even for a simple retrieval question. When using expensive LLM models, this
4. 4.▲
   by pchunduri6 3y ago

   This is really useful! Using LLM-assisted evaluation seems like the way to go for evaluating RAG applications. One issue I've faced while evaluating responses using GPT-4 is that the evaluation cost can go out of hand rather quickly. D
5. 5.▲
   by pchunduri6 3y ago

   Yes, this is an excellent RAG use-case! The vector index that I use in the repository uses EvaDB [1] to retrieve the top-K matches to the user queries from the available data sources. So, you can manually inspect the best matches to your qu
6. 6.▲
   by pchunduri6 3y ago

   Thanks for the kind words! +1 for chainlit. I love their documentation. Do you have any specific use-cases in mind that would benefit from such pipelines?
7. 7.▲
   by pchunduri6 3y ago

   Thanks for the kind words and the great questions! -- LlamaIndex has some excellent abstractions. In fact, I started off this project with LlamaIndex using their sub-question query engine. However, I found that the abstractions often obfusc
8. 8.▲
   Show HN: Demystifying Advanced RAG Pipelines(github.com)

   131 points by pchunduri6 3y ago|19 comments