4 ms·
I spent $100 to learn how to build and run RAG application in production
- battles 3y agoIf you're gonna use bots to hype up your submissions, at least make them sound different.
- staranjeet 3y agoThis post is about learnings by running a RAG application in production. Here are the learnings: • Always customise your prompt. • Set soft & hard limit on your LLM cost before launching any project. • Choose the LLM model wisely. • Context length matters a lot. • Cache your queries. • Have a router to choose LLM model wisely. • Have a UI to see all queries, answers, context & metrics like response time. • Memory management in chat is painful.
- d-even 3y agoGreat! This is very helpful, thank you for sharing such insights!
- shubhi712 3y agoGreat insights on do's and dont's of RAG app development! Embedchain seems quite promising. Will give it a try for my next project.