2 ms·
Those are just the defaults, and spRAG is designed to be flexible in terms of the models you can use with it. For AutoContext (which is just a summarization tas
by zmccormick7 2y ago
Those are just the defaults, and spRAG is designed to be flexible in terms of the models you can use with it. For AutoContext (which is just a summarization task) Haiku offers a great balance of price and performance. Llama 3-8B would also be a great choice there, especially if you want something you can run locally. For reranking, the Cohere v3 reranker is by far the best performer on the market right now. And for embeddings, it's really a toss-up between OpenAI, Cohere, and Voyage.
- Cheer2171 2y agoI bet you'll get a lot more adoption if you put info about using it with local self-hosted LLMs there. I'll never trust a cloud service with the documents I want to RAG.
- zmccormick7 2y agoAgreed. I've gotten a lot of feedback along those lines today, so that's my top priority now.