2 ms·
Fat citation needed on RAG being expensive. Most embeddings models these days are smaller than most LLM models, and run more cheaply and more effectively than t
by Der_Einzige 3y ago
Fat citation needed on RAG being expensive. Most embeddings models these days are smaller than most LLM models, and run more cheaply and more effectively than the ones provided by OpenAI.
If you mean that increasing token counts in expensive, I suppose sure - but the retrieval side itself is not the cost center in general.
- MattDaEskimo 3y agoI am speaking of the OPs implementation of RAG, not in general. The retrieval part can be expensive if an LLM is used to confirm that it is sufficient, and if it's decided to continue searching if it's not. OpenAIs retrieval is a perfect example. It works, but it's very expensive