3 ms·
It's still much cheaper to run RAG in production (at least if you are using closed models). I'd love to use the entire context of GPT4, but if I do that in prod
by Narciss 3y ago
It's still much cheaper to run RAG in production (at least if you are using closed models). I'd love to use the entire context of GPT4, but if I do that in production it'll cost much more than using some RAG-dependent implementation.
- humansareok1 3y agoBut this is just current state. Token costs continue to go down and contexts will continue to get larger.