3 ms·
Is there a version of this set up to be cpu-only, as in something that can use ggml tech? I'd love to deploy this on some servers with lots of ram and cpu horse
by generalizations 3y ago
Is there a version of this set up to be cpu-only, as in something that can use ggml tech? I'd love to deploy this on some servers with lots of ram and cpu horsepower, but no gpus.
- jasonwcfan 3y agoNot yet, but we can definitely add it. Created an issue: https://github.com/psychic-api/rag-stack/issues/2 https://github.com/psychic-api/rag-stack/issues/2 In the meantime it uses GPT4all when running locally so you can technically deploy it as well, but it's not very good.