3 ms·
Is there a way to run it self-hosted locally?
by cjbprime 7y ago
Is there a way to run it self-hosted locally?
- idle_zealot 7y agoYeah, just clone the repo, download the model, and run the start script. If you don't have a GPU with ~14GB of RAM expect each response to take a few minutes to be generated though.
- cjbprime 7y agoThanks! It's not that bad -- am on a 2080 8GB and it takes less than a minute for each response.
- flatiron 7y agoat least for me it complains my python is too new (arch) so I installed it on a nvidia cuda docker container (which is using 18.04) and for some reason it still complains it can't use CUDA but works fine on the CPU although slow