4 ms·
No idea yet. Which you recommend to start? It will be hosted on a Ubuntu server (Digital Ocean, Linode etc..)
by marcopicentini 3y ago
No idea yet. Which you recommend to start?
It will be hosted on a Ubuntu server (Digital Ocean, Linode etc..)
- reustle 3y agoYou can definitely start by checking out ollama, it was super helpful for me
- marcopicentini 3y agoIt’s only for MacOSX. I expect to load the model on a Ubuntu server, not on my local dev machine.
- Menatombo 3y agoYou have to build it if you want it for Ubuntu, Windows, or anything else. Just build Go on your machine and have at it.
- detente18 3y agoAny reason you're doing that vs. using Lambda Labs / Replicate / together.ai / Banana.dev, etc. There's a lot of good model deployment platforms that would make it easy to call your model behind a hosted endpoint -- If you do want to self-host - there's some great libraries like https://github.com/lm-sys/FastChat https://github.com/lm-sys/FastChat and https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp that might be helpful If none of these really solve your issue - feel free to email me and I'm happy to help you figure something out - krrish@berri.ai