4 ms·
This isnt specifically Colab or HF, but have you checked out any of the community Runpod templates? Theres a few out there that give you a mostly turnkey way to
by spmurrayzzz 3y ago
This isnt specifically Colab or HF, but have you checked out any of the community Runpod templates? Theres a few out there that give you a mostly turnkey way to deploy models and test via Oobabooga, koboldai, or similar.
I use the one-click UI from TheBloke pretty frequently for inference testing, and I know theres some newer ones that also give you fine-tuning capabilities as well.
- brianjking 3y agoLink to the runpod templates? These run on your local machine or can I use Colab/Huggingface GPUs?
- spmurrayzzz 3y agohttps://www.runpod.io/console/templates https://www.runpod.io/console/templates This is the readme for the one I mentioned: https://github.com/TheBlokeAI/dockerLLM/blob/main/README_Runpod_LocalLLMsUIandAPI.md https://github.com/TheBlokeAI/dockerLLM/blob/main/README_Run... > can I use Colab/Huggingface GPUs? You use these templates on the runpod platform itself. Theres no free tier equivalent like you have with Colab/HF, but currently you can rent an RTX 4090 for $0.69/hr so its pretty affordable.
- brianjking 3y agoYeah, but I'd like to try deploying this on Huggingface or Lambda Labs. Do you know if that is possible?
- spmurrayzzz 3y agoIn theory yes, since its fundamentally just a Dockerfile, any provider that supports docker is possible. I know HF Spaces have a "docker" sdk option[1], but I personally haven't used them myself. [1] https://huggingface.co/docs/hub/spaces-sdks-docker-first-demo https://huggingface.co/docs/hub/spaces-sdks-docker-first-dem...