3 ms·
With DeepSpeed an SSD and lots of RAM you should be able to run inference with a 8GB card. There’s a thread somewhere on https://discuss.huggingface.co/ https:/
by chrisMyzel 4y ago
With DeepSpeed an SSD and lots of RAM you should be able to run inference with a 8GB card. There’s a thread somewhere on https://discuss.huggingface.co/ https://discuss.huggingface.co/ doing the math around this