3 ms·
Just curious, will I be able to use it using my Nvidia card with 10GB of memory? Does it require multiple graphic cards?
by f311a 4y ago
Just curious, will I be able to use it using my Nvidia card with 10GB of memory?
Does it require multiple graphic cards?
- woodson 4y agoAs the model weights (even quantized) would be several hundred GBs, it’s unlikely, unless special inference code is written that loads and processes only a small subset of weights and calculations at a time. But running it that way would be painfully slow.
- lostmsu 4y agoThe code is already there: DeepSpeed
- robbedpeter 4y agoThe smaller models, yes. I'd bet dollars to donuts that gpt-neo and EleutherAI models outperform most, if not all, of Facebook's. Check out huggingface, you'll be able to run a 2.7b model or smaller. https://huggingface.co/EleutherAI/gpt-neo-2.7B/tree/main https://huggingface.co/EleutherAI/gpt-neo-2.7B/tree/main