3 ms·
With 8 bit training you can do ~13B pram LLM on 3090/4090. https://huggingface.co/blog/trl-peft https://huggingface.co/blog/trl-peft But it is pretty cheap to
by tetete 4y ago
With 8 bit training you can do ~13B pram LLM on 3090/4090. https://huggingface.co/blog/trl-peft https://huggingface.co/blog/trl-peft
But it is pretty cheap to rent something at vast.ai or whatever to get 40GB for a final run.
- nullsense 4y agoAwesome. Between crypto hype in 2017 and AI hype in 2023 I've acquired a collection of 2x 1080ti, an RTX 3060 and an RTX 4090. All together it's a total of 58GB of VRAM. Is there a way I can pool it all across a distributed cluster of 2 machines for doing anything? I'm assuming it would bottleneck on both network speeds and the slowest GPUs if it's possible at all...
- all2 4y agoCould you package all that compute into a "virtual" graphics card?