3 ms·
This particular model has 83.66gb of model weights so you'll need to 2x Nvidia 80gb A100 at a minimum unless you're loading it in 8bit mode.
by brianjking 3y ago
This particular model has 83.66gb of model weights so you'll need to 2x Nvidia 80gb A100 at a minimum unless you're loading it in 8bit mode.
- brianjking 3y agoWith that said, there are ggml/gptq and other optimization techniques.