Older generation RTX 8000 / 48GB are reasonable. One big disadvantage for older Turing card, no bfloat16. But if you run a quantized/mixed precision model or Q

Older generation RTX 8000 / 48GB are reasonable. One big disadvantage for older Turing card, no bfloat16. But if you run a quantized/mixed precision model or QLoRA, it doesn’t hurt as much.