2 ms·
I'd just get an RTX Pro 6000 Blackwell and call it a day. More VRAM. Somewhat less bandwidth but it's your bandwidth.
by CamperBob2 8mo ago
I'd just get an RTX Pro 6000 Blackwell and call it a day. More VRAM. Somewhat less bandwidth but it's your bandwidth.
- k2so 8mo agoYeah, for a single GPU inference, considering the higher VRAM and FP4 support on the RTX 6000, it should fit larger models as well than the H100.