3 ms·

> seriously, measuring graphics cards in gigabytes is like measuring battery life in volts. it's the wrong unit. Yes, but, if you need 48GB to run inference on

by senko 3y ago

> seriously, measuring graphics cards in gigabytes is like measuring battery life in volts. it's the wrong unit. Yes, but, if you need 48GB to run inference on a model, and you only have 24GB available, you don't get to enjoy the tflops difference. If nVidia released somewhat lower-performance GPUs but with more VRAM, we could talk. But they're not stupid :)

nightski 3y ago
You can get an A6000 for cheaper than an M2 Ultra. That said it's still only 48GB.
- senko 3y ago
  I'm not sure if it's cheaper when you compare the whole machine price, not just GPU, since you can't buy piecewise with the Mac.
  - nightski 3y ago
    The GPU is $4500. That gives you $2000 to build a machine (which you can build a top of the line consumer PC for cheaper than that, I just did recently w/128GB DDR5 and Zen 4 7950X3D).
    - senko 3y ago
      Cool if you can find it at that price. On my side of the pond, it's around €6,000.
sterlind 3y ago
yes you do. you can use DeepSpeed. it's only like 2x slower.