2 ms·
While it was a mistake, should it be an amazing price? Sure, if you want 10x the memory bandwidth of a smaller card, that should be expensive. But 80GB of GDD
by Dylan16807 3y ago
While it was a mistake, should it be an amazing price?
Sure, if you want 10x the memory bandwidth of a smaller card, that should be expensive.
But 80GB of GDDR6 would currently cost something like... $300. Or if you looked 1-3 years ago it would have been $1000.
GDDR6 is already designed so that you can attach 8 data lines per chip. A high end GPU with a 384-bit memory bus could attach 48 chips that way and have 96GB. Or exactly 80GB on a 320-bit bus.
$1000-1500 retail baseline for the GPU, $250-800 for extra RAM, $1000+ for the extra design hassle... I think you'd be able to buy that for $3500 if we had better competition.
For a lot of use cases, you want a balance between compute and memory. For big AI models outside of a datacenter, memory is far more important. It's worth putting half the budget into RAM chips if that means your model can fit, even if you "only" get 100 teraflops at FP16.