4 ms·

I have a feeling regular consumers will have trouble buying 5090s. RTX 5090: 32 GB GDDR7, ~1.8 TB/s bandwidth. H100 (SXM5): 80 GB HBM3, ~3+ TB/s bandwidth. RT

I have a feeling regular consumers will have trouble buying 5090s. RTX 5090: 32 GB GDDR7, ~1.8 TB/s bandwidth. H100 (SXM5): 80 GB HBM3, ~3+ TB/s bandwidth. RTX 5090: ~318 TFLOPS in ray tracing, ~3,352 AI TOPS. H100: Optimized for matrix and tensor computations, with ~1,000 TFLOPS for AI workloads (using Tensor Cores). RTX 5090: 575W, higher for enthusiast-class performance. H100 (PCIe): 350W, efficient for data centers. RTX 5090: Expected MSRP ~$2,000 (consumer pricing). H100: Pricing starts at ~$15,000–$30,000+ per unit.

topherjaynes 2y ago
That's my worry too, I'd like one or two, but 1) will either never be in line for them 2) or can only find via secondary market at 3 or 4x the price...
boroboro4 2y ago
H100 has 3958 TFLOPS sparse fp8 compute. I’m pretty sure listed tflops for 5090 are sparse (and probably) fp4/int4.
- rfoo 2y ago
  Yes, that's the case. Check the (partial) spec of 5090 D, which is the nerfed version for export to China. It is marketed as having 2375 "AI TOPS". BIS demands it to be less than $4800 TOPS \times Bit-Width$, and the most plausible explanation for the number is - 2375 sparse fp4/int4 TOPS, which means 1187.5 dense TOPS for 4 bit, or $4750 TOPS \times Bit-Width$.
- boroboro4 2y ago
  And just for the context RTX 4090 has 2642 sparse int4 TOPS, so it’s about 25% increase
bee_rider 2y ago
How well do these models do at parallelizing across multiple GPUs? Is spending $4k on the 5090 a good idea for training, slightly better performance for much cheaper? Or a bad idea, 0x as good performance because you can’t fit your 60GB model on the thing?
Havoc 2y ago
> regular consumers will have trouble buying 5090s. They’re not really supposed to either judging by how they priced this. For non AI uses the 5080 is infinitely better positioned
- kllrnohj 2y ago
  > For non AI uses the 5080 is infinitely better positioned ...and also slower than a 4090. Only the 5090 got a gen/gen upgrade in shader counts. Will have to wait for benchmarks of course, but the rest of the 5xxx lineup looks like a dud
  - Havoc 2y ago
    You’re comparing different price classes. New 5080 is 999 usd. Even secondhand 4090s from questionable eBay sellers are substantially more

topherjaynes 2y ago

That's my worry too, I'd like one or two, but 1) will either never be in line for them 2) or can only find via secondary market at 3 or 4x the price...

boroboro4 2y ago

H100 has 3958 TFLOPS sparse fp8 compute. I’m pretty sure listed tflops for 5090 are sparse (and probably) fp4/int4.

rfoo 2y ago
Yes, that's the case. Check the (partial) spec of 5090 D, which is the nerfed version for export to China. It is marketed as having 2375 "AI TOPS". BIS demands it to be less than $4800 TOPS \times Bit-Width$, and the most plausible explanation for the number is - 2375 sparse fp4/int4 TOPS, which means 1187.5 dense TOPS for 4 bit, or $4750 TOPS \times Bit-Width$.
boroboro4 2y ago
And just for the context RTX 4090 has 2642 sparse int4 TOPS, so it’s about 25% increase

bee_rider 2y ago

How well do these models do at parallelizing across multiple GPUs? Is spending $4k on the 5090 a good idea for training, slightly better performance for much cheaper? Or a bad idea, 0x as good performance because you can’t fit your 60GB model on the thing?

Havoc 2y ago

> regular consumers will have trouble buying 5090s. They’re not really supposed to either judging by how they priced this. For non AI uses the 5080 is infinitely better positioned

kllrnohj 2y ago
> For non AI uses the 5080 is infinitely better positioned ...and also slower than a 4090. Only the 5090 got a gen/gen upgrade in shader counts. Will have to wait for benchmarks of course, but the rest of the 5xxx lineup looks like a dud
- Havoc 2y ago
  You’re comparing different price classes. New 5080 is 999 usd. Even secondhand 4090s from questionable eBay sellers are substantially more