5 ms·

I wouldn't be shocked if the 5090 is also the only one with 24GB. Seems like NVidia is trying their hardest to suppress memory increases.

by Salgat 3y ago

anonym29 3y ago
When you can sell an enterprise-grade card with 40-80GB of VRAM for $50k, selling consumer cards with 24GB for $2k is almost a form of charity, by comparison. AMD and Intel GPUs do not have the software ecosystem for AI workloads that Nvidia does, though AMD is rapidly improving. Nvidia has had an effective monopoly on the AI hardware space for the last year or so, and continues to have an effective near-monopoly, but that won't last forever as AMD and Intel catch up. The VRAM is one of the largest differentiators of their cards. Sufficient VRAM allows you to run huge LLMs like 65B in-memory, which is orders of magnitudes faster than system RAM + CPU. Smaller amounts of VRAM require swapping between VRAM and system RAM and incur a major performance penalty. Businesses are fighting to fork over $50k+/card for 40/80GB cards with the same processor as the 24GB consumer cards - it doesn't make economic sense for Nvidia to offer more on the consumer cards, lest they start cannibalizing demand for the enterprise cards.
- dragonwriter 3y ago
  > When you can sell an enterprise-grade card with 40-80GB of VRAM for $50k ADA 6000 RTX (48GB) — an enterprise (workstation) cars — is about $10K sticker price. The differentiator between those and data center cards in the 40GB-80GB range is, obviously, not just VRAM.
  - samspenc 3y ago
    And yet if you look at Blender (open source & free) GPU benchmarks https://opendata.blender.org/benchmarks/query/?compute_type=OPTIX&compute_type=CUDA&group_by=device_name&blender_version=4.0.0 https://opendata.blender.org/benchmarks/query/?compute_type=... The RTX ADA 6000 actually slightly performs worse than a consumer-grade 4090 (~10% less performant), even though it retails for 4-5x more ($8000 for ADA 6000 compared to $1500-2000 for a 4090).
  - anonym29 3y ago
    Check out what H100's are going for. Also, as samspenc points out, that RTX 6000 is using the same AD102 chip found in a consumer RTX 4090, just with marginally more CUDA cores, TMUs, ROPs, etc. The most substantial difference between the two is that the $10k card has twice the VRAM of the $2k card. I know it sounds outrageously oversimplified, but Nvidia can indeed more or less print money by attaching a few extra memory chips to what is otherwise a flagship consumer-oriented graphics processor. The profit margins on the H100, for reference, are estimated to be around one thousand percent (1000%), i.e., they sell for ~10x as much as it costs Nvidia to make them, and demand for them still grossly exceeds the total supply. See: https://the-decoder.com/nvidias-h100-gpu-sells-like-hot-cakes-with-high-profit-margins/ https://the-decoder.com/nvidias-h100-gpu-sells-like-hot-cake...
sevagh 3y ago
I've been hoping for a >24 GB ADA Titan consumer/gamer/retail card for a long time. The 4090 is awesome but I don't want the same VRAM as my 3090.
bbatha 3y ago
Because the 5090 ostensibly targets gaming which only needs sufficient vram to display images on 4k textures on 4k, 5k and ultra-wide monitors. A large portion of the 5090 audience is not doing ML training and that vram would sit idle for a few monitor generations. As a gamer I would be kind of upset if they included that very expensive unneeded vram in their already very expensive cards.
- Sohcahtoa82 3y ago
  As someone who is a gamer but also wants to dabble in ML, it kind of hurts tbh that this is probably true. The xx90 cards (3090, 4090, etc) have always been aimed at the gamer with a lot of cash. But games aren't really designed with that hardware in mind, so you won't see games that will take advantage of that much VRAM, so NVIDIA isn't inclined to increase the memory on them. I have a 3080 right now, so haven't really been able to play around with training a model, but I'd like to see the 5090 have 32 GB, but I'm having my doubts.
paulmd 3y ago
I think it strongly depends on whether 24gbit GDDR7 is ready to go by that point or not. If so, they'll do 36GB.
two_in_one 3y ago
5880 with 48GB has been announced. but I suspect it will be not in consumers range, more like $3k++.
- Salgat 3y ago
  The 5880 is the same generation as the 4090 and is a workstation card (it has higher memory but lower CUDA cores available). The price is expected to be around $5-6k. What I mean by 5000 series is the non-workstation next-gen lineup.

I wouldn't be shocked if the 5090 is also the only one with 24GB. Seems like NVidia is trying their hardest to suppress memory increases.

by Salgat 3y ago

I wouldn't be shocked if the 5090 is also the only one with 24GB. Seems like NVidia is trying their hardest to suppress memory increases.

anonym29 3y ago

When you can sell an enterprise-grade card with 40-80GB of VRAM for $50k, selling consumer cards with 24GB for $2k is almost a form of charity, by comparison. AMD and Intel GPUs do not have the software ecosystem for AI workloads that Nvidia does, though AMD is rapidly improving. Nvidia has had an effective monopoly on the AI hardware space for the last year or so, and continues to have an effective near-monopoly, but that won't last forever as AMD and Intel catch up. The VRAM is one of the largest differentiators of their cards. Sufficient VRAM allows you to run huge LLMs like 65B in-memory, which is orders of magnitudes faster than system RAM + CPU. Smaller amounts of VRAM require swapping between VRAM and system RAM and incur a major performance penalty. Businesses are fighting to fork over $50k+/card for 40/80GB cards with the same processor as the 24GB consumer cards - it doesn't make economic sense for Nvidia to offer more on the consumer cards, lest they start cannibalizing demand for the enterprise cards.

dragonwriter 3y ago
> When you can sell an enterprise-grade card with 40-80GB of VRAM for $50k ADA 6000 RTX (48GB) — an enterprise (workstation) cars — is about $10K sticker price. The differentiator between those and data center cards in the 40GB-80GB range is, obviously, not just VRAM.
- samspenc 3y ago
  And yet if you look at Blender (open source & free) GPU benchmarks https://opendata.blender.org/benchmarks/query/?compute_type=OPTIX&compute_type=CUDA&group_by=device_name&blender_version=4.0.0 https://opendata.blender.org/benchmarks/query/?compute_type=... The RTX ADA 6000 actually slightly performs worse than a consumer-grade 4090 (~10% less performant), even though it retails for 4-5x more ($8000 for ADA 6000 compared to $1500-2000 for a 4090).
- anonym29 3y ago
  Check out what H100's are going for. Also, as samspenc points out, that RTX 6000 is using the same AD102 chip found in a consumer RTX 4090, just with marginally more CUDA cores, TMUs, ROPs, etc. The most substantial difference between the two is that the $10k card has twice the VRAM of the $2k card. I know it sounds outrageously oversimplified, but Nvidia can indeed more or less print money by attaching a few extra memory chips to what is otherwise a flagship consumer-oriented graphics processor. The profit margins on the H100, for reference, are estimated to be around one thousand percent (1000%), i.e., they sell for ~10x as much as it costs Nvidia to make them, and demand for them still grossly exceeds the total supply. See: https://the-decoder.com/nvidias-h100-gpu-sells-like-hot-cakes-with-high-profit-margins/ https://the-decoder.com/nvidias-h100-gpu-sells-like-hot-cake...

sevagh 3y ago

I've been hoping for a >24 GB ADA Titan consumer/gamer/retail card for a long time. The 4090 is awesome but I don't want the same VRAM as my 3090.

bbatha 3y ago

Because the 5090 ostensibly targets gaming which only needs sufficient vram to display images on 4k textures on 4k, 5k and ultra-wide monitors. A large portion of the 5090 audience is not doing ML training and that vram would sit idle for a few monitor generations. As a gamer I would be kind of upset if they included that very expensive unneeded vram in their already very expensive cards.

Sohcahtoa82 3y ago
As someone who is a gamer but also wants to dabble in ML, it kind of hurts tbh that this is probably true. The xx90 cards (3090, 4090, etc) have always been aimed at the gamer with a lot of cash. But games aren't really designed with that hardware in mind, so you won't see games that will take advantage of that much VRAM, so NVIDIA isn't inclined to increase the memory on them. I have a 3080 right now, so haven't really been able to play around with training a model, but I'd like to see the 5090 have 32 GB, but I'm having my doubts.

paulmd 3y ago

I think it strongly depends on whether 24gbit GDDR7 is ready to go by that point or not. If so, they'll do 36GB.

two_in_one 3y ago

5880 with 48GB has been announced. but I suspect it will be not in consumers range, more like $3k++.

Salgat 3y ago
The 5880 is the same generation as the 4090 and is a workstation card (it has higher memory but lower CUDA cores available). The price is expected to be around $5-6k. What I mean by 5000 series is the non-workstation next-gen lineup.