4 ms·
Why is memory stuck at such low values while applications clearly demand more?
by amelius 2y ago
Why is memory stuck at such low values while applications clearly demand more?
- mckirk 2y agoMy guess would be 'artificial scarcity for the purpose of market segmentation', because people probably wouldn't buy that many of the expensive professional cards if the consumer cards had a ton of VRAM.
- amelius 2y agoOk, time to start supporting another brand folks.
- cubefox 2y agoNvidia indeed does this with the *60 cards, which are limited to 8 GB. They probably copied this upselling strategy from Apple laptops.
- nacs 2y agoExcept now, Apple with it's shared VRAM/RAM model now has better deals especially past 24GB of VRAM than you get with Nvidia now (for inference at least). A Macbook or Mac Mini with 32GB as a whole system is now cheaper than a 24GB Nvidia card.
- cubefox 2y agoThat's an interesting point about unified memory. But I assume most people will use their graphics card for video games rather than machine learning inference. And most non-console games are programmed for Windows and non-unified memory.
- YetAnotherNick 2y agoHBM are in very limited supply and NVidia tries to buy all the stock it could find at any price[1][2]. So the memory literally couldn't be increased. [1]: https://www.nextplatform.com/2024/02/27/he-who-can-pay-top-dollar-for-hbm-memory-controls-ai-training/ https://www.nextplatform.com/2024/02/27/he-who-can-pay-top-d... [2]: https://www.reuters.com/technology/nvidia-clears-samsungs-hbm3-chips-use-china-market-processor-sources-say-2024-07-23/#:~:text=In%20contrast%20to%20Samsung%2C%20SK,will%20supply%20Nvidia%20with%20HBM3E. https://www.reuters.com/technology/nvidia-clears-samsungs-hb...
- WithinReason 2y agohow about GDDR?
- YetAnotherNick 2y agoIf bandwidth is not the issue, you could directly use system memory via PCIe[1]. No need for on chip memory. [1]: https://developer.nvidia.com/gpudirect https://developer.nvidia.com/gpudirect
- WithinReason 2y agobandwidth is almost always is an issue, but not enough to be worth buying dedicated HW for it.
- karmakaze 2y agoI'm surprised the RTX cards don't have a Terms of Use that prohibits running CUDA on them. They already removed NVLink from the 40-series onward. Maybe running 8k VR could use the 32GB on the 5090 but I can't imagine much else that's not compute. I'm looking forward to newer APUs with onboard 'discrete' GPUs and quad or more channel LPDDR5X+ and 128GB+ unified memory that costs less than an M3 Ultra.
- sokoloff 2y agoApplications that consumers use (games and desktop) work fine with the amount of memory that consumer GPUs have. GPUs targeting more RAM-hungry applications exist, but they’re quite a bit more expensive, so people who play games buy gaming GPUs while people who need more VRAM buy cards targeting that application. Why would a consumer want to pay for 40GB of VRAM if 12GB will do everything they need?
- Const-me 2y ago> work fine with the amount of memory that consumer GPUs have Most consumers buy GPUs to play videogames. Recently, nVidia launched two flavors of 5060 Ti consumer GPU with 8GB and 16GB memory, the cards are otherwise identical. Apparently, the 8GB version is only good for 1080p resolution with no DLSS. In many games, the difference between these versions is very substantial. The article says 8GB version is deprecated right at launch: https://videocardz.com/newz/nvidia-geforce-rtx-5060-ti-with-8gb-memory-is-far-behind-16gb-model-in-gaming-performance-review-shows https://videocardz.com/newz/nvidia-geforce-rtx-5060-ti-with-...
- sokoloff 2y agoIt looks like the 8GB cards are about $60 (10-12%) cheaper than the 16GB cards. I sure don't want a world where we only have 32GB 5090s and nVidia reaching farther down the price-vs-performance curve to offer consumers a more affordable (but lower performing) choice seems like a good, rather than a bad, thing to me. (I genuinely don't see the controversy here.)
- Const-me 2y ago> 8GB cards are about $60 (10-12%) cheaper nVidia says the launch prices for them are $400 and $500, respectively. > seems like a good, rather than a bad, thing to me The bad thing is the most affordable current generation card capable of playing modern videogames in decent quality now costs $500. That’s too expensive for most consumers I’m afraid. Steam hardware survey says 7 out of 10 most popular cards are nVidia [234]060 which were sold for around $300. Despite most of them also have 8GB VRAM, when consumers bought these cards a few years ago, 8 GB was good enough for videogames released at that time.
- os2warpman 2y agoThere are supply constraints at almost every single step in the GPU supply chain. An earthquake three months ago, production issues, and insatiable demand mean that every single GDDR/HBM chip being made at factories already operating at maximum capacity has been sold to a waiting customer. If Nvidia wanted to double the amount of VRAM on their products, the only thing that would happen is the supply of finished products would be halved. No amount of money can fix it, only time.