3 ms·
> Why are gpu vendors deciding how much ram the gpu gets? ...because they make them? You can go buy 256gb systems if you want, at extreme cost. You can also u
by smoldesu 3y ago
> Why are gpu vendors deciding how much ram the gpu gets?
...because they make them?
You can go buy 256gb systems if you want, at extreme cost. You can also use an 8gb card for whatever you want too. What are they doing wrong?
> To add complication, I really don’t want to deal with copying 256GB models between cpu/card.
For deployment, it doesn't really matter. You just memory-map it to the CPU or keep it hot-loaded in GPU memory. With a unified memory model you're still limited by your disk speed, so the performance difference probably wouldn't come out great either way.
- lumost 3y agoIntel makes microchips, but they do not control how much memory I put in them. There are no 256 GB GPUs available today outside of multi-GPU setups. Would love to see giant NUMA GPU boards.
- smoldesu 3y ago> Would love to see giant NUMA GPU boards. To my knowledge, this is pretty much how InfiniBand works on datacenter-scale systems like the GH200. The incentive and upsell for bringing this to consumer systems seems weak to me.