4 ms·

Unless Sam gets his 7T I can’t see the data center growth continuing on pace. I still would love to see a consumer card with 128GB of VRAM

by tmaly 3y ago

samspenc 3y ago
I would also love to see a consumer GPU with a lot of VRAM. I think 128 GB would be too expensive BUT I'm hoping the Nvidia 50 series comes with 32+ GB VRAM for at least the top tier cards of the family (today they max out at 24 GB VRAM in the 3090 and 4090.)
- outside415 3y ago
  imagine how impossibly sold out those cards would be.
  - ls612 3y ago
    4090s are only sold out because of the China sanctions and are moving back towards MSRP. Fundamentally even a 48GB desktop card would be no substitute for a B100.
    - outside415 3y ago
      yeah but if you had 4090s with H100 equiv VRAM (prob don't even need HBM equiv) that wouldn't require quantizing larger models you would end up with a ton of LLM workstations at a fraction of what they charge for H100s. The demand would be high. What China was doing with the 4090s requires a great deal of technical/manufacturing prowess that doesn't apply to most.
whalesalad 3y ago
I would love to see a rethinking of desktop hardware. GPU's are huge. They used to be dinky little add-on cards ... but at this point we almost need to have a dedicated mainboard for them. Imagine this - instead of a motherboard mounted to the back of the case, it is mounted in the center. Behind it would be another motherboard, with a socket for a GPU and ram slots for memory. Kinda like a right/left brain connected with a high-speed interconnect. You open the left side of the case to access CPU and its memory, you open the right side of the case to access the GPU and its memory. You would be able to upgrade your GPU chip independently of everything else. It wouldn't be mounted sideways in a PCI slot where additional brackets and support arms are needed. You could upgrade the RAM independently. Cooler, etc.
- vizzier 3y ago
  What you're describing is fairly close to Intel's Beast Canion NUC [0] from a couple of years back. The Mobo and GFX essentially sit back to back. Though intel seemed to envisage upgrading the whole "CPU side" as a single piece (including ram) which probably, along with its high price, put off buyers. Following on from that though, there are a number of miniitx cases that followed on from that type of layout such as the fractal terra [1] which utilize a pcie riser to pass it rearward to the graphics card behind the motherboard. ATX/ITX wasn't entirely designed for it, but the layout does seem to be quite elegant. [0] https://www.tomshardware.com/reviews/nuc-11-extreme-kit-beast-canyon https://www.tomshardware.com/reviews/nuc-11-extreme-kit-beas... [1] https://www.fractal-design.com/products/cases/terra/terra/terra-jade/ https://www.fractal-design.com/products/cases/terra/terra/te...
outside415 3y ago
based on what I've seen from IDC and Gartner in fact it will keep growing for another 2-3 years in insane fashion at least.
seec 3y ago
There is no consumer use case for that so basically inexistent market. It would be rather dumb of Nvidia (or any other for that matter) to willingly cut its entreprise margins for such a non product. The only one doing lots of VRAM are Apple and nobody really care because performance is not that competitive and price is about as bad as entreprise stuff. I still think modular arch will stay for consumers products there much more benefits to that than chasing a bit more perf for low volume products. Apple can get away wit their shenanigans because they subsidize their high-end chips with the iPhone. If they could only sell Macs it would extremely unsustainable (like the PowerPC venture showed).
bicepjai 3y ago
There is one. Get Apple M3 with 128GB ram :)

Unless Sam gets his 7T I can’t see the data center growth continuing on pace. I still would love to see a consumer card with 128GB of VRAM

by tmaly 3y ago

Unless Sam gets his 7T I can’t see the data center growth continuing on pace. I still would love to see a consumer card with 128GB of VRAM

samspenc 3y ago

I would also love to see a consumer GPU with a lot of VRAM. I think 128 GB would be too expensive BUT I'm hoping the Nvidia 50 series comes with 32+ GB VRAM for at least the top tier cards of the family (today they max out at 24 GB VRAM in the 3090 and 4090.)

outside415 3y ago
imagine how impossibly sold out those cards would be.
- ls612 3y ago
  4090s are only sold out because of the China sanctions and are moving back towards MSRP. Fundamentally even a 48GB desktop card would be no substitute for a B100.
  - outside415 3y ago
    yeah but if you had 4090s with H100 equiv VRAM (prob don't even need HBM equiv) that wouldn't require quantizing larger models you would end up with a ton of LLM workstations at a fraction of what they charge for H100s. The demand would be high. What China was doing with the 4090s requires a great deal of technical/manufacturing prowess that doesn't apply to most.

whalesalad 3y ago

I would love to see a rethinking of desktop hardware. GPU's are huge. They used to be dinky little add-on cards ... but at this point we almost need to have a dedicated mainboard for them. Imagine this - instead of a motherboard mounted to the back of the case, it is mounted in the center. Behind it would be another motherboard, with a socket for a GPU and ram slots for memory. Kinda like a right/left brain connected with a high-speed interconnect. You open the left side of the case to access CPU and its memory, you open the right side of the case to access the GPU and its memory. You would be able to upgrade your GPU chip independently of everything else. It wouldn't be mounted sideways in a PCI slot where additional brackets and support arms are needed. You could upgrade the RAM independently. Cooler, etc.

vizzier 3y ago
What you're describing is fairly close to Intel's Beast Canion NUC [0] from a couple of years back. The Mobo and GFX essentially sit back to back. Though intel seemed to envisage upgrading the whole "CPU side" as a single piece (including ram) which probably, along with its high price, put off buyers. Following on from that though, there are a number of miniitx cases that followed on from that type of layout such as the fractal terra [1] which utilize a pcie riser to pass it rearward to the graphics card behind the motherboard. ATX/ITX wasn't entirely designed for it, but the layout does seem to be quite elegant. [0] https://www.tomshardware.com/reviews/nuc-11-extreme-kit-beast-canyon https://www.tomshardware.com/reviews/nuc-11-extreme-kit-beas... [1] https://www.fractal-design.com/products/cases/terra/terra/terra-jade/ https://www.fractal-design.com/products/cases/terra/terra/te...

outside415 3y ago

based on what I've seen from IDC and Gartner in fact it will keep growing for another 2-3 years in insane fashion at least.

seec 3y ago

There is no consumer use case for that so basically inexistent market. It would be rather dumb of Nvidia (or any other for that matter) to willingly cut its entreprise margins for such a non product. The only one doing lots of VRAM are Apple and nobody really care because performance is not that competitive and price is about as bad as entreprise stuff. I still think modular arch will stay for consumers products there much more benefits to that than chasing a bit more perf for low volume products. Apple can get away wit their shenanigans because they subsidize their high-end chips with the iPhone. If they could only sell Macs it would extremely unsustainable (like the PowerPC venture showed).

bicepjai 3y ago

There is one. Get Apple M3 with 128GB ram :)