3 ms·

Manufacturers could have never anticipated that they could charge $1,500 for an accelerator on the one hand, and $15,000 for the same accelerator on the other.

Manufacturers could have never anticipated that they could charge $1,500 for an accelerator on the one hand, and $15,000 for the same accelerator on the other. They’re the same thing! CXL doesn’t really make sense. Does even NVLink? And yet, we have no more 2 slot GPUs, we lost NVLink on 4090s. There are no AM5 motherboards with 4 2 spaced PCIe 16x slots. Why are the consumer chips even limited for IO? It’s just a bunch of bullshit to segregate a market and extract more rents. We’re going backwards. That said end users are also to blame, for every 1 person building a PC to do something innovative there are 19 who buy stuff with the brightest RGBs. So it’s up to the market really. Individual consumers might spend an amortized $12k/y on their cars. It’s not outside the realm of reason that hardware manufacturers can take way more of the surplus they’ve been giving to end users. Whether you think this is good or bad is a subjective thing. I personally don’t think much innovation comes out of NVIDIA, they have somehow tricked the world that thousands of very simple processors, tiled neatly, is as sophisticated as a thing as a general purpose CPU, or that CUDA is especially innovative on its own and not riding on a huge amount of inertia. You should transfer all the surplus to the people taking the most risks, basically the people writing the software with CUDA, which once was rendering for CG and physics simulation and now is people trying to make ML models more efficient.

buildbot 3y ago
Manufacturers do segment the market, but also a 4090 is a totally different, cheaper piece of silicon than an H100. Not really just a price increase. HBM and the requisite packaging is very very expensive. Why does CXL not make sense? Coherency is great! NVLink is great for say 4x + gpus. With two 4090s , the effective speedup is the same as with two NVlinked A100s from my testing. So it’s really not a big deal it was pulled out. Consumers really really don’t build dual GPU systems anymore. Gaming stopped using SLI/crossfire over ten years ago. So AM5 and Intel’s 1700 socket are designed to that idea, and price point. Why is socket SP3 so insanely massive? All those PCIE lanes and extra memory bandwidth take pins. More costly boards, more costly socket, etc. Also, how are you going to cool 4x 4090s on air, when they are next to each other? That’s 1800w of heat!! How are you going to power it? That’s more than you average American house can power with a 15A 120v circuit! I just built a dual 4090 system, while I was waiting for my waterblocks to come in, I was testing everything on air. The second card getting fed hot air by the first eventually had to downclock to half the clock of the first card, in order to keep the GPU under 90c at 100% fans. The waterblocks are single slot and go into a server class SP3 epyc board with 4x 1 slot spaced pcie. I have a 1600w psu that has a special plug, and it sits on the special single plug circuit that’s for an AC. Nvidia is hugely innovative, and a 4090 core is really not simple at all. The cores were simple back in the 8800GTX days, kinda, now, you have pretty significant scheduling resources and tensor cores and different kinds of ALUs and a bonkers massive register file. CUDA is their secret sauce, way easier to use than openCL.
- jorvi 3y ago
  There are blower-style 4090s. If noise is not an issue, you could just run two of those together with some solid case fans.
  - buildbot 3y ago
    There are?
- doctorpangloss 3y ago
  A 4090 280W limit will perform similarly to its unlimited configuration on CUDA workloads. As another commenter said blower style fans are fine for this. The segmentation around NVLink is what makes this worthless. And anyway, they could equip it with 48GB of RAM, like the L40 part. It's all bullshit.
  - buildbot 3y ago
    Just tried this, it's 30% slower. 24GB of GGDR6X is $$$$. Especially if they are double density chips, to keep the RAM on a single side, important for cooling! Again, NVlink is not required for 99% of people.
c0balt 3y ago
> There are no AM5 motherboards with 4 2 spaced PCIe 16x slots. Why are the consumer chips even limited for IO? Because PCIe Slots, especially Gen 5, is a good chunk of the complexity/ cost of a consumer Motherboard. They are difficult to handle as they have quite hard limits ln timing and signal integrity requirements. Hitting these requirements for more full-width slots is harder and for most use cases just not feasible from a cost : usability perspective (how many extension cards can even saturate an 8x Gen 5 Bus?). This is also why PCIe Risers for newer PCIe generations are getting very expensive.

buildbot 3y ago

Manufacturers do segment the market, but also a 4090 is a totally different, cheaper piece of silicon than an H100. Not really just a price increase. HBM and the requisite packaging is very very expensive. Why does CXL not make sense? Coherency is great! NVLink is great for say 4x + gpus. With two 4090s , the effective speedup is the same as with two NVlinked A100s from my testing. So it’s really not a big deal it was pulled out. Consumers really really don’t build dual GPU systems anymore. Gaming stopped using SLI/crossfire over ten years ago. So AM5 and Intel’s 1700 socket are designed to that idea, and price point. Why is socket SP3 so insanely massive? All those PCIE lanes and extra memory bandwidth take pins. More costly boards, more costly socket, etc. Also, how are you going to cool 4x 4090s on air, when they are next to each other? That’s 1800w of heat!! How are you going to power it? That’s more than you average American house can power with a 15A 120v circuit! I just built a dual 4090 system, while I was waiting for my waterblocks to come in, I was testing everything on air. The second card getting fed hot air by the first eventually had to downclock to half the clock of the first card, in order to keep the GPU under 90c at 100% fans. The waterblocks are single slot and go into a server class SP3 epyc board with 4x 1 slot spaced pcie. I have a 1600w psu that has a special plug, and it sits on the special single plug circuit that’s for an AC. Nvidia is hugely innovative, and a 4090 core is really not simple at all. The cores were simple back in the 8800GTX days, kinda, now, you have pretty significant scheduling resources and tensor cores and different kinds of ALUs and a bonkers massive register file. CUDA is their secret sauce, way easier to use than openCL.

jorvi 3y ago
There are blower-style 4090s. If noise is not an issue, you could just run two of those together with some solid case fans.
- buildbot 3y ago
  There are?
doctorpangloss 3y ago
A 4090 280W limit will perform similarly to its unlimited configuration on CUDA workloads. As another commenter said blower style fans are fine for this. The segmentation around NVLink is what makes this worthless. And anyway, they could equip it with 48GB of RAM, like the L40 part. It's all bullshit.
- buildbot 3y ago
  Just tried this, it's 30% slower. 24GB of GGDR6X is $$$$. Especially if they are double density chips, to keep the RAM on a single side, important for cooling! Again, NVlink is not required for 99% of people.

c0balt 3y ago

> There are no AM5 motherboards with 4 2 spaced PCIe 16x slots. Why are the consumer chips even limited for IO? Because PCIe Slots, especially Gen 5, is a good chunk of the complexity/ cost of a consumer Motherboard. They are difficult to handle as they have quite hard limits ln timing and signal integrity requirements. Hitting these requirements for more full-width slots is harder and for most use cases just not feasible from a cost : usability perspective (how many extension cards can even saturate an 8x Gen 5 Bus?). This is also why PCIe Risers for newer PCIe generations are getting very expensive.