8 ms·

> consumer level devices that might otherwise be perfectly effective for this sort of stuff are prevented by Nvidia from being used for such purposes. Citation

> consumer level devices that might otherwise be perfectly effective for this sort of stuff are prevented by Nvidia from being used for such purposes. Citation? To the contrary, millions of consumer-level Nvidia customers have access to datacenter-grade HPC APIs because of their vertical integration. Nvidia's "monopoly hold" on GPGPU compute exists because the other competitors (eg. AMD and Apple) completely abandoned OpenCL. When the time came to build a successor, neither company ante'd up. So now we're here. CUDA is not a monopoly. If Apple or Microsoft wanted, they could start translating CUDA calls into native instructions for their own hardware. They don't though, because it would be an investment that doesn't make sense for their customers, costs tens of millions of dollars, and wouldn't meaningfully hurt Nvidia unless it was Open Source.

vGPU 3y ago
The refusal to increase RAM in the 4xxx generation, the stuff they pulled with the 4080, cutting the 4060’s vram bandwidth. Etc.
- sva_ 3y ago
  Removing SLI/NVLink
- nl 3y ago
  It’s true they price partion their products basically perfectly (as in there isn't a magical good deal anywhere in the range). But these specific critisms don't really ring true. Lower VRAM bandwidth lets them use lower binned VRAM and there isn't really a need for more RAM than the 24G in the 4090 in gaming. The naming screw up they did with the 4080 was dumb and fortunately corrected quickly. But it doesn't seem related to the OPs point.
YetAnotherNick 3y ago
Consumer GPUs are very good for single GPU inference(or training small models), but Nvidia deliberately made GPU networking slower. eg 3090 has support for NVlink, but it was removed in 4090.
- smoldesu 3y ago
  SLI was never well-supported in the first place. It's a shame it's gone, but compared to multi-GPU tiling solutions I don't think its much better, at least for AI. Nvidia is certainly hostile to Open Source and not the kindest hardware vendor to boot, but that alone does not suffice a monopoly.
  - girvo 3y ago
    > but that alone does not suffice a monopoly You're right. CUDA is what does.
    - nl 3y ago
      AMD has know about CUDA for decades. Everyone has been begging AMD to put resources into AI since 2014. AMD doesn't care and it's difficult to blame that on NVIDIA.
      - girvo 3y ago
        I don’t blame Nvidia. I think caring about blame when discussing this topic is missing the forest for the trees anyway. The bigger problem is that Nvidia can and will abuse their monopoly power. Whether it’s Nvidias fault or not, once upon a time governments would step in when that happened. Thing is, I don’t think that solution even exists for something as high tech as GPGPU stacks and vertical integration. I don’t see a solution at all really, which worries me a little in the medium-long term.
    - smoldesu 3y ago
      How? CUDA is a software API, any sufficiently motivated competitor could even legally re-impliment it for their hardware if they wanted to.
      - dragonwriter 3y ago
        Contrary to the dogma of certain politico-economic camps, a monopoly (the actual presence of market power and absence of substitution effect marking an absence of actual competition in some space) can exist without competition being illegal. So, “Any…competitor could even legally re-implement [CUDA] for their hardware” is not a counter-argument to CUDA being the basis for an actual existing monopoly. It might be an argument that, to the extent that that is the sole basis for the monopoly, the monopoly is unlikely to be a long-term stable condition, but its not a counterargument to it existing.
joe_the_user 3y ago
While OpenCL was simply not equivalent to CUDA, I think you're correct that those other enterprises (Apple, AMD and similar) that could challenge Nvidia on the high-end GPU front simply choose not to. The thing is, the reason is if there was competition in this market, prices would sink much closer to costs and no one would be making bank whereas a large enterprise would want a higher return. Also, a consumer-grade GPU can used for neural net training at the researcher level but large corporate use requires H100/A100 and that is what's getting traction.
- smoldesu 3y ago
  > prices would sink much closer to costs and no one would be making bank For Apple and AMD, that's not really a problem. Both of them drive considerable (40%+) margins on their products and can afford to drive things closer to the wire. I also think more competition here would be good (and I do love lower prices) but Nvidia charges more here because they know they can. It's value-based marketing that works, because their software APIs aren't vaporware. > large corporate use requires H100/A100 and that is what's getting traction. I guess... you really need a strict definition of "requires" for that to hold true. For every non-"competing with ChatGPT" application, you could probably train and deploy with consumer-grade cards. You're technically right here though, and it invites the conversation around what actually constitutes abusive market positioning. Nvidia's actions here really aren't much different than AMD and Intel separating their datacenter and PC product lines. It's a risky move from a "keeping both users happy" standpoint, but hardly anticompetitive.
  - joe_the_user 3y ago
    Both of them drive considerable (40%+) margins on their products and can afford to drive things closer to the wire. They could that - but the reason they command these margins is exactly because they don't do that. I think do something like for fairly some investment but producing products that would compete with Nvidia would require a significant percentage amount of capital for any company - those dealing with tens of billions of dollar chunks expect above commodity revenues.
l33tman 3y ago
Also, it is starting to become the case that CUDA isn't that important anymore, both pyTorch and TF have numerous other backends and the programmer doesn't need to know what it runs on. And the GGML project has shown that you can come a long way with a good CPU and large "normal" RAM and 4/8 bit weights, with no CUDA in sight. You can definitely enter this domain without having a full-fledged CUDA replacement from the start.

vGPU 3y ago

The refusal to increase RAM in the 4xxx generation, the stuff they pulled with the 4080, cutting the 4060’s vram bandwidth. Etc.

sva_ 3y ago
Removing SLI/NVLink
nl 3y ago
It’s true they price partion their products basically perfectly (as in there isn't a magical good deal anywhere in the range). But these specific critisms don't really ring true. Lower VRAM bandwidth lets them use lower binned VRAM and there isn't really a need for more RAM than the 24G in the 4090 in gaming. The naming screw up they did with the 4080 was dumb and fortunately corrected quickly. But it doesn't seem related to the OPs point.

YetAnotherNick 3y ago

Consumer GPUs are very good for single GPU inference(or training small models), but Nvidia deliberately made GPU networking slower. eg 3090 has support for NVlink, but it was removed in 4090.

smoldesu 3y ago
SLI was never well-supported in the first place. It's a shame it's gone, but compared to multi-GPU tiling solutions I don't think its much better, at least for AI. Nvidia is certainly hostile to Open Source and not the kindest hardware vendor to boot, but that alone does not suffice a monopoly.
- girvo 3y ago
  > but that alone does not suffice a monopoly You're right. CUDA is what does.
  - nl 3y ago
    AMD has know about CUDA for decades. Everyone has been begging AMD to put resources into AI since 2014. AMD doesn't care and it's difficult to blame that on NVIDIA.
    - girvo 3y ago
      I don’t blame Nvidia. I think caring about blame when discussing this topic is missing the forest for the trees anyway. The bigger problem is that Nvidia can and will abuse their monopoly power. Whether it’s Nvidias fault or not, once upon a time governments would step in when that happened. Thing is, I don’t think that solution even exists for something as high tech as GPGPU stacks and vertical integration. I don’t see a solution at all really, which worries me a little in the medium-long term.
  - smoldesu 3y ago
    How? CUDA is a software API, any sufficiently motivated competitor could even legally re-impliment it for their hardware if they wanted to.
    - dragonwriter 3y ago
      Contrary to the dogma of certain politico-economic camps, a monopoly (the actual presence of market power and absence of substitution effect marking an absence of actual competition in some space) can exist without competition being illegal. So, “Any…competitor could even legally re-implement [CUDA] for their hardware” is not a counter-argument to CUDA being the basis for an actual existing monopoly. It might be an argument that, to the extent that that is the sole basis for the monopoly, the monopoly is unlikely to be a long-term stable condition, but its not a counterargument to it existing.

joe_the_user 3y ago

While OpenCL was simply not equivalent to CUDA, I think you're correct that those other enterprises (Apple, AMD and similar) that could challenge Nvidia on the high-end GPU front simply choose not to. The thing is, the reason is if there was competition in this market, prices would sink much closer to costs and no one would be making bank whereas a large enterprise would want a higher return. Also, a consumer-grade GPU can used for neural net training at the researcher level but large corporate use requires H100/A100 and that is what's getting traction.

smoldesu 3y ago
> prices would sink much closer to costs and no one would be making bank For Apple and AMD, that's not really a problem. Both of them drive considerable (40%+) margins on their products and can afford to drive things closer to the wire. I also think more competition here would be good (and I do love lower prices) but Nvidia charges more here because they know they can. It's value-based marketing that works, because their software APIs aren't vaporware. > large corporate use requires H100/A100 and that is what's getting traction. I guess... you really need a strict definition of "requires" for that to hold true. For every non-"competing with ChatGPT" application, you could probably train and deploy with consumer-grade cards. You're technically right here though, and it invites the conversation around what actually constitutes abusive market positioning. Nvidia's actions here really aren't much different than AMD and Intel separating their datacenter and PC product lines. It's a risky move from a "keeping both users happy" standpoint, but hardly anticompetitive.
- joe_the_user 3y ago
  Both of them drive considerable (40%+) margins on their products and can afford to drive things closer to the wire. They could that - but the reason they command these margins is exactly because they don't do that. I think do something like for fairly some investment but producing products that would compete with Nvidia would require a significant percentage amount of capital for any company - those dealing with tens of billions of dollar chunks expect above commodity revenues.

l33tman 3y ago

Also, it is starting to become the case that CUDA isn't that important anymore, both pyTorch and TF have numerous other backends and the programmer doesn't need to know what it runs on. And the GGML project has shown that you can come a long way with a good CPU and large "normal" RAM and 4/8 bit weights, with no CUDA in sight. You can definitely enter this domain without having a full-fledged CUDA replacement from the start.