5 ms·
The outcome of Nvidia's monopoly hold on AI/GPU computing is that consumer level devices that might otherwise be perfectly effective for this sort of stuff are
by andrewstuart 3y ago
The outcome of Nvidia's monopoly hold on AI/GPU computing is that consumer level devices that might otherwise be perfectly effective for this sort of stuff are prevented by Nvidia from being used for such purposes.
If there was real competition -like two or more other suppliers on par in terms of capability - then artificially constraining devices just to plump up prices would not be a thing.
- psshank 3y agoThis is what we've done at Salad (www.salad.com). SaladCloud (officially launching on July 11th) has 10k+ Nvidia GPUs at the lowest prices in the market. Not conducive for training but for serving inferences at massive scale.
- MacsHeadroom 3y agoYour pricing is around 50-100% higher than vast.ai, a relatively mature competitor in this space who offers the same GPUs and more with beefier rigs. For example. An '8 core/16GB RAM/3080 Ti 12GB' is 50% more expensive on salad than on Vast. In fact, I can get more than double the resources for the same cost as Salad in the form of a '28core/32GB RAM/2x3080 Ti 12GB' on Vast. So, what is your differentiation? Why would I ever use Salad?
- smoldesu 3y ago> consumer level devices that might otherwise be perfectly effective for this sort of stuff are prevented by Nvidia from being used for such purposes. Citation? To the contrary, millions of consumer-level Nvidia customers have access to datacenter-grade HPC APIs because of their vertical integration. Nvidia's "monopoly hold" on GPGPU compute exists because the other competitors (eg. AMD and Apple) completely abandoned OpenCL. When the time came to build a successor, neither company ante'd up. So now we're here. CUDA is not a monopoly. If Apple or Microsoft wanted, they could start translating CUDA calls into native instructions for their own hardware. They don't though, because it would be an investment that doesn't make sense for their customers, costs tens of millions of dollars, and wouldn't meaningfully hurt Nvidia unless it was Open Source.
- vGPU 3y agoThe refusal to increase RAM in the 4xxx generation, the stuff they pulled with the 4080, cutting the 4060’s vram bandwidth. Etc.
- sva_ 3y agoRemoving SLI/NVLink
- nl 3y agoIt’s true they price partion their products basically perfectly (as in there isn't a magical good deal anywhere in the range). But these specific critisms don't really ring true. Lower VRAM bandwidth lets them use lower binned VRAM and there isn't really a need for more RAM than the 24G in the 4090 in gaming. The naming screw up they did with the 4080 was dumb and fortunately corrected quickly. But it doesn't seem related to the OPs point.
- YetAnotherNick 3y agoConsumer GPUs are very good for single GPU inference(or training small models), but Nvidia deliberately made GPU networking slower. eg 3090 has support for NVlink, but it was removed in 4090.
- smoldesu 3y agoSLI was never well-supported in the first place. It's a shame it's gone, but compared to multi-GPU tiling solutions I don't think its much better, at least for AI. Nvidia is certainly hostile to Open Source and not the kindest hardware vendor to boot, but that alone does not suffice a monopoly.
- girvo 3y ago> but that alone does not suffice a monopoly You're right. CUDA is what does.
- 3y ago