4 ms·
Those whales - probably major cloud vendors - likely have the resources to develop their own hardware at some point. Right now it’s “buy GPUs at any cost”. If
by softwaredoug 2y ago
Those whales - probably major cloud vendors - likely have the resources to develop their own hardware at some point.
Right now it’s “buy GPUs at any cost”. If things slow, there will be a chance for these customers to consider how to optimize this cost. NVIDIA can’t sit on its laurels like Intel did with x86.
- HDThoreaun 2y agoGoogle did this a decade ago. Amazon has a CPU but I think no GPU yet although Im sure its being worked on. The problem for them is the CUDA moat. Their hardware is mostly used for inference because no one trains on non-nvidia hardware.
- JoshTriplett 2y ago> The problem for them is the CUDA moat. Compared to the problem of developing a new cutting-edge GPU, building a CUDA compatibility layer is a much smaller problem. Hire the author of ZLUDA, throw a small team at it, and have a legal department on standby. And separately, there'd also be value in some source-translation projects to help people migrate to some better native framework.
- vidarh 2y agoBoth in terms of hardware and software, they need to be able to support the small set of operations that their training or inference uses, so the problem on both sides is much smaller than both a full GPU and a full CUDA replacement.
- codedokode 2y agoWhy support proprietary libraries? Isn't it better to make an open-source library. Also as for AMD, as I understand, they are unwilling to make ML libraries for consumer-grade GPUs and GPUs built into CPUs.
- JoshTriplett 2y agoAn Open Source compatibility layer for CUDA to allow people to run on non-NVIDIA GPUs is a first step; it removes the lock-in between GPU and library. Once people can run all their existing software on a different GPU, they can then consider adopting a better standard to build on top of. The reverse approach, of trying to entice people to move from CUDA to a different library and switch GPUs at the same time, has been tried repeatedly and has not yet succeeded. Trying something different seems warranted.
- Ekaros 2y agoI don't really understand how CUDA is proper moat... You can scale software engineers much more readily than hardware supply chain. And basically for your own models, it should not be impossible to train your staff to use your layer instead.
- HDThoreaun 2y agoIm inclined to agree, I dont think it's a sustainable moat. But AMD and intel have been trying to break through and have had minimal success so far. Until someone actually releases a CUDA competitor that is used the moat exists.
- Ekaros 2y agoCritical thing here in my mind it is not so much general moat, but lot of individual moats. Each of these companies investing billions can build their own. Easily. So most probable end result is that we end up with multiple competing alternatives all with their own vendor lock ins. And general public might be lucky to get one or two options.
- xnyan 2y ago>it should not be impossible to train your staff to use your layer instead. First, it's the classic chicken and egg problem. Why would you invest in a CUDA alternative when you're going to be using nvidia hardware anyway? Second, something can be not impossible but still quite difficult. As AMD and Intel have shown, creating a GPGPU API for your hardware that people want to use is not a trivial task and to date have not managed to do it. Lastly this must just be differences in our experiences with cooperate management, because mine has been that in general they would always prefer to spend on stuff over headcount if said stuff reduces the headcount required.
- tester756 2y ago>You can scale software engineers much more People do not scale.
- gpm 2y agoAmazon/AWS has Trainium: https://aws.amazon.com/machine-learning/trainium/ https://aws.amazon.com/machine-learning/trainium/
- yyyfb 2y agoWhat if having the resources to develop hardware are not the point? This is a physical business, and supply chain is the bottleneck at some point. Right now it seems that all the money in the world can't build fabs fast enough to manufacture alternatives to Nvidia's chips. As long as they maintain dominance over the supply chain, having developed equivalent technology might not matter. Someone correct me if this is wrong, I'm mostly speculating.
- I_AM_A_SMURF 2y agoI mean it's all TMSC at the end of the day. They have finite capacity that's all tied up for a while.
- Ekaros 2y agoThe future order book is the interesting part. Does Nvidia has as big share as they do now or has someone else paid more that is outbid them.