4 ms·
> NVidia has software moat for specialized applications but not for AI, which is responsible for most of their sales now. Almost everyone in AI uses pytorch/jax
by david-gpu 2y ago
> NVidia has software moat for specialized applications but not for AI, which is responsible for most of their sales now. Almost everyone in AI uses pytorch/jax/triton/flash attention and not CUDA directly
And what does pytorch et al. use under the hood? cuBLAS and cuDNN, proprietary libraries written by NVidia. That is where most of the heavy lifting is done. If you think that replicating the functionality and performance that these libraries provide is easy, feel free to apply for a job at NVidia or their competitors. It is pretty well paid.
- YetAnotherNick 2y agoDid you read the last part? Pytorch uses drivers, and drivers exists for Google's TPU and Apple's M1 GPU as well and both works pretty well. I have tested both and it reaches similar MFU as Nvidia.
- gajjanag 2y agoMaybe on a particular model/dataset but extremely unlikely in general. Again, like another commenter pointed out: if you truly believe it isn't that hard we would love to hire you at Meta ;)
- YetAnotherNick 2y agoYes some operations are not supported in MPS/TPU and falls back to slower CPU. But for common architectures like transformers and convnets, it works very well for all the datasets. I never claimed it was easy. I meant in my opinion it is in the order of 10s of millions dollars of investment, not a trillion dollar CUDA moat that people comment here.
- CharlieDigital 2y agoAre M1 GPUs available for data center deployment at scale? Are Google TPUs available outside of Google? Can Amazon or Microsoft or other third parties deploy them? Anyone that wants off the shelf parts at scale is going to turn to Nvidia.
- YetAnotherNick 2y agoThat's the point I am making. And the reason Amazon or Microsoft can't deploy them is the hardware, not CUDA.
- CharlieDigital 2y agoYeah, and if you're using Nvidia, you're using CUDA.
- Jlagreen 2y agoAnd because the world buys all the shelf stuff from Nvidia and uses CUDA, Nvidia get the largest debugging user base there is. This way Nvidia can continue to make CUDA even better and cycle repeats.
- Wytwwww 2y agoSo Nvidia's moat is mainly hardware and not software?