4 ms·

In this scale, is CUDA support etc be a big factor in whether cloud providers buy this?

vessenes 2y ago
I'm an interested observer, not someone with direct knowledge. Jensen says: "yes" -- he pitches that NVIDIA has spent 20 years building a quality integrated software / hardware ecosystem that gets more out of each watt / process node than competitors. George Hotz and friends say: "meh" -- they propose at tinygrad that you could decompose deep learning tasks into a small enough subset that you can efficiently target ROCm in a few microkernels. In practice at datacenter levels, it seems like NV is still king - certainly in what's sold, but also at support in the stack. That said, these are large, sophisticated tech companies that are purchasers - MS, Apple, Google. These groups do not want a monopoly hardware provider for a key business function, they want at least two, ideally three hardware vendors who will then become 'commodity' providers, leaving margin back where it belongs (ahem). I don't think AMD has played their cards right in this area for the last 10 years, but they have done well as a company overall, and have good relationships / success in the server datacenter space, and therefore very good distribution. I would think if they want to win they would be discounting these more heavily and putting like 10x the people on software support flowing through to contributing upstream to TF/Pytorch, sure, even tinygrad, why not. I don't think they're doing that so far, which to me means they're counting on getting a lot of support as a 'break nvidia' contender. EDIT: one more point, at these purchase scales, 'tape out your own' can look attractive, as well, or 'buy Groq'. The spend here is pretty staggering, Manhattan project-grade spend. So, there are some unusual alternatives for the large cloud monopolies which makes for an interesting strategic problem space. Right now the short answer to your question is "Nobody ever got fired for buying NVIDIA". That'll probably be true for another few cycles is my guess.

I'm an interested observer, not someone with direct knowledge. Jensen says: "yes" -- he pitches that NVIDIA has spent 20 years building a quality integrated software / hardware ecosystem that gets more out of each watt / process node than competitors. George Hotz and friends say: "meh" -- they propose at tinygrad that you could decompose deep learning tasks into a small enough subset that you can efficiently target ROCm in a few microkernels. In practice at datacenter levels, it seems like NV is still king - certainly in what's sold, but also at support in the stack. That said, these are large, sophisticated tech companies that are purchasers - MS, Apple, Google. These groups do not want a monopoly hardware provider for a key business function, they want at least two, ideally three hardware vendors who will then become 'commodity' providers, leaving margin back where it belongs (ahem). I don't think AMD has played their cards right in this area for the last 10 years, but they have done well as a company overall, and have good relationships / success in the server datacenter space, and therefore very good distribution. I would think if they want to win they would be discounting these more heavily and putting like 10x the people on software support flowing through to contributing upstream to TF/Pytorch, sure, even tinygrad, why not. I don't think they're doing that so far, which to me means they're counting on getting a lot of support as a 'break nvidia' contender. EDIT: one more point, at these purchase scales, 'tape out your own' can look attractive, as well, or 'buy Groq'. The spend here is pretty staggering, Manhattan project-grade spend. So, there are some unusual alternatives for the large cloud monopolies which makes for an interesting strategic problem space. Right now the short answer to your question is "Nobody ever got fired for buying NVIDIA". That'll probably be true for another few cycles is my guess.