3 ms·
> I guess ML workloads are huge and here to stay. My only caution on that is that I think their moat on ML is much shallower than people think. nVidia has trad
by Tuna-Fish 4y ago
> I guess ML workloads are huge and here to stay.
My only caution on that is that I think their moat on ML is much shallower than people think. nVidia has traditionally maintained their position as the top dog in GPGPU by being first to provide good hardware and software, having markets lock in on that software, and then able to later sell more hardware at very good margins. Even when AMD (and others) later provide hardware that provides much better perf/cost for the same workload, at that point everyone already has a small mountain of tightly optimized cuda, and the cost/benefit of switching hardware providers just isn't there.
I really don't think this is going to work in ML. At their core, the ML algorithms are generally simple enough to fit on a blackboard, and are relatively much more easy to efficiently parallelize and optimize on diverse hardware. nVidia has again managed to capture most of a nascent market by being the first with adequate hardware and software to support it. But I think in a couple of years there are going to be a lot more competitors in this space, and unlike with previous GPGPU work, it will not be that painful for the end-user to shop around.
This won't mean that AMD (or someone else! this space is much easier to enter than GPUs in general) will capture the value from nVidia. Instead it will mean that the entire space will be commoditized and the margins will absolutely collapse.
- smoldesu 4y agoThe same should have been true about Bitcoin mining, but 32nm ASIC miners would get smoked by Nvidia cards in price, performance, and power efficiency. Nvidia's real power here is that they're the only company competing with Apple for the latest TSMC silicon. Their ability to drop 9 billion dollars on a foundry investment secured them the 4nm node, which is their bid for future-proofing their products. So far, it's worked.
- jdaxe 4y agoTotally agree with your sentiment. I think the inertia of switching is lower for ML because libraries like pytorch support more backends besides just CUDA, such as ROCm (AMD) and MPS (Apple Metal). I used the pytorch ROCm backend recently and there's certainly some more work that needs to be done in this space to get it to the same level as the CUDA backend (performance and compatibility wise), but at least it makes other GPU vendors an option.
- startupsfail 4y agoConnectivity at the datacenter level can’t be solved that easily. NVIDIA built its switch chip to connect GPUs and acquired MLNX.
- lmeyerov 4y agoThis. Most ML people we work with, including new hires, are CPU-first and don't think in bandwidth/throughput, so we have to untrain basically every DS hire to this. Nvidia SW+HW is so far ahead here as they got it early. Sure, Intel will pay startups to port individual algs to make some steps do well, but the ecosystem gap is just getting bigger. AMD/Intel/etc are still playing catch-up to status quo from years ago, while we work with Nvidia on next-gen and making accessible. Ex: GPUDirect over parallel SSD arrays is nuts.. and you can do it in python! Only really Google is competitive here (limited to ~TF), and maybe one day Apple.