4 ms·
What’s baffling about all this is both AMD and Intel have competing offerings and those offerings see next to no traction in spite of being much more attractive
by ein0p 2y ago
What’s baffling about all this is both AMD and Intel have competing offerings and those offerings see next to no traction in spite of being much more attractive from the cost standpoint. I understand why they aren’t taking off on the training side: fragmentation is very counterproductive there. But why not deploy them in large quantities for inference at least? The effort of porting transformer based models is trivial for both vendors, and the performance is very competitive
- Clubber 2y agoCUDA is their API and is proprietary. It's very fast and it's a significant competitive advantage. https://analyticsindiamag.com/is-cuda-nvidias-competitive-moat https://analyticsindiamag.com/is-cuda-nvidias-competitive-mo...
- ein0p 2y agoIt really doesn’t matter if you have cuda or not if you’re going to run inference at scale. As I said above (speaking from experience), porting models for inference is not a technically difficult problem. Indeed with both Intel Gaudi or AMD MI series of accelerators a lot of the popular architectures and their derivatives are supported either out of the box or with minimal tweaks.
- joe_the_user 2y agoI think the problem the other players have is they don't actually want to compete head-on with Nvidia. AMD actually commissioned a drop-in CUDA emulator and we found out 'cause they stopped financing it and they open-sourced it as parts of the contract. I would speculate that no one actually wants a "clone wars" situation since it would commodify the GPU and reduce everyone's profit rate.
- acchow 2y agoThe companies buying billions of dollars of Nvidia GPUs definitely do want to commodify the GPU
- latchkey 2y agoAMD's only offering that matters, MI300x, is literally a couple months old. Give it time.
- StressedDev 2y agoThe question is what is the performance per watt of AMD's and Intel's products. My guess is both have significantly worse performance per watt. Energy and cooling are huge data center expenses and paying less for a product which requires more energy and cool is not a good idea because it costs more.
- ein0p 2y agoNope. Not for LLM workloads at least. They’re competitive across the board.