5 ms·
Has AMD completely missed the ML/AI train? I’m quite surprised that even for inference, there doesn’t seem to be a viable competitor to nvidia. Is there anythin
by Karupan 3y ago
Has AMD completely missed the ML/AI train? I’m quite surprised that even for inference, there doesn’t seem to be a viable competitor to nvidia. Is there anything in AMD’s roadmap to suggest they are planning to even compete?
- cavisne 3y agoThe "even for inference" thing has turned into a bit of a trap imo. Data parallel models scaled up for training and then could run on individual chips, but these massive model parallel models require a couple of chips directly linked together even to do inference. So the idea that a competitor could come in with a simple, cheap inference chip doesn't really work.
- jiggawatts 3y agoTheir upcoming AMD MI300 APU looks very competitive with NVIDIA's GH200. The problem remains the lack of software support.
- rapsey 3y agoAMD is bad at software and that is not changing so they will always be behind.
- dsign 3y agoI agree with this. Even their GPU drivers for gaming are bad, they get stuck at “basic” things like MPEG encoding. But I don’t understand why software is a problem for them with their deep pockets. It can’t possibly be dearth of talent, or that it is expensive. Here in Europe a good software engineer earns half of what a mediocre software engineer earns in USA, to say nothing of India or China. They could just hire a bunch of teams and up their software game.
- rapsey 3y agoHardware development is very different to software development. AMD is a hardware company and they probably run their software development badly.
- singhrac 3y agoIntel is much closer with Habana Gaudi2. Sometimes I’m unsure whether they even recognize it. They also published MLPerf results today: https://habana.ai/blog/gaudi2-demonstrates-competitive-llm-performance-on-industry-benchmark-mlperf https://habana.ai/blog/gaudi2-demonstrates-competitive-llm-p... With 384 Gaudi2s they did the LLM task in 312 minutes, compared to 46 minutes for 768 H100s. It’ll come down to cost, but given the H100 is a process node or two ahead (and much more expensive I imagine?), Intel is actually much closer than I had realized. They’re the only other to submit an MLPerf result for the LLM task, I think. All credit of course to Habana Labs which was acquired by Intel.
- Karupan 3y agoAs a ML outsider, this was the first I’ve ever heard of it. Obviously I’m not the target audience, but I’m just shocked that I didn’t even know Intel was in the game.
- Dalewyn 3y agoIntel's going big in their own way by putting "AI" accelerators and the like in their latest and future processors, kind of need to be living under a rock to just miss it in these kinds of tech circles. But that said, Intel suffers from the same fundamental problem as AMD: They aren't Nvidia and they can't CUDA.
- singhrac 3y agoI’d argue you don’t need CUDA if PyTorch runs well on your platform, which is what everyone is trying to show with these MLPerf results. I think Intel’s strategy with oneAPI is not bad, it’s just late.
- Karupan 3y agoI do live under a rock when it comes to Intel I guess, since I haven't really looked into buying anything with their processor in 7 years :)
- varispeed 3y ago