4 ms·
The AI race is so hot right now that AMD will sit in #2 and pick up all of the people who can't get access to NVIDIA cards. There is a huge centralization issue
by latchkey 3y ago
The AI race is so hot right now that AMD will sit in #2 and pick up all of the people who can't get access to NVIDIA cards. There is a huge centralization issue around only writing your code for NVIDIA... and that is a huge business risk. People are going to wake up to that fast as the supplies of NVIDIA go to zero.
That said, I don't think people realize that there is literally no more large scale tier 3 (redundant) data center power in the US. Even if you are sitting on 1000 or 10k NVIDIA cards, you can't deploy them anywhere.
They also need to be in the same data center for speed. You can't just colo in 2-3 data centers to get what you want. If you want to train a large model across 1000 gpus? You're screwed.
Think you can just go to the cloud? Go try to sign up for coreweave. They are full and not taking any more customers. A lot of the other sites advertising nvidia gpus are just reselling coreweave under the covers.
Forget the software problems. There are far bigger issues and they are not getting better any time soon.
- imtringued 3y agoAMD's AI cookie cutter business model is getting million dollar contracts with a handful of companies. The peasants don't get access to or use AMDs data center cards.
- latchkey 3y agoAnd why do you think that is?
- Aeolos 3y agoThe problem is AMD is not #2 in this field, not by a long shot. Google and Amazon offer their own hardware that is both better supported and actually available for customers. Apple has fast inference hardware in every computer and mobile device they offer. I raised this issue with AMD 8 years ago in a technology conference, and the answer I got back then was a shoulder shrug "we don't think this is an important market." 8 years later and they have all but lost the war.
- latchkey 3y agoG/A can't be used for large scale training, nobody is going to give their data to them. Major trust issues there. Apple is Apple. Not public. Let's also not mix consumer needs with enterprise. You're correct, 8 years ago and up until recently, AMD only cared about gamers. They are waking up fast though. ROCm 5.6 is a visible first step in that regard. MI300 will blow A/H100's out of the water. But again, hardware/software isn't the problem here. The problem is much deeper than that... even if you have those things resolved, you can't put them anywhere.
- jjoonathan 3y ago> Let's also not mix consumer needs with enterprise. NVidia mixed them and now everything is written in CUDA. Lol.
- latchkey 3y agoAnd we now have hipcc to go back to AMD. Sweet!
- pjmlp 3y agohipcc is a joke, it doesn't handle everything CUDA is capable of, specially not the polyglot capabilities.
- ColonelPhantom 3y agoWhat do you mean by polyglot? As in multiple hardware, or mixed-source? HIP is mostly API-compatible with CUDA, so you can just mix host code and device code with it. That said, ROCm does indeed work with machine code instead of IR. You can compile fat binaries with more than one type of machine code and they'll work on any of the chips you compiled for, at the cost of the binaries becoming, well, basically obese if you want a decent range of hardware supported.
- jjoonathan 3y ago
- photonbeam 3y ago> no more large scale tier 3 (redundant) data center power in the US. This is interesting to me, what is the constraining factor? Raw generator output? Transmission lines getting power to the right places?
- latchkey 3y agoBoth. Transformers are a big one. All the large FAANGs have been sucking up availability.
- deleted 3y ago[deleted]