5 ms·
Azure announces new AI optimized VM series featuring AMD's flagship MI300X GPU
- CharlesW 3y agoI could've sworn that it was once a given that people doing serious AI work needed CUDA-capable GPUs. Is that no longer the case?
- deleted 3y ago[deleted]
- qeternity 3y agoNecessity is the mother of invention.
- kibibu 3y agoThe article highlights that onnx, deepspeed, pytorch and tensorflow all support these AMD processors
- CharlesW 3y agoThanks, so I think you're saying that there are two related things happening that are eliminating NVIDIA's CUDA moat: (1) Developers are mostly using libraries instead of writing "to the metal" (CUDA), and (2) many popular libraries have added support for AMD GPUs. Does that capture it?
- latchkey 3y agohttps://www.databricks.com/blog/training-llms-scale-amd-mi250-gpus https://www.databricks.com/blog/training-llms-scale-amd-mi25... https://embeddedllm.com/blog/vllm_rocm/ https://embeddedllm.com/blog/vllm_rocm/
- brucethemoose2 3y agoIt's the opposite in my experience. Pure PyTorch mostly works OK, but some libraries implementing crazy optimized, hand written kernels and such will have some trouble. So (for instance) maybe you can run an LLM in a particular PyTorch framework, but flash attention 2 doesn't support your AMD card, so performance and memory use takes a hit. Or maybe the library works on an Intel XPU with like 5 changed lines in the entire library (rename "cuda" to "xpu"), but no one bothered to add it, or maybe the dev doesn't even want to support the PR.
- sevagh 3y agoNot that I maintain any hugely popular deep learning code, but my own "hobby"/dev hesitance at adding multiple devices is that I don't own them to test them. If I have an NVIDIA GPU, the only PyTorch backend I can promise even "somewhat works on my machine" is CUDA.
- tails4e 3y agoAMD also have HIP which is almost exactly CUDA, but target agnostic, so basically now CUDA code will now run on AMD hardware. So they have the low level via HIP, and have also integrated into the libraries like pytorch. It's a great bit chip out of that moat.
- faeriechangling 3y agoThere aren't enough CUDA-capable GPUs to buy. So people will enter learn how to use products like the MI300X or they won't have AI.
- epolanski 3y agoIt's also a matter of price. Nvidia sells their top GPUs at insane premiums. For Microsoft which is such a huge player and the major AI vendor and owns a lot of OpenAI they have no point into cornering themselves into Nvidia dependence. They benefit from competition, not from bending to one vendor.
- latchkey 3y ago> Nvidia sells their top GPUs at insane premiums. It is all sold out, for years. You can't sell something you don't have. https://www.theregister.com/2023/09/08/tsmc_ai_chip_crunch/ https://www.theregister.com/2023/09/08/tsmc_ai_chip_crunch/
- modeless 3y agoNow that AI made Nvidia a trillion dollar company AMD has finally woken up and realized what they should have ten years ago: that they need to invest in the software side of things more. There is movement now, and you can actually do some AI things on AMD hardware, but it will take a long time for them to catch up to Nvidia.
- latchkey 3y agoAMD beat Intel in the server CPU market.
- modeless 3y agoAMD and Apple both have TSMC to thank for their ability to make chips that beat Intel's. Unfortunately for AMD, Nvidia uses TSMC too, so they'll have to invest in software to compete.
- deleted 3y ago[deleted]
- latchkey 3y agoThey are investing in the software. https://fortune.com/europe/2023/10/04/amd-lisa-su-nvidia-rocm-cuda-ai-processor/ https://fortune.com/europe/2023/10/04/amd-lisa-su-nvidia-roc...
- malfist 3y agoThat's a bold assertion with no evidence to back it up
- latchkey 3y agoI've been running large scale data centers for years now with tens of thousands of machines. I'd personally never put an Intel cpu into something that I cared about. We used $30 cheapo intel cpus for our mining rigs cause they literally didn't need anything more than that. But, anything else we buy for servers, is always going to be AMD. One caveat to that would be if we have some customer that demanded Intel and we'd lose the business otherwise. So yea, in my eyes, AMD beat Intel. I don't think I need any more evidence than that. ¯\_(ツ)_/¯
- treprinum 3y agoThat's the joke. It still is. ROCm is nowhere near production-ready and if MS thinks devs will want to waste their time on random errors then good luck with that business. AMD cards are also super expensive so not sure what their competitiveness is supposed to be?
- epolanski 3y agoMicrosoft can develop their own software. We are talking a company that places billions in orders to Nvidia, they can afford to develop the tools and obviously AMD will throw millions and millions in support to court such a customer.
- latchkey 3y agoExpensive is not entirely relevant. They are available (for now). https://twitter.com/sama/status/1724626002595471740 https://twitter.com/sama/status/1724626002595471740 ROCm has also made a lot of advances in recent times. https://www.databricks.com/blog/training-llms-scale-amd-mi250-gpus https://www.databricks.com/blog/training-llms-scale-amd-mi25...
- kristianp 3y agoThread about that tweet (or should I say xeet) from SamA: https://news.ycombinator.com/item?id=38274427 https://news.ycombinator.com/item?id=38274427
- brucethemoose2 3y agoThe MI300 is a massive GPU compared to even the new H200. It was designed as a combined CPU/GPU for supercomputers, with shared memory. But then the AI craze hit, so AMD spun a variant into a pure GPU AI accelerator real quick, which they could actually pull off because the GPU silicon is modular. ...So thats why it cost a fortune. Its really a jury rigged HPC product.
- pclmulqdq 3y agoThe H100 is, to some degree, also a jury rigged HPC product. The H200 is starting to reach a more "AI" balance between compute and memory bandwidth.
- brucethemoose2 3y agoAMD is "CUDA compatible" in theory. Intel is taking a slightly different approach, and is going for "PyTorch compatible." You will hear endless negative anecdotes about ROCm/OpenVINO, but they both do seem to be getting better with each update.
- yieldcrv 3y agoROCm support for a lot of AI processes fixed that
- epolanski 3y agoI'm surprised by the negativity. Microsoft has the know-how from hardware and software, drivers, APIs, firmware the resources, it's a major AMD customer in Cloud and consumer devices. Do people think Microsoft and AMD will watch Nvidia cannibalize the market snd Microsoft writing cheques for whatever Nvidia demands? It's like people forget a major economic rule: when margins are high you will attract competition.
- partiallypro 3y agoSimilar to the old Slashdot days, HN is still plagued with just outright anti-Microsoft people under any circumstance, even in cases where there is no justification.
- jiggawatts 3y agoA more recent thing is just pretending Microsoft (and especially Azure) doesn’t even exist. If pressed, sure, people will admit that these may not be entirely imaginary entities, but if listing technologies or platforms then “oops” they’ll just forget. The best example I saw was a “poster” of cloud big data technologies. Started with Amazon S3, went through Google Bigtable, and then in the corners had companies so small that their own marketing page is the only search result. No mention of Azure anywhere. There were dozens of logos on that page representing companies with annual revenues smaller than what one my customers spent on a single Azure Storage Account by accident.
- dboreham 3y agoMeanwhile they're all committing code to github and pulling packages from npmjs.
- justahuman74 3y agoIf often wondered if microsoft should rebrand the entire company to github
- ChrisArchitect 3y agoConfusing as they also announced their own Cobalt AI chips too
- kristianp 3y agoNo, that's about Microsoft's chips. This is about AMDs GPUs.
- deleted 3y ago[deleted]
- taspeotis 3y agoImagine the hallucinations we're gonna see with AMD drivers in the mix...
- yieldcrv 3y agoROCm was a bull signal