10 ms·
HPC admin here. ROCm, which is AMD's equivalent of CUDA. The thing is you don't have to directly interface with CUDA or ROCm. Once the framework you want to us
by bayindirh 3y ago
HPC admin here.
ROCm, which is AMD's equivalent of CUDA. The thing is you don't have to directly interface with CUDA or ROCm. Once the framework you want to use supports these, you're done.
AMD is consistently getting used on the TOP500 machines, and this gives them insane amounts of money to improve ROCm. CUDA has an ecosystem and hardware moat, but not because they're vastly superior, but because AMD both prioritized processors first, and NVIDIA played dirty (OpenCL support and performance shenanigans, anyone?).
This moat is bound to be eaten away by both Intel and AMD, and compute will be commoditized. NVIDIA foresaw this and bought Mellanox and wanted ARM to be a complete black box, but it didn't work out.
Ethernet consortium woke up and got agitated by the fact that only ultra-low latency fabric provider is not independent anymore, so they're started to build alternatives to Infiniband ecosystem.
Interesting times are ahead.
Also there's OneAPI, but I'm not very knowledgeable about it. It's a superAPI which can target many compute platforms, like OpenCL, but takes a different approach. It compiles to platform native artifacts (CUDA/ROCm/Intel/x86/custom, etc.) IIRC.
- justinclift 3y ago> they're started to build alternatives to Infiniband ecosystem. Cool, that sounds interesting. Anything you can point at? :)
- timschmidt 3y agoUltraEthernet
- justinclift 3y agoThanks. :)
- viewtransform 3y agoDiscussion on ultrafast ethernet and partners at AMD's December presentation. https://youtu.be/tfSZqjxsr0M?t=5198 https://youtu.be/tfSZqjxsr0M?t=5198
- formerly_proven 3y agoTOP500 has had all sorts of "odd" systems near the top. Unusual or custom architectures and fabrics are far more viable for "this is our nation's nukeputer" or "this is going to run the nation's weather forecasting model" type of systems compared to e.g. a commercial HPC system where you wanna run all sorts of commercial/proprietary software. AMD being successful in the former niche doesn't mean their software stack is viable for the latter niche.
- bayindirh 3y agoThe systems running AMD cards are not "Nukeputers", and not all "Nukeputers" and "Fusionputers" are running on custom silicon. When you look to latest list [0], #1 system, Frontier, is a Nukeputer, but it's not only a Nukeputer. #5, LUMI is definitely not a Nukeputer. They are very close to us, we work together under a project with them (we're equals. They operate under a different consortium, we have stakes on another computer which is also very famous and also in top 10 in TOP500, and is not a Nukeputer). We also have our smaller systems in our own datacenter. We operate mostly in the long tail of science, and this means we see heavy use of popular software packages, and many special software packages optimized for these long tail problems. ROCm was invisible in this niche before, but with Frontier and LUMI, and with AMD's announcements, this started to change very quickly. ROCm libraries are more open w.r.t. their CUDA counterparts, and they started to land to mainstream distribution repositories directly. This is important for our niche. AMD is sponsoring integration of their libraries to popular packages and it started to payoff already. Also, small-guy-programmers start to optimize LLM training routines for AMD cards, getting 99% of the performance of NVIDIA counterparts with way less power consumption. As a result, AMD is already much more visible and more capable position when compared to last year. [0]: https://top500.org/lists/top500/list/2023/11/ https://top500.org/lists/top500/list/2023/11/