4 ms·

AMD cards + ROCm are used in top supercomputers for (non-deep learning) HPC. Why is this the case? I understand that AMD GPUs offer better cost efficiency for

by quanto 3y ago

AMD cards + ROCm are used in top supercomputers for (non-deep learning) HPC. Why is this the case? I understand that AMD GPUs offer better cost efficiency for F32 & F64 FLOPs, RAM, and wattage. But however, if ROCm is such a half baked piece, shouldn't that advantage be gone? What drives AMD adoption in the HPC space then?

throwawaymaths 3y ago
That's not how sales works for those kinds of installations
taw28 3y ago
There is certainly a requirement that prevents payout on the contract for those supers if performance numbers aren’t met. Further, those performance numbers are from real apps, so the system WILL be useful. Not getting paid is not an option, so performance/usability will come.
- quanto 3y ago
  how does that contractual obligation translate to technical implementation? Do those supercomputers get an optimized version of ROCm to fulfill said obligation?
  - p_l 3y ago
    Vendor iterates until client gets an usable environment, even if that means 50 forks of different libraries with custom patches that in the end work only on that one system.
  - izacus 3y ago
    Absolutely. You usually end up with software stack where NOTHING can be updated, most of the stuff is forked with custom patches and the learnings there aren't reusable elsewhere because the code is full of "// replace this with hardcoded constant 59843 because it prevents crash on HPC machine". It's a good marketing metric, but probably contraproductive the AMDs longterm success in the field. They're spending engineering time building something they'll unlikely to be able to translate into other fields.
ip26 3y ago
I assume a deal like that involves a lot of direct support and optimization by ROCm team for the workload the supercomputer is running, rather than generic awesomeness at everything. That’s been my experience with most large enterprise software, really. Amazing at its most critical competency, core dumps when I try something a little unusual.
cavisne 3y ago
Nvidia has pretty epic margins in the datacenter space, so some of it will be that. Epic because their software advantage in AI lets them charge a premium, so why bother competing for the absolute cheapest $/FLOPS contract. More generally LLM training is weird because its a supercomputing workload being executed by Silicon Valley devs. So they want to use open source frameworks, find answers on stack overflow, use cloud providers etc. And it's new enough that random PHD's can invent stuff like Flash Attention without being employed by Nvidia. Normal supercomputer users dont care about any of that.
nestorD 3y ago
Part of the reason is that at the scale of a country like the US, if you are going to have several supercomputers build in the coming years, you want to diversify to avoid putting all your eggs in the same basket (and to giving a preferential treatment to a single vendor). So, as long as they can get a given set of applications to run above a given performance threshold, alternative vendors have a shot.
my123 3y ago
> AMD cards + ROCm are used in top supercomputers for (non-deep learning) HPC. Why is this the case? Public bidding processes are different and much more reliant solely on price. As such you can expect them to pick different choices than the private market. This can end up picking outright duds (see Aurora)
JonChesterfield 3y ago
ROCm works better in HPC than on consumer systems because it was initially written for HPC and arguably is still tested with HPC in mind. ROCm development was substantially funded by the Frontier supercomputer and the DoE developers get to complain directly at AMD developers when they're not happy. The DoE doesn't like having single suppliers for their tech. AMD/HPE's Frontier bid was a bit of a gamble from the DoE - it wasn't remotely obvious whether they'd be able to deliver it or not, and was against a background of Intel broadly failing to deliver Aurora - whereas nvidia was a known safe bet. However placing a big order with Intel and another one with AMD was their best shot at getting away from being wholly reliant on nvidia. Frontier shipped. It's a real thing, people run code on it. I'd guess the DoE labs talk to each other to some extent and thus AMD ended up winning the El Capitan bid. That means AMD has a flagship HPC machine that sales people can point to and a big ongoing revenue stream to continue funding development from. It looks like the DoE plan to have two HPC vendors has worked. Aurora seems to be somewhat in existence now but is less compelling as a story other potential customers might want to copy. I'm curious whether Intel end up making a loss on it.

by quanto 3y ago

throwawaymaths 3y ago

That's not how sales works for those kinds of installations

taw28 3y ago

There is certainly a requirement that prevents payout on the contract for those supers if performance numbers aren’t met. Further, those performance numbers are from real apps, so the system WILL be useful. Not getting paid is not an option, so performance/usability will come.

quanto 3y ago
how does that contractual obligation translate to technical implementation? Do those supercomputers get an optimized version of ROCm to fulfill said obligation?
- p_l 3y ago
  Vendor iterates until client gets an usable environment, even if that means 50 forks of different libraries with custom patches that in the end work only on that one system.
- izacus 3y ago
  Absolutely. You usually end up with software stack where NOTHING can be updated, most of the stuff is forked with custom patches and the learnings there aren't reusable elsewhere because the code is full of "// replace this with hardcoded constant 59843 because it prevents crash on HPC machine". It's a good marketing metric, but probably contraproductive the AMDs longterm success in the field. They're spending engineering time building something they'll unlikely to be able to translate into other fields.

ip26 3y ago

I assume a deal like that involves a lot of direct support and optimization by ROCm team for the workload the supercomputer is running, rather than generic awesomeness at everything. That’s been my experience with most large enterprise software, really. Amazing at its most critical competency, core dumps when I try something a little unusual.

cavisne 3y ago

Nvidia has pretty epic margins in the datacenter space, so some of it will be that. Epic because their software advantage in AI lets them charge a premium, so why bother competing for the absolute cheapest $/FLOPS contract. More generally LLM training is weird because its a supercomputing workload being executed by Silicon Valley devs. So they want to use open source frameworks, find answers on stack overflow, use cloud providers etc. And it's new enough that random PHD's can invent stuff like Flash Attention without being employed by Nvidia. Normal supercomputer users dont care about any of that.

nestorD 3y ago

Part of the reason is that at the scale of a country like the US, if you are going to have several supercomputers build in the coming years, you want to diversify to avoid putting all your eggs in the same basket (and to giving a preferential treatment to a single vendor). So, as long as they can get a given set of applications to run above a given performance threshold, alternative vendors have a shot.

my123 3y ago

> AMD cards + ROCm are used in top supercomputers for (non-deep learning) HPC. Why is this the case? Public bidding processes are different and much more reliant solely on price. As such you can expect them to pick different choices than the private market. This can end up picking outright duds (see Aurora)

JonChesterfield 3y ago

ROCm works better in HPC than on consumer systems because it was initially written for HPC and arguably is still tested with HPC in mind. ROCm development was substantially funded by the Frontier supercomputer and the DoE developers get to complain directly at AMD developers when they're not happy. The DoE doesn't like having single suppliers for their tech. AMD/HPE's Frontier bid was a bit of a gamble from the DoE - it wasn't remotely obvious whether they'd be able to deliver it or not, and was against a background of Intel broadly failing to deliver Aurora - whereas nvidia was a known safe bet. However placing a big order with Intel and another one with AMD was their best shot at getting away from being wholly reliant on nvidia. Frontier shipped. It's a real thing, people run code on it. I'd guess the DoE labs talk to each other to some extent and thus AMD ended up winning the El Capitan bid. That means AMD has a flagship HPC machine that sales people can point to and a big ongoing revenue stream to continue funding development from. It looks like the DoE plan to have two HPC vendors has worked. Aurora seems to be somewhat in existence now but is less compelling as a story other potential customers might want to copy. I'm curious whether Intel end up making a loss on it.