6 ms·

I would sincerely hope for a competitive AMD GPU for deep learning. But as long as it's a week-long journey with unknown ending to try to recompile TensorFlow t

I would sincerely hope for a competitive AMD GPU for deep learning. But as long as it's a week-long journey with unknown ending to try to recompile TensorFlow to support ROCm, everyone I know in AI will firmly stick with NVIDIA and their production-proven drivers and CUDA APIs. I wish AMD would offer something like NVIDIA's Inception program to gift some accelerators and GPUs to suitable C++ coders (like me) so that there's at least a few tutorials on the internet on how other people managed to successfully use AMD + ROCm for deep learning. EDIT: And it seems ROCm doesn't even support any of those new RDNA2 accelerators or gaming GPUs: https://github.com/RadeonOpenCompute/ROCm/issues/1344 https://github.com/RadeonOpenCompute/ROCm/issues/1344 So this is great hardware, but absolutely useless unless you are big enough to write your own GPU drivers from scratch ~_~

loosescrews 5y ago
https://github.com/RadeonOpenCompute/ROCm/issues/1617 https://github.com/RadeonOpenCompute/ROCm/issues/1617 appears to be the issue to track if you are interested in ROCm support for RDNA2.
my123 5y ago
ROCm 4.5 is also the last release to support their own Vega 10 based accelerator. (Radeon Instinct MI25) https://github.com/RadeonOpenCompute/ROCm#amd-instinct-mi25-end-of-life https://github.com/RadeonOpenCompute/ROCm#amd-instinct-mi25-... aka AMD doesn't care... they just want the supercomputer contracts where the customers are savvy enough to build their own very specific SW stack.
- bradknowles 5y ago
  I have friends here in the Austin area who have worked for AMD. From what I’ve heard from them, it’s not that AMD doesn’t care, it’s that AMD is clueless and hopelessly disorganized, and they’re constantly doing whatever they can to chase the latest supercomputing contract, to the exclusion of all else. It’s a target-rich environment if you want to learn all the bad anti-patterns, so that you can avoid doing them in the rest of your career. So, it’s not that they don’t care. It’s that they don’t have enough hours in the every-day-is-a-hair-on-fire-day that they would be capable of caring.
- generalizations 5y ago
  Sounds like AMD might still be using the 'tesla roadster' strategy, of selling fewer, more lucrative contracts for the time being. Probably not that they don't care, just that for now, they have to focus.
  - sorenjan 5y ago
    GPGPU have been a thing for two decades, they should have had this solved years ago. Lets hope SYCL gets wide support, Intel seems to bet on it and there's a Cuda backend available. If AMD wants to make themselves irrelevant for anything but gaming and HPC that's their choice.
    - mastax 5y ago
      Years ago AMD was almost bankrupt and put everything into one last attempt at a CPU. Now AMD is hiring like crazy, and I hope they are able to catch up to their larger competitors in tooling and software. Here's what I think they'll do for the near future. They can't build an all-encompassing library ecosystem that supports all the hardware, all the existing software, has really good tooling, etc. in a reasonable amount of time. In consumer they can mostly get away with just supporting Microsoft's APIs, which they already do pretty well. If they design a really good compute GPU, they probably won't be able to make enough of them. They can sell all of their production with a few big deals to important customers, and they can attach engineers to those deals to help support the specific needs of those customers. That is much more practical than trying to catch up to CUDA in its entirety. Basically, status quo. As someone who isn't a big customer and who is mad at Nvidia I don't hope for this to happen, but it seems the most likely path.
  - rhn_mk1 5y ago
    That sounds reasonable on the surface, but cards don't cost a lot, and sending them out for free to leverage community interest for the open source stack could be extremely cost effective.
    - paulmd 5y ago
      Even better, you can provide support for gaming cards and then the community pays you for them instead of you having to pay for it out of dev-rel budget. And yet AMD is actively moving in the opposite direction and dropping support for older hardware from both their Windows driver, as well as from ROCm and their other compute software. It's hard to overstate what a positive impact gaming card support and donations of hardware to educational institutions and other devrel have been for CUDA - those are the people who are writing the next 10 years of your software, that sells the actual enterprise hardware. And at this point it's not just "can't afford to support it", AMD is doing fine these days, they just don't want to. Kind of baffling.
- jacquesm 5y ago
  Is there even a single machine in the supercomputer top 10 that uses AMD GPUs? I see NVIDIA all over the place there but I'm not aware of any of them using AMD GPUs, though a couple do use AMD CPUs.
  - paulmd 5y ago
    Yeah frankly it's a little misleading to frame this as "AMD won this", this is a gimmie contract to keep them in the game. The DoE is throwing gimmie contracts to Intel too for Xe and they haven't even produced a working product yet. Their CPUs pretty much fall into the same boat too - is it justifiable to buy Intel CPUs right now for HPC applications, especially with AMD supporting AVX-512 with their Zen4 chips (which are the counterparts to the Sapphire Rapids DoE is buying)? Not really but their interests are in keeping Intel in the game, an AMD monoculture doesn't benefit anyone anymore than an Intel monoculture did. Although of course this is not fab-related, it's the same basic strategy - the US wants as diverse and thriving a tech ecosystem as they can get in the west, and particularly in the US, to counterbalance a rising China. Not that China is anywhere close today, but in the 20-year timeframe it's a major concern.
  - my123 5y ago
    None today. those HPC machines will be the first ones
- deleted 5y ago
  [deleted]
esistgut 5y ago
ROCm doesn't support RDNA at all https://github.com/RadeonOpenCompute/ROCm/issues/887 https://github.com/RadeonOpenCompute/ROCm/issues/887
- Certhas 5y ago
  It's promised for ROCm 5.0 now... https://github.com/RadeonOpenCompute/ROCm/issues/1180#issuecomment-942101147 https://github.com/RadeonOpenCompute/ROCm/issues/1180#issuec...
  - meragrin_ 5y ago
    "might expect good news with 5.0" is not a promise.
    - cyber_kinetist 5y ago
      And they've been delaying this for months. Last time on April they've said on Github that 5700XT support should be roughly available in 2~4 months, and its already November.
Certhas 5y ago
ROCm support for Gaming cards has been poor (and not advertised) but it's part of the tech stack they are selling with these accelerators: https://www.olcf.ornl.gov/wp-content/uploads/2019/05/frontier_specsheet.pdf https://www.olcf.ornl.gov/wp-content/uploads/2019/05/frontie... It's clearly a real problem that AMD's ML Software stack isn't quite there, and lacks in support for the non-specialized cards, but that's not really an issue for these HPC use cases....
- visionscaper 5y ago
  Interesting, thanks for sharing this. Apparently they are going to use “HIP” to convert CUDA applications to be able to run on AMD: > The OLCF plans to make HIP available on Summit so that users can begin using it prior to its availability on Frontier. HIP is a C++ runtime API that allows developers to write portable code to run on AMD and NVIDIA GPUs. It is essentially a wrapper that uses the underlying CUDA or ROCm platform that is installed on a system. The API is very similar to CUDA so transitioning existing codes from CUDA to HIP should be fairly straightforward in most cases. In addition, HIP provides porting tools which can be used to help port CUDA codes to the HIP layer, with no loss of performance as compared to the original CUDA application. HIP is not intended to be a drop-in replacement for CUDA, and developers should expect to do some manual coding and performance tuning work to complete the port. Would this one day lead to a HIP based PyTorch? I hope so!
  - paulmd 5y ago
    you can think of HIP as being a fancy batch script to convert CUDA to OpenCL and it works about as well. If you're going to have to re-debug it anyway, what's the point? AMD's focus certainly should have been on something like GPU Ocelot or compiling PTX to GCN/RDNA instead.
  - ajtulloch 5y ago
    Good news! https://pytorch.org/docs/stable/notes/hip.html https://pytorch.org/docs/stable/notes/hip.html
  - slavik81 5y ago
rektide 5y ago
AMD's not nowhere. https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learning.html https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learni... shows what should be a followable happy path to getting TensorFlow going (2 year old TF 1.15, and a 2.2beta). I'm curious what is prickly or hard about it. IMO the deep learning folk need to be working more actively towards the future. The CUDA free ride is amazing, but AMD's HIP already does a good job being CUDA compliant in a general sense. But CUDA also sort of encompasses the massive collection of libraries that Nvidia has written to accelerate a huge amount of use cases. Trying to keep pace with that free-ride is hard. My hope is eventually we start to invest in Vulkan Compute. Vulkan is way way way harder than CUDA, but it's the only right way I can see to do things. Getting TensorFlow & other libraries ported to run atop Vulkan is a herculean feat, but once there's a start, I tend to believe most ML practitioners won't have to think about the particulars, and I think the deep engineering talent will be able to come, optimize, improve the Vulkan engines very quickly, rapidly be able to improve whatever it is we start with. It's a huge task, but it just seems like it's got to happen. I don't see what alternative there is, long term, to starting to get good with Vulkan.
- spijdar 5y ago
  > I'm curious what is prickly or hard about it. I don't want to presume, but it sounds like you haven't actually tried using ROCm "what should be". My experience with it was an absolute nightmare, I've never gotten ROCm working before. Just as well, since it turns out my systems never would have supported it for various reasons (lacking PCIe atomics for one), but I never actually got so far as to run into the driver problem, I never got the whole custom LLVM fork/ROCm software stack to work. Caveat, I'm not professionally involved in deep learning or HPC, and as others have mentioned, the framework was only intended for a few specific cards running on very specific hardware for HPC cases. But pretending like this is even a fraction as useful for the "average" person trying to experiment or even work at a low-medium level in machine learning feels off to me. I don't think people will be swayed by platitudes about creating a competitive open-systems ecosystem to use plainly inferior software. Companies aren't going to spend oodles of money (and individuals won't volunteer tons of time) to suffer porting frameworks to target bare-bones APIs for the sake of being good sports. Until either nvidia screws over everyone so much that using AMD cards becomes the path of least resistance, or AMD/Intel offers products at significantly lower prices than nvidia, I don't see the status quo changing much.
esistgut 5y ago
Today a Blender beta version with HIP support has been released. This is working on RDNA hardware (RDNA2 officially supported, RDNA1 enabled but not supported). I guess a release date for ROCm is approaching after all.
Sebb767 5y ago
> I wish AMD would offer something like NVIDIA's Inception program to gift some accelerators and GPUs to suitable C++ coders (like me) so that there's at least a few tutorials on the internet on how other people managed to successfully use AMD + ROCm for deep learning. Why not go a step further and pay some people to integrate good first-party support in the most popular libraries? It would probably be quite cheap in comparison and kickstart the adaption.
- frozenport 5y ago
  NVIDIA, Intel, Apple, Google (TPU) have hundreds of engineers working on squeezing the performance out of their chips. This is not trivial.
  - dagmx 5y ago
    Sometimes it's cheaper to pay a third party to implement something than going through the hiring process and extraneous legalities of contribution. Sponsoring a third party to do something or contracting them, is very common.
    - joconde 5y ago
      It will be expensive no matter who does it. Probably more for a contractor, given that they’d have less experience and internal knowledge of the hardware.
YetAnotherNick 5y ago
There is a competitive port of tensorflow for AMD GPU. Just the only issue is that it is from Apple and works only on Mac. Tensorflow fully supports AMD GPU in Mac and it could max out the graphics card usage as well. Makes me wonder if Apple could do it, why can't AMD work to use Vulkan instead of Metal. https://developer.apple.com/metal/tensorflow-plugin/ https://developer.apple.com/metal/tensorflow-plugin/

loosescrews 5y ago

https://github.com/RadeonOpenCompute/ROCm/issues/1617 https://github.com/RadeonOpenCompute/ROCm/issues/1617 appears to be the issue to track if you are interested in ROCm support for RDNA2.

my123 5y ago

ROCm 4.5 is also the last release to support their own Vega 10 based accelerator. (Radeon Instinct MI25) https://github.com/RadeonOpenCompute/ROCm#amd-instinct-mi25-end-of-life https://github.com/RadeonOpenCompute/ROCm#amd-instinct-mi25-... aka AMD doesn't care... they just want the supercomputer contracts where the customers are savvy enough to build their own very specific SW stack.

bradknowles 5y ago
I have friends here in the Austin area who have worked for AMD. From what I’ve heard from them, it’s not that AMD doesn’t care, it’s that AMD is clueless and hopelessly disorganized, and they’re constantly doing whatever they can to chase the latest supercomputing contract, to the exclusion of all else. It’s a target-rich environment if you want to learn all the bad anti-patterns, so that you can avoid doing them in the rest of your career. So, it’s not that they don’t care. It’s that they don’t have enough hours in the every-day-is-a-hair-on-fire-day that they would be capable of caring.
generalizations 5y ago
Sounds like AMD might still be using the 'tesla roadster' strategy, of selling fewer, more lucrative contracts for the time being. Probably not that they don't care, just that for now, they have to focus.
- sorenjan 5y ago
  GPGPU have been a thing for two decades, they should have had this solved years ago. Lets hope SYCL gets wide support, Intel seems to bet on it and there's a Cuda backend available. If AMD wants to make themselves irrelevant for anything but gaming and HPC that's their choice.
  - mastax 5y ago
    Years ago AMD was almost bankrupt and put everything into one last attempt at a CPU. Now AMD is hiring like crazy, and I hope they are able to catch up to their larger competitors in tooling and software. Here's what I think they'll do for the near future. They can't build an all-encompassing library ecosystem that supports all the hardware, all the existing software, has really good tooling, etc. in a reasonable amount of time. In consumer they can mostly get away with just supporting Microsoft's APIs, which they already do pretty well. If they design a really good compute GPU, they probably won't be able to make enough of them. They can sell all of their production with a few big deals to important customers, and they can attach engineers to those deals to help support the specific needs of those customers. That is much more practical than trying to catch up to CUDA in its entirety. Basically, status quo. As someone who isn't a big customer and who is mad at Nvidia I don't hope for this to happen, but it seems the most likely path.
- rhn_mk1 5y ago
  That sounds reasonable on the surface, but cards don't cost a lot, and sending them out for free to leverage community interest for the open source stack could be extremely cost effective.
  - paulmd 5y ago
    Even better, you can provide support for gaming cards and then the community pays you for them instead of you having to pay for it out of dev-rel budget. And yet AMD is actively moving in the opposite direction and dropping support for older hardware from both their Windows driver, as well as from ROCm and their other compute software. It's hard to overstate what a positive impact gaming card support and donations of hardware to educational institutions and other devrel have been for CUDA - those are the people who are writing the next 10 years of your software, that sells the actual enterprise hardware. And at this point it's not just "can't afford to support it", AMD is doing fine these days, they just don't want to. Kind of baffling.
jacquesm 5y ago
Is there even a single machine in the supercomputer top 10 that uses AMD GPUs? I see NVIDIA all over the place there but I'm not aware of any of them using AMD GPUs, though a couple do use AMD CPUs.
- paulmd 5y ago
  Yeah frankly it's a little misleading to frame this as "AMD won this", this is a gimmie contract to keep them in the game. The DoE is throwing gimmie contracts to Intel too for Xe and they haven't even produced a working product yet. Their CPUs pretty much fall into the same boat too - is it justifiable to buy Intel CPUs right now for HPC applications, especially with AMD supporting AVX-512 with their Zen4 chips (which are the counterparts to the Sapphire Rapids DoE is buying)? Not really but their interests are in keeping Intel in the game, an AMD monoculture doesn't benefit anyone anymore than an Intel monoculture did. Although of course this is not fab-related, it's the same basic strategy - the US wants as diverse and thriving a tech ecosystem as they can get in the west, and particularly in the US, to counterbalance a rising China. Not that China is anywhere close today, but in the 20-year timeframe it's a major concern.
- my123 5y ago
  None today. those HPC machines will be the first ones
deleted 5y ago
[deleted]

esistgut 5y ago

ROCm doesn't support RDNA at all https://github.com/RadeonOpenCompute/ROCm/issues/887 https://github.com/RadeonOpenCompute/ROCm/issues/887

Certhas 5y ago
It's promised for ROCm 5.0 now... https://github.com/RadeonOpenCompute/ROCm/issues/1180#issuecomment-942101147 https://github.com/RadeonOpenCompute/ROCm/issues/1180#issuec...
- meragrin_ 5y ago
  "might expect good news with 5.0" is not a promise.
  - cyber_kinetist 5y ago
    And they've been delaying this for months. Last time on April they've said on Github that 5700XT support should be roughly available in 2~4 months, and its already November.

Certhas 5y ago

ROCm support for Gaming cards has been poor (and not advertised) but it's part of the tech stack they are selling with these accelerators: https://www.olcf.ornl.gov/wp-content/uploads/2019/05/frontier_specsheet.pdf https://www.olcf.ornl.gov/wp-content/uploads/2019/05/frontie... It's clearly a real problem that AMD's ML Software stack isn't quite there, and lacks in support for the non-specialized cards, but that's not really an issue for these HPC use cases....

visionscaper 5y ago
Interesting, thanks for sharing this. Apparently they are going to use “HIP” to convert CUDA applications to be able to run on AMD: > The OLCF plans to make HIP available on Summit so that users can begin using it prior to its availability on Frontier. HIP is a C++ runtime API that allows developers to write portable code to run on AMD and NVIDIA GPUs. It is essentially a wrapper that uses the underlying CUDA or ROCm platform that is installed on a system. The API is very similar to CUDA so transitioning existing codes from CUDA to HIP should be fairly straightforward in most cases. In addition, HIP provides porting tools which can be used to help port CUDA codes to the HIP layer, with no loss of performance as compared to the original CUDA application. HIP is not intended to be a drop-in replacement for CUDA, and developers should expect to do some manual coding and performance tuning work to complete the port. Would this one day lead to a HIP based PyTorch? I hope so!
- paulmd 5y ago
  you can think of HIP as being a fancy batch script to convert CUDA to OpenCL and it works about as well. If you're going to have to re-debug it anyway, what's the point? AMD's focus certainly should have been on something like GPU Ocelot or compiling PTX to GCN/RDNA instead.
- ajtulloch 5y ago
  Good news! https://pytorch.org/docs/stable/notes/hip.html https://pytorch.org/docs/stable/notes/hip.html
- slavik81 5y ago

rektide 5y ago

AMD's not nowhere. https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learning.html https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learni... shows what should be a followable happy path to getting TensorFlow going (2 year old TF 1.15, and a 2.2beta). I'm curious what is prickly or hard about it. IMO the deep learning folk need to be working more actively towards the future. The CUDA free ride is amazing, but AMD's HIP already does a good job being CUDA compliant in a general sense. But CUDA also sort of encompasses the massive collection of libraries that Nvidia has written to accelerate a huge amount of use cases. Trying to keep pace with that free-ride is hard. My hope is eventually we start to invest in Vulkan Compute. Vulkan is way way way harder than CUDA, but it's the only right way I can see to do things. Getting TensorFlow & other libraries ported to run atop Vulkan is a herculean feat, but once there's a start, I tend to believe most ML practitioners won't have to think about the particulars, and I think the deep engineering talent will be able to come, optimize, improve the Vulkan engines very quickly, rapidly be able to improve whatever it is we start with. It's a huge task, but it just seems like it's got to happen. I don't see what alternative there is, long term, to starting to get good with Vulkan.

spijdar 5y ago
> I'm curious what is prickly or hard about it. I don't want to presume, but it sounds like you haven't actually tried using ROCm "what should be". My experience with it was an absolute nightmare, I've never gotten ROCm working before. Just as well, since it turns out my systems never would have supported it for various reasons (lacking PCIe atomics for one), but I never actually got so far as to run into the driver problem, I never got the whole custom LLVM fork/ROCm software stack to work. Caveat, I'm not professionally involved in deep learning or HPC, and as others have mentioned, the framework was only intended for a few specific cards running on very specific hardware for HPC cases. But pretending like this is even a fraction as useful for the "average" person trying to experiment or even work at a low-medium level in machine learning feels off to me. I don't think people will be swayed by platitudes about creating a competitive open-systems ecosystem to use plainly inferior software. Companies aren't going to spend oodles of money (and individuals won't volunteer tons of time) to suffer porting frameworks to target bare-bones APIs for the sake of being good sports. Until either nvidia screws over everyone so much that using AMD cards becomes the path of least resistance, or AMD/Intel offers products at significantly lower prices than nvidia, I don't see the status quo changing much.

esistgut 5y ago

Today a Blender beta version with HIP support has been released. This is working on RDNA hardware (RDNA2 officially supported, RDNA1 enabled but not supported). I guess a release date for ROCm is approaching after all.

Sebb767 5y ago

> I wish AMD would offer something like NVIDIA's Inception program to gift some accelerators and GPUs to suitable C++ coders (like me) so that there's at least a few tutorials on the internet on how other people managed to successfully use AMD + ROCm for deep learning. Why not go a step further and pay some people to integrate good first-party support in the most popular libraries? It would probably be quite cheap in comparison and kickstart the adaption.

frozenport 5y ago
NVIDIA, Intel, Apple, Google (TPU) have hundreds of engineers working on squeezing the performance out of their chips. This is not trivial.
- dagmx 5y ago
  Sometimes it's cheaper to pay a third party to implement something than going through the hiring process and extraneous legalities of contribution. Sponsoring a third party to do something or contracting them, is very common.
  - joconde 5y ago
    It will be expensive no matter who does it. Probably more for a contractor, given that they’d have less experience and internal knowledge of the hardware.

YetAnotherNick 5y ago

There is a competitive port of tensorflow for AMD GPU. Just the only issue is that it is from Apple and works only on Mac. Tensorflow fully supports AMD GPU in Mac and it could max out the graphics card usage as well. Makes me wonder if Apple could do it, why can't AMD work to use Vulkan instead of Metal. https://developer.apple.com/metal/tensorflow-plugin/ https://developer.apple.com/metal/tensorflow-plugin/