8 ms·

Radeon Open Compute 3.0

208 points by mroche 7y ago

deleted 7y ago
[deleted]
barkingcat 7y ago
For people trying to install this on Debian/Ubuntu, the group that enables access to the gpgpu is actually "render" instead of "video" I spent some time wondering why I couldn't get access to the device until some kind soul pointed me to the different group name. https://fosstodon.org/@dctrud/103315274791608145 https://fosstodon.org/@dctrud/103315274791608145
- app4soft 7y ago
  Does ROC support ATI RS600 GPUs?
  - microcolonel 7y ago
    No, only GCN and later, AFAIK.
neximo64 7y ago
I wish OS X would do better with AMD and also get it working with Tensorflow. No OS X support here sadly.
- nknealk 7y ago
  Check out PlaidML (from of all companies intel) to get AMD GPUs working with the Keras APIs for tensorflow https://github.com/plaidml/plaidml https://github.com/plaidml/plaidml
  - neximo64 7y ago
    I've used this, unfortunately I've not had a good experience with it. Say I was to have a keras model, and use a tensorflow backend, and then use the plaidml backend and compare the results: They're different in a radical way. Also I don't think they've got their driver usage right. Metal on the GPU gets different results than OpenCL. OpenCL is supposed to be the 'beta' and the results are more stable with it - and again different radically between the PlaidML devices.
    - woadwarrior01 7y ago
      I remember running inference on a model that I trained on a pair of NVIDIA 1080TIs with a NVIDIA 2080TI (after upgrading) and observing slightly (albeit not significantly) different results. We’re dealing with single precision floats for everything. FWIW, every implementation of IEE 754 floats has its own set of minor oddities and quirks.
- nileshk 7y ago
  I wish macOS would allow for PCI passthrough to VMs (like you can do on Linux via OVMF, using QEMU). I wish I could connect an eGPU and pass full access of it to my Linux VM (or Windows VM), whether that GPU is AMD or NVidia shouldn't matter. That would remove the disadvantages Macs have when it comes to GPU and make it the best VM host (currently Linux seems to be the best option for VM host). I'm sure Mac Pro owners could take advantage of this were it possible. However, I suspect Apple has no intention of doing this, as they are more focus on pushing Metal and reliance on macOS. I wouldn't mind seeing PCI or GPU passthrough capabilities this in Windows 10 Pro (not just Server). With their Linux Subsystem progressing the way it is, they really have an opportunity here. Though since NVidia and CUDA are supported in Windows, maybe not as necessary as the situation on macOS.
- ahartmetz 7y ago
  You bought into a more closed ecosystem and that is what you have now.
- shmerl 7y ago
  Why do you need macOS? You can use Linux for your needs, it lets you do what you want and how you want it, instead of Apple telling you you can't do it.
- slezyr 7y ago
  That's entirely self inflicted wound by Apple they did their best to stop any vendor from developing the driver.
Athas 7y ago
For anyone else using NixOS, https://github.com/nixos-rocm/nixos-rocm https://github.com/nixos-rocm/nixos-rocm by Anthony Cowley works very well. While ROCm works well, I think it is yet another example of what AMD gets wrong software-wise. I like to think I'm fairly familiar with GPGPU, in that I have used OpenCL and CUDA for years, and even wrote my own optimising compiler that generates GPU code. I even use ROCm every day on my home system. Yet, I would be hard pressed to succinctly state what ROCm is. The GitHub page states the following: > ROCm is designed to be a universal platform for gpu-accelerated computing. This modular design allows hardware vendors to build drivers that support the ROCm framework. ROCm is also designed to integrate multiple programming languages and makes it easy to add support for other languages. Okay? It then lists some other stuff: > • Drivers > • Tools > • Libraries > • Source Code The AMD GPU kernel driver, amdgpu, is upstreamed in Linux, so what exactly does ROCm contain? The OpenCL ICD? (I'm pretty sure it does.) As far as I can see, I don't have any ROCm-specific kernel drivers loaded, but OpenCL on my AMD GPU works fine. The tools are also confusing. There's an "HCC compiler", but are we even still supposed to use HCC? As I recall, it was/is some heterogeneous compute C++ dialect. There's also HIP, which converts CUDA to something that can run on AMD GPUs. Trying to refresh my memory, I actually can't figure out in two minutes what language the output is. HCC? OpenCL? Among the other tools, there is inconsistency between ROCM and ROCm (pedantry, I know, I assume it's the same), but there is also the "ROC Profiler" and "ROCr Debug Agent". Are "ROC" and "ROCr" significant terms? And when should I use the "Radeon Compute Profiler" (rocm-profiler) and when should I use the "ROC Profiler" (rocprofiler-dev)? I do so dearly want to support AMD. Their hardware is good and their fully open source drivers are a wonderful accomplishment. Seriously, I want to underline just how amazed I am that I have a fully free high-performance GPGPU stack running. However, if you compare AMDs software efforts to NVIDIA, it is clear that NVIDIA deserves their success. It's not just about breadth (NVIDIA is bigger, I can understand they have more resources), but the fact that AMD seems to spread itself too thin over too many effort, and it makes everything feel kind of shoddy and poorly documented.
- jedbrown 7y ago
  > There's also HIP, which converts CUDA to something that can run on AMD GPUs. Kinda making your point, HIP is actually an (open source) one-to-one replacement for the CUDA API. While there are tools (https://github.com/ROCm-Developer-Tools/HIP/tree/master/hipify-clang https://github.com/ROCm-Developer-Tools/HIP/tree/master/hipi... and https://github.com/ROCm-Developer-Tools/HIP/blob/master/bin/hipify-perl https://github.com/ROCm-Developer-Tools/HIP/blob/master/bin/...) that convert CUDA to HIP, these are meant to be run once and perhaps touched up manually. HIP source can then be compiled (no human interaction) for AMD (ROCm) devices and NVIDIA (CUDA) devices. The CUDA brand refers to both the API and the device. AMD certainly could have done a better job, but wanted to distinguish these, presumably in hopes that existing users of the CUDA API would consider porting their code to HIP even when targeting CUDA devices.
e12e 7y ago
So is this AMDs response to CUDA? Is there any reason to believe this will end up practically cross-platform? Is the implication that opencl is dead?
- Athas 7y ago
  This contains an OpenCL implementation (with support for 2.0 at that!) and the kitchen sink. It's a bit unclear what AMD actually suggests you use for programming their GPUs.
continuational 7y ago
What is it, exactly?
- Athas 7y ago
  More or less AMD CUDA: a collection of compilers, tools, and libraries. The main difference is that NVIDIA markets CUDA strongly based on CUDA C++. You know the way you write CUDA is through nvcc and the language it accepts, unless you turn out to have exotic needs. Instead, AMD just provides a whole bunch of various things with fuzzy documentation, and lets you to pick whatever you like (although I'm not quite sure how you're supposed to know what you like).
  - cwt137 7y ago
    Will this help any on getting acceleration for Tensorflow?
    - Athas 7y ago
      Yes, there is supposedly Tensorflow support in ROCm. I have never used it, so I cannot say how well it works.
      - dannyw 7y ago
        Just don't expect it to be as good as Tensorflow on CUDA.
        hgoel 7y ago
        What makes you say that? It works perfectly fine and so far I haven't encountered anything I couldn't do with tensorflow-rocm that should be doable with Tensorflow on Nvidia
- imtringued 7y ago
  Well they have decided to put the most important information at the bottom of the readme... https://github.com/RadeonOpenCompute/ROCm/tree/roc-3.0.0#rocm-development-toolchains https://github.com/RadeonOpenCompute/ROCm/tree/roc-3.0.0#roc... It's a collection of GPU development tools and libraries.
bcatanzaro 7y ago
No Navi support? Surprising.
- vetrom 7y ago
  The ROCm compilers are slightly different, the newest GCN arch is probably not well tested for compute yet. The ROCm stack's development and testing effort strongly focuses on their 'pro/server' line cards. Is there a Navi WX or the equivalent yet?
  - hgoel 7y ago
    There is a W5700 GPU but it was only made public about a month ago. I think they're still working on improving the base drivers for Navi so they haven't gotten to adding in proper support. Granted that I haven't really tried properly, as the card does show up as a compatible device, but trying to run Tensorflow on it fails due to associated GPU code not being available. Perhaps it doesn't have official support yet due to needing more compiler work.
- mroche 7y ago
  I’m not predicting support until there’s a server oriented model available. So 2020 with Navi 20 and the new Radeon Instinct cards. They do have a single Radeon Pro W card (Navi 10), but I think they’re waiting for their Tesla competitor to be ready. It’ll be interesting to see how these cards and ROCm perform in comparison to NVIDIA’s offering and CUDA.
deleted 7y ago
[deleted]
ilaksh 7y ago
If I want to do OpenCL on Linux with a new AMD GPU do I need to now separately install a Radeon Open Compute package before I install normal Open CL stuff?
- Athas 7y ago
  This is what I would recommend, but there are also other OpenCL stacks you could use. At least two from AMD (amdgpu-pro and ROCm), and and there is also one from the Mesa project (Clover).
duked 7y ago
would this help in running tensorflow on the latest MBP 16 ? If so any guide out there, the documentation is not super helpful
- darknoon 7y ago
  No, this is Linux-only.
  - rvz 7y ago
    My Linux-running Dual booting Macbook with an AMD GPU would disagree.
gnufx 7y ago
Note that one reason AMD have to make this stuff work is their big supercomputing win: https://www.amd.com/en/products/frontier https://www.amd.com/en/products/frontier I don't know if there's any relationship between that and the free software infrastructure, but it's a relief to see an alternative to the CUDA-ish stuff. In the absence of a shared library dummy interface, we can't distribute OS packages of performance engineering tools with NVIDIA support, for instance.

208 points by mroche 7y ago

deleted 7y ago

[deleted]

For people trying to install this on Debian/Ubuntu, the group that enables access to the gpgpu is actually "render" instead of "video" I spent some time wondering why I couldn't get access to the device until some kind soul pointed me to the different group name. https://fosstodon.org/@dctrud/103315274791608145 https://fosstodon.org/@dctrud/103315274791608145

app4soft 7y ago
Does ROC support ATI RS600 GPUs?
- microcolonel 7y ago
  No, only GCN and later, AFAIK.

neximo64 7y ago

I wish OS X would do better with AMD and also get it working with Tensorflow. No OS X support here sadly.

nknealk 7y ago
Check out PlaidML (from of all companies intel) to get AMD GPUs working with the Keras APIs for tensorflow https://github.com/plaidml/plaidml https://github.com/plaidml/plaidml
- neximo64 7y ago
  I've used this, unfortunately I've not had a good experience with it. Say I was to have a keras model, and use a tensorflow backend, and then use the plaidml backend and compare the results: They're different in a radical way. Also I don't think they've got their driver usage right. Metal on the GPU gets different results than OpenCL. OpenCL is supposed to be the 'beta' and the results are more stable with it - and again different radically between the PlaidML devices.
  - woadwarrior01 7y ago
    I remember running inference on a model that I trained on a pair of NVIDIA 1080TIs with a NVIDIA 2080TI (after upgrading) and observing slightly (albeit not significantly) different results. We’re dealing with single precision floats for everything. FWIW, every implementation of IEE 754 floats has its own set of minor oddities and quirks.
nileshk 7y ago
I wish macOS would allow for PCI passthrough to VMs (like you can do on Linux via OVMF, using QEMU). I wish I could connect an eGPU and pass full access of it to my Linux VM (or Windows VM), whether that GPU is AMD or NVidia shouldn't matter. That would remove the disadvantages Macs have when it comes to GPU and make it the best VM host (currently Linux seems to be the best option for VM host). I'm sure Mac Pro owners could take advantage of this were it possible. However, I suspect Apple has no intention of doing this, as they are more focus on pushing Metal and reliance on macOS. I wouldn't mind seeing PCI or GPU passthrough capabilities this in Windows 10 Pro (not just Server). With their Linux Subsystem progressing the way it is, they really have an opportunity here. Though since NVidia and CUDA are supported in Windows, maybe not as necessary as the situation on macOS.
ahartmetz 7y ago
You bought into a more closed ecosystem and that is what you have now.
shmerl 7y ago
Why do you need macOS? You can use Linux for your needs, it lets you do what you want and how you want it, instead of Apple telling you you can't do it.
slezyr 7y ago
That's entirely self inflicted wound by Apple they did their best to stop any vendor from developing the driver.

Athas 7y ago

For anyone else using NixOS, https://github.com/nixos-rocm/nixos-rocm https://github.com/nixos-rocm/nixos-rocm by Anthony Cowley works very well. While ROCm works well, I think it is yet another example of what AMD gets wrong software-wise. I like to think I'm fairly familiar with GPGPU, in that I have used OpenCL and CUDA for years, and even wrote my own optimising compiler that generates GPU code. I even use ROCm every day on my home system. Yet, I would be hard pressed to succinctly state what ROCm is. The GitHub page states the following: > ROCm is designed to be a universal platform for gpu-accelerated computing. This modular design allows hardware vendors to build drivers that support the ROCm framework. ROCm is also designed to integrate multiple programming languages and makes it easy to add support for other languages. Okay? It then lists some other stuff: > • Drivers > • Tools > • Libraries > • Source Code The AMD GPU kernel driver, amdgpu, is upstreamed in Linux, so what exactly does ROCm contain? The OpenCL ICD? (I'm pretty sure it does.) As far as I can see, I don't have any ROCm-specific kernel drivers loaded, but OpenCL on my AMD GPU works fine. The tools are also confusing. There's an "HCC compiler", but are we even still supposed to use HCC? As I recall, it was/is some heterogeneous compute C++ dialect. There's also HIP, which converts CUDA to something that can run on AMD GPUs. Trying to refresh my memory, I actually can't figure out in two minutes what language the output is. HCC? OpenCL? Among the other tools, there is inconsistency between ROCM and ROCm (pedantry, I know, I assume it's the same), but there is also the "ROC Profiler" and "ROCr Debug Agent". Are "ROC" and "ROCr" significant terms? And when should I use the "Radeon Compute Profiler" (rocm-profiler) and when should I use the "ROC Profiler" (rocprofiler-dev)? I do so dearly want to support AMD. Their hardware is good and their fully open source drivers are a wonderful accomplishment. Seriously, I want to underline just how amazed I am that I have a fully free high-performance GPGPU stack running. However, if you compare AMDs software efforts to NVIDIA, it is clear that NVIDIA deserves their success. It's not just about breadth (NVIDIA is bigger, I can understand they have more resources), but the fact that AMD seems to spread itself too thin over too many effort, and it makes everything feel kind of shoddy and poorly documented.

jedbrown 7y ago
> There's also HIP, which converts CUDA to something that can run on AMD GPUs. Kinda making your point, HIP is actually an (open source) one-to-one replacement for the CUDA API. While there are tools (https://github.com/ROCm-Developer-Tools/HIP/tree/master/hipify-clang https://github.com/ROCm-Developer-Tools/HIP/tree/master/hipi... and https://github.com/ROCm-Developer-Tools/HIP/blob/master/bin/hipify-perl https://github.com/ROCm-Developer-Tools/HIP/blob/master/bin/...) that convert CUDA to HIP, these are meant to be run once and perhaps touched up manually. HIP source can then be compiled (no human interaction) for AMD (ROCm) devices and NVIDIA (CUDA) devices. The CUDA brand refers to both the API and the device. AMD certainly could have done a better job, but wanted to distinguish these, presumably in hopes that existing users of the CUDA API would consider porting their code to HIP even when targeting CUDA devices.

e12e 7y ago

So is this AMDs response to CUDA? Is there any reason to believe this will end up practically cross-platform? Is the implication that opencl is dead?

Athas 7y ago
This contains an OpenCL implementation (with support for 2.0 at that!) and the kitchen sink. It's a bit unclear what AMD actually suggests you use for programming their GPUs.

continuational 7y ago

What is it, exactly?

Athas 7y ago
More or less AMD CUDA: a collection of compilers, tools, and libraries. The main difference is that NVIDIA markets CUDA strongly based on CUDA C++. You know the way you write CUDA is through nvcc and the language it accepts, unless you turn out to have exotic needs. Instead, AMD just provides a whole bunch of various things with fuzzy documentation, and lets you to pick whatever you like (although I'm not quite sure how you're supposed to know what you like).
- cwt137 7y ago
  Will this help any on getting acceleration for Tensorflow?
  - Athas 7y ago
    Yes, there is supposedly Tensorflow support in ROCm. I have never used it, so I cannot say how well it works.
    - dannyw 7y ago
      Just don't expect it to be as good as Tensorflow on CUDA.
      - hgoel 7y ago
        What makes you say that? It works perfectly fine and so far I haven't encountered anything I couldn't do with tensorflow-rocm that should be doable with Tensorflow on Nvidia
imtringued 7y ago
Well they have decided to put the most important information at the bottom of the readme... https://github.com/RadeonOpenCompute/ROCm/tree/roc-3.0.0#rocm-development-toolchains https://github.com/RadeonOpenCompute/ROCm/tree/roc-3.0.0#roc... It's a collection of GPU development tools and libraries.

bcatanzaro 7y ago

No Navi support? Surprising.

vetrom 7y ago
The ROCm compilers are slightly different, the newest GCN arch is probably not well tested for compute yet. The ROCm stack's development and testing effort strongly focuses on their 'pro/server' line cards. Is there a Navi WX or the equivalent yet?
- hgoel 7y ago
  There is a W5700 GPU but it was only made public about a month ago. I think they're still working on improving the base drivers for Navi so they haven't gotten to adding in proper support. Granted that I haven't really tried properly, as the card does show up as a compatible device, but trying to run Tensorflow on it fails due to associated GPU code not being available. Perhaps it doesn't have official support yet due to needing more compiler work.
mroche 7y ago
I’m not predicting support until there’s a server oriented model available. So 2020 with Navi 20 and the new Radeon Instinct cards. They do have a single Radeon Pro W card (Navi 10), but I think they’re waiting for their Tesla competitor to be ready. It’ll be interesting to see how these cards and ROCm perform in comparison to NVIDIA’s offering and CUDA.

deleted 7y ago

[deleted]

ilaksh 7y ago

If I want to do OpenCL on Linux with a new AMD GPU do I need to now separately install a Radeon Open Compute package before I install normal Open CL stuff?

Athas 7y ago
This is what I would recommend, but there are also other OpenCL stacks you could use. At least two from AMD (amdgpu-pro and ROCm), and and there is also one from the Mesa project (Clover).

duked 7y ago

would this help in running tensorflow on the latest MBP 16 ? If so any guide out there, the documentation is not super helpful

darknoon 7y ago
No, this is Linux-only.
- rvz 7y ago
  My Linux-running Dual booting Macbook with an AMD GPU would disagree.

gnufx 7y ago

Note that one reason AMD have to make this stuff work is their big supercomputing win: https://www.amd.com/en/products/frontier https://www.amd.com/en/products/frontier I don't know if there's any relationship between that and the free software infrastructure, but it's a relief to see an alternative to the CUDA-ish stuff. In the absence of a shared library dummy interface, we can't distribute OS packages of performance engineering tools with NVIDIA support, for instance.