9 ms·

For anyone else using NixOS, https://github.com/nixos-rocm/nixos-rocm https://github.com/nixos-rocm/nixos-rocm by Anthony Cowley works very well. While ROCm wo

by Athas 7y ago

For anyone else using NixOS, https://github.com/nixos-rocm/nixos-rocm https://github.com/nixos-rocm/nixos-rocm by Anthony Cowley works very well. While ROCm works well, I think it is yet another example of what AMD gets wrong software-wise. I like to think I'm fairly familiar with GPGPU, in that I have used OpenCL and CUDA for years, and even wrote my own optimising compiler that generates GPU code. I even use ROCm every day on my home system. Yet, I would be hard pressed to succinctly state what ROCm is. The GitHub page states the following: > ROCm is designed to be a universal platform for gpu-accelerated computing. This modular design allows hardware vendors to build drivers that support the ROCm framework. ROCm is also designed to integrate multiple programming languages and makes it easy to add support for other languages. Okay? It then lists some other stuff: > • Drivers > • Tools > • Libraries > • Source Code The AMD GPU kernel driver, amdgpu, is upstreamed in Linux, so what exactly does ROCm contain? The OpenCL ICD? (I'm pretty sure it does.) As far as I can see, I don't have any ROCm-specific kernel drivers loaded, but OpenCL on my AMD GPU works fine. The tools are also confusing. There's an "HCC compiler", but are we even still supposed to use HCC? As I recall, it was/is some heterogeneous compute C++ dialect. There's also HIP, which converts CUDA to something that can run on AMD GPUs. Trying to refresh my memory, I actually can't figure out in two minutes what language the output is. HCC? OpenCL? Among the other tools, there is inconsistency between ROCM and ROCm (pedantry, I know, I assume it's the same), but there is also the "ROC Profiler" and "ROCr Debug Agent". Are "ROC" and "ROCr" significant terms? And when should I use the "Radeon Compute Profiler" (rocm-profiler) and when should I use the "ROC Profiler" (rocprofiler-dev)? I do so dearly want to support AMD. Their hardware is good and their fully open source drivers are a wonderful accomplishment. Seriously, I want to underline just how amazed I am that I have a fully free high-performance GPGPU stack running. However, if you compare AMDs software efforts to NVIDIA, it is clear that NVIDIA deserves their success. It's not just about breadth (NVIDIA is bigger, I can understand they have more resources), but the fact that AMD seems to spread itself too thin over too many effort, and it makes everything feel kind of shoddy and poorly documented.

jedbrown 7y ago
> There's also HIP, which converts CUDA to something that can run on AMD GPUs. Kinda making your point, HIP is actually an (open source) one-to-one replacement for the CUDA API. While there are tools (https://github.com/ROCm-Developer-Tools/HIP/tree/master/hipify-clang https://github.com/ROCm-Developer-Tools/HIP/tree/master/hipi... and https://github.com/ROCm-Developer-Tools/HIP/blob/master/bin/hipify-perl https://github.com/ROCm-Developer-Tools/HIP/blob/master/bin/...) that convert CUDA to HIP, these are meant to be run once and perhaps touched up manually. HIP source can then be compiled (no human interaction) for AMD (ROCm) devices and NVIDIA (CUDA) devices. The CUDA brand refers to both the API and the device. AMD certainly could have done a better job, but wanted to distinguish these, presumably in hopes that existing users of the CUDA API would consider porting their code to HIP even when targeting CUDA devices.
- Athas 7y ago
  But CUDA C++ is a single-source programming language incorporating both CPU and GPU parts, so what is the one-to-one replacement for writing a kernel and launching it with the angle bracket syntax? Looking at the whitepaper[0] it looks like there is still some language extension going on to support the single-source case (the __global function). So HIP is also a C++ compiler frontend? [0]: https://gpuopen.com/wp-content/uploads/2016/01/7637_HIP_Datasheet_V1_7_PrintReady_US_WE.pdf https://gpuopen.com/wp-content/uploads/2016/01/7637_HIP_Data...
roenxi 7y ago
I'd like to highlight similar issues. I've been a fanatical AMD supporter since 2008 and refuse to buy or invest in learning about Nvidia GPUs because I honestly care about open source drivers even if that means giving up opportunities. I'm very interested in experimenting with neural networks as an amateur. I couldn't figure out what ROCM is, what it does, what parts of it I was meant to install (if anything?) and what was a brand name for coordinated upstream efforts in the linux kernel. Does it have an API? Who is meant to be using it for what? I dunno. I honestly don't understand. I don't know anything about CUDA because, as mentioned, no Nvidia GPU in a decade. My theory is if I pour all my effort into learning about Vulkan that will be good enough. I'll run tensorflow on my CPU or just do without and suffer for it. Whatever ROCMs use case and target audience is I hope it isn't me.
- jacquesm 7y ago
  Wow, you're principled.
  - roenxi 7y ago
    Aww, thanks :). It's easy for anyone to be principled if they don't need a GPU, although I've been blessed by nature to enjoy tracking the progress of Open Source up close more than having a working graphics card. If I wanted to be serious about getting in to AI research I'd probably need to buckle and buy Nvidia.
- teamspirit 7y ago
  https://rocm.github.io https://rocm.github.io maybe a good place to start. I've been meaning to checkout TensorFlow for ROCm myself but it's challenging finding general info like what cards are supported (I'm pretty sure Radeon VII is).
  - roenxi 7y ago
    The issue with that page is it assumes (1) that the reader is very familiar with CUDA and (2) has a feature checklist vs CUDA. Eg, there is an "Important features include the following" section where a typical example is "User-mode queues and DMA". We've had user mode queues since Sedgewick's Algorithms in C. Or "Large memory allocations" which we've had since the introduction of the 64 bit CPU. Obviously these must be features in the context of a GPU, but they don't really have a lot of explanatory power for anyone not already intimately familiar with the field. I suppose the obvious conclusion is ROCM is professionals only; but presumably those professionals are already perfectly happy with CUDA. I'd find it helpful to know what these professionals are expected to implement with all this so I can go figure that out. Specifically for me I know Vulkan (aka OpenGL++) is a general purpose API for running things on a GPU. I don't understand if that supersedes ROCM, compliments ROCM or what. I'm assuming that Vulkan is all anyone needs and then ROCM is going to ... provide something, maybe libraries? ... as middleware to Tensorflow and related tools. But blow me down if I can find anything that fills me with confidence that I've got the right take. ROCM seems to be very low level which makes little sense to me given that Vulkan (and the slowly obsoleting OpenCL) seem to be where the compute work is being done. If I wanted to summarise PyTorch it is "a Python API & library that provides convenience functions and data models for machine learning." Or CUDA, at a guess, is "a C++ API for highly parallel compute on Nvidia GPU". ROCm claims to be "the first open-source HPC/Hyperscale-class platform for GPU computing that’s also programming-language independent" and none of that means much to me. Doesn't hint what programming language I'm expecting to be using, and I don't think I even want Hyperscale computing. But I get the feeling I want ROCm because it is associated with PyTorch on AMD GPUs. Befuddling. I dunno, the ROCm version of PyTorch looked complicated and convoluted so I'm just going to sit it out and wait for someone to explain who the circus is performing for.
floatboth 7y ago
> I don't have any ROCm-specific kernel drivers loaded I'm pretty sure it needs amdkfd, which is not really "specific" but doesn't have any other consumers. > but OpenCL on my AMD GPU works fine Well, if instead of ROCm (or proprietary "pro" stuff) you have Mesa's Clover… that's the opposite of fine. I mean, fine for simple things, but Darktable won't work (no image support), Blender won't work (just too complex lol)… --- my casual user opinion: I HATE ALL the special compute-specific GPU APIs. I don't want a special fancy kernel driver that loads everything differently, I don't like OpenCL and other unusual special garbage. Please please please just use Vulkan compute shaders for everything. (And thankfully, people are starting to do that https://github.com/nihui/waifu2x-ncnn-vulkan https://github.com/nihui/waifu2x-ncnn-vulkan https://github.com/hanatos/vkdt https://github.com/hanatos/vkdt …)
sytelus 7y ago
I can never understand why AMD has not thrown its weight around to get support added in PyTorch and TensorFlow. NVidia is minting money on this by charging 2-4X on data center GPUs and eating theri lunch, dinner and breakfast. AMD could invest as little as 10 developers and get it done in few months. It's astounding that they can go all the way to do silicon, febs and just fall short of adding this final framework support. https://towardsdatascience.com/on-the-state-of-deep-learning-outside-of-cudas-walled-garden-d88c8bbb4342 https://towardsdatascience.com/on-the-state-of-deep-learning...
- m0zg 7y ago
  Because until very, very recently they didn't have much weight to throw around. They were hemorrhaging money. These things take time. CUDA took what, a decade to become what it is today? And it was developed by a company that wasn't on its deathbed. Arguably AMD still doesn't have much weight to throw around. Their profits are meager. Their P/E is absolutely insane for a hardware company (240!), suggesting a dramatic correction in the foreseeable future. Stocks on an upward trajectory help attract top talent. On a downward trajectory they actively repel it. So they probably have like half a dozen engineers working on this, and not those magical $500K/yr engineers that can actually turn shit into gold, because a hardware company wouldn't be able to hire them (or afford them, for that matter).
  - sudosysgen 7y ago
    AMD is a fabless semiconductors company. They are more akin to low-level software than to the majority of hardware companies.
    - m0zg 7y ago
      Not really, no. Unlike a software company it still costs them a significant amount of cold hard cash to produce each unit they sell. Captial intensive AF, especially at 7nm. They don't have to build their own fabs, but they DO have to pay for those fabs to be built, one way or another. Being fabless doesn't free you from that.
    - 7y ago
justicezyx 7y ago
What kind of compiler you have wetting for GPU? I am curious on what's the motivation given that nvcc is fairly competent.
danieldk 7y ago
For anyone else using NixOS, https://github.com/nixos-rocm/nixos-rocm https://github.com/nixos-rocm/nixos-rocm by Anthony Cowley works very well. Not just NixOS. Our work AMD GPU server uses Ubuntu 18.04, but I use Anthony's package set for easy use of PyTorch and Tensorflow in various environments (using nix-shell + direnv).
Proven 7y ago
If they could fix most of these problems, their products wouldn't be so cheap.

by Athas 7y ago

jedbrown 7y ago

> There's also HIP, which converts CUDA to something that can run on AMD GPUs. Kinda making your point, HIP is actually an (open source) one-to-one replacement for the CUDA API. While there are tools (https://github.com/ROCm-Developer-Tools/HIP/tree/master/hipify-clang https://github.com/ROCm-Developer-Tools/HIP/tree/master/hipi... and https://github.com/ROCm-Developer-Tools/HIP/blob/master/bin/hipify-perl https://github.com/ROCm-Developer-Tools/HIP/blob/master/bin/...) that convert CUDA to HIP, these are meant to be run once and perhaps touched up manually. HIP source can then be compiled (no human interaction) for AMD (ROCm) devices and NVIDIA (CUDA) devices. The CUDA brand refers to both the API and the device. AMD certainly could have done a better job, but wanted to distinguish these, presumably in hopes that existing users of the CUDA API would consider porting their code to HIP even when targeting CUDA devices.

Athas 7y ago
But CUDA C++ is a single-source programming language incorporating both CPU and GPU parts, so what is the one-to-one replacement for writing a kernel and launching it with the angle bracket syntax? Looking at the whitepaper[0] it looks like there is still some language extension going on to support the single-source case (the __global function). So HIP is also a C++ compiler frontend? [0]: https://gpuopen.com/wp-content/uploads/2016/01/7637_HIP_Datasheet_V1_7_PrintReady_US_WE.pdf https://gpuopen.com/wp-content/uploads/2016/01/7637_HIP_Data...

roenxi 7y ago

I'd like to highlight similar issues. I've been a fanatical AMD supporter since 2008 and refuse to buy or invest in learning about Nvidia GPUs because I honestly care about open source drivers even if that means giving up opportunities. I'm very interested in experimenting with neural networks as an amateur. I couldn't figure out what ROCM is, what it does, what parts of it I was meant to install (if anything?) and what was a brand name for coordinated upstream efforts in the linux kernel. Does it have an API? Who is meant to be using it for what? I dunno. I honestly don't understand. I don't know anything about CUDA because, as mentioned, no Nvidia GPU in a decade. My theory is if I pour all my effort into learning about Vulkan that will be good enough. I'll run tensorflow on my CPU or just do without and suffer for it. Whatever ROCMs use case and target audience is I hope it isn't me.

jacquesm 7y ago
Wow, you're principled.
- roenxi 7y ago
  Aww, thanks :). It's easy for anyone to be principled if they don't need a GPU, although I've been blessed by nature to enjoy tracking the progress of Open Source up close more than having a working graphics card. If I wanted to be serious about getting in to AI research I'd probably need to buckle and buy Nvidia.
teamspirit 7y ago
https://rocm.github.io https://rocm.github.io maybe a good place to start. I've been meaning to checkout TensorFlow for ROCm myself but it's challenging finding general info like what cards are supported (I'm pretty sure Radeon VII is).
- roenxi 7y ago
  The issue with that page is it assumes (1) that the reader is very familiar with CUDA and (2) has a feature checklist vs CUDA. Eg, there is an "Important features include the following" section where a typical example is "User-mode queues and DMA". We've had user mode queues since Sedgewick's Algorithms in C. Or "Large memory allocations" which we've had since the introduction of the 64 bit CPU. Obviously these must be features in the context of a GPU, but they don't really have a lot of explanatory power for anyone not already intimately familiar with the field. I suppose the obvious conclusion is ROCM is professionals only; but presumably those professionals are already perfectly happy with CUDA. I'd find it helpful to know what these professionals are expected to implement with all this so I can go figure that out. Specifically for me I know Vulkan (aka OpenGL++) is a general purpose API for running things on a GPU. I don't understand if that supersedes ROCM, compliments ROCM or what. I'm assuming that Vulkan is all anyone needs and then ROCM is going to ... provide something, maybe libraries? ... as middleware to Tensorflow and related tools. But blow me down if I can find anything that fills me with confidence that I've got the right take. ROCM seems to be very low level which makes little sense to me given that Vulkan (and the slowly obsoleting OpenCL) seem to be where the compute work is being done. If I wanted to summarise PyTorch it is "a Python API & library that provides convenience functions and data models for machine learning." Or CUDA, at a guess, is "a C++ API for highly parallel compute on Nvidia GPU". ROCm claims to be "the first open-source HPC/Hyperscale-class platform for GPU computing that’s also programming-language independent" and none of that means much to me. Doesn't hint what programming language I'm expecting to be using, and I don't think I even want Hyperscale computing. But I get the feeling I want ROCm because it is associated with PyTorch on AMD GPUs. Befuddling. I dunno, the ROCm version of PyTorch looked complicated and convoluted so I'm just going to sit it out and wait for someone to explain who the circus is performing for.

floatboth 7y ago

> I don't have any ROCm-specific kernel drivers loaded I'm pretty sure it needs amdkfd, which is not really "specific" but doesn't have any other consumers. > but OpenCL on my AMD GPU works fine Well, if instead of ROCm (or proprietary "pro" stuff) you have Mesa's Clover… that's the opposite of fine. I mean, fine for simple things, but Darktable won't work (no image support), Blender won't work (just too complex lol)… --- my casual user opinion: I HATE ALL the special compute-specific GPU APIs. I don't want a special fancy kernel driver that loads everything differently, I don't like OpenCL and other unusual special garbage. Please please please just use Vulkan compute shaders for everything. (And thankfully, people are starting to do that https://github.com/nihui/waifu2x-ncnn-vulkan https://github.com/nihui/waifu2x-ncnn-vulkan https://github.com/hanatos/vkdt https://github.com/hanatos/vkdt …)

sytelus 7y ago

I can never understand why AMD has not thrown its weight around to get support added in PyTorch and TensorFlow. NVidia is minting money on this by charging 2-4X on data center GPUs and eating theri lunch, dinner and breakfast. AMD could invest as little as 10 developers and get it done in few months. It's astounding that they can go all the way to do silicon, febs and just fall short of adding this final framework support. https://towardsdatascience.com/on-the-state-of-deep-learning-outside-of-cudas-walled-garden-d88c8bbb4342 https://towardsdatascience.com/on-the-state-of-deep-learning...

m0zg 7y ago
Because until very, very recently they didn't have much weight to throw around. They were hemorrhaging money. These things take time. CUDA took what, a decade to become what it is today? And it was developed by a company that wasn't on its deathbed. Arguably AMD still doesn't have much weight to throw around. Their profits are meager. Their P/E is absolutely insane for a hardware company (240!), suggesting a dramatic correction in the foreseeable future. Stocks on an upward trajectory help attract top talent. On a downward trajectory they actively repel it. So they probably have like half a dozen engineers working on this, and not those magical $500K/yr engineers that can actually turn shit into gold, because a hardware company wouldn't be able to hire them (or afford them, for that matter).
- sudosysgen 7y ago
  AMD is a fabless semiconductors company. They are more akin to low-level software than to the majority of hardware companies.
  - m0zg 7y ago
    Not really, no. Unlike a software company it still costs them a significant amount of cold hard cash to produce each unit they sell. Captial intensive AF, especially at 7nm. They don't have to build their own fabs, but they DO have to pay for those fabs to be built, one way or another. Being fabless doesn't free you from that.
  - 7y ago

justicezyx 7y ago

What kind of compiler you have wetting for GPU? I am curious on what's the motivation given that nvcc is fairly competent.

danieldk 7y ago

For anyone else using NixOS, https://github.com/nixos-rocm/nixos-rocm https://github.com/nixos-rocm/nixos-rocm by Anthony Cowley works very well. Not just NixOS. Our work AMD GPU server uses Ubuntu 18.04, but I use Anthony's package set for easy use of PyTorch and Tensorflow in various environments (using nix-shell + direnv).

Proven 7y ago

If they could fix most of these problems, their products wouldn't be so cheap.