10 ms·

A dive into the AMD driver workflow

153 points by tikkun 3y ago

sarakasimova 3y ago
[flagged]
dur-randir 3y ago
That's why "ROCm support" is pure marketing and FUD. And people rightfully won't buy into it.
tikkun 3y ago
A possible prescription for AMD regarding AI and CUDA: 1) Open-source driver development as mentioned in this post 2) Set up 24/7 free software tech support on discord. Maybe for all use cases, maybe only AI use cases. Do the tech support via screen sharing, and have a writer join for all calls, so that every issue once solved gets a blog post 3) Have employees run all popular AI tools and get them working on AMD hardware, publish written guides and videos showing how to do it.
- zirgs 3y ago
  4) Release a consumer GPU with 32-80 GB of VRAM.
  - ciupicri 3y ago
    What are you planning to play on that thing?
    - freedomben 3y ago
      I think Hogwart's Legacy could benefit from it. It crashes on my 16GB system due to being out of video memory regularly. It might be lying about the cause but that's what it claims.
    - Filligree 3y ago
      Llama 2.
  - Tuna-Fish 3y ago
    It sounds silly, but people would endure a lot of pain to fit significantly larger models into ram.
  - fbdab103 3y ago
    It could even be 2-3 generations behind and that thing would still sell.
  - latchkey 3y ago
    Or at least make it so that you could rent the larger gpus by the hour. As far as I know, there isn't a single service that offers bare metal access to MI210/MI250's.
  - codedokode 3y ago
    AMD has CPUs with built-in GPU with hundreds of ALUs and unified memory (GPU uses system memory). Cannot such CPU be used for ML tasks without purchasing expensive graphic cards? 64 Gb of ordinary RAM will be cheaper than 64 Gb of VRAM. Currently such built-in GPUs seem to be optimized for games only.
    - dur-randir 3y ago
      VRAM speed is a major factor. Even ddr5 with 4 channels would be slow compared to current GDDRs.
      - codedokode 3y ago
        That's correct, but if you are, for example, multiplying two 1024x1024 matrices (typical for ML loads) then you need to perform roughly 1B operations on 1M numbers. There is no need for loading lot of data. So we can do without fast memory in this case, can't we? Hopefully the cache and registers of GPU can handle this.
- hedgehog 3y ago
  The problem is not that people within the company don't have good ideas of how to improve deep learning research end user experience, it's just not a priority for AMD. It's annoying as a potential customer but arguably whatever their overall strategy is it's working.
  - deleted 3y ago
    [deleted]
- lul_open 3y ago
  Yes, let's build an open source community on top of a closed platform. It's not like twitter, reddit, Facebook etc have taught us anything. Open a mailing list like every project that survives more than 5 years. Hell the barrier to entry will ensure you get people who can use a text editor and save you half the question of 'how do I install this on on an Intel a2 Mac???'
  - branon 3y ago
    Yes, absolutely do not use Discord for this, Matrix would be better
oldgradstudent 3y ago
> A note culturally, I do sadly feel like what they responded to was george is upset and saying bad things about us which is bad for our brand and not holy shit we have a broken driver released panicing thousands of people’s kernels and crashing their GPUs. But it’s a start.
- LargeTomato 3y ago
  George is right. But He has a history of being an arrogant prick. That will color people's perceptions of you even if it's not relevant to the immediate interaction.
  - brucethemoose2 3y ago
    I didn't even know who Hotz was, but that YouTube rant on this very issue made me extremely skeptical. Should someone who will just drop a GPU vendor if they get flustered really be leading a ML framework?
    - vbezhenar 3y ago
      You're welcome to fork and find another leader, I guess.
      - brucethemoose2 3y ago
        I don't think some stability or a formal decision making structure is too much to ask. TVM, MLIR based projects, GGML seem to have stable leadership.
    - BearOso 3y ago
      Hotz is known for hacking the iPhone and PS3. But you should look deeper into that before giving him any credit. His post is mostly hot air. AMD driver development is very open, excepting only ROCm. Not knowing where things are discussed or how the various driver systems, excluding Nvidia, interact on Linux is not an excuse to rant. The corporate amd kernel development starts amd-staging-drm-next, with internal AMD pull requests available on the mailing list or here: https://patchwork.freedesktop.org/project/amd-xorg-ddx/series/?ordering=-last_updated https://patchwork.freedesktop.org/project/amd-xorg-ddx/serie..., before going through airlied, the maintainer of drm in the main kernel, then to Linus. Everything user-space regarding OpenGL/Vulkan and some rusty OpenCL, aside from the proprietary amdgpu-pro driver which should almost never be used, is in Mesa. ROCm is the only thing with huge code dumps, obviously because it's a new effort, and it's said outright that it's unsupported for consumer GPUs. Yes, it's a buggy mess. Did his issue warrant the corporate response he got? No.
znpy 3y ago
> And in order to beat them, you must be playing to win, not just playing not to lose. Combine the driver openness with public hardware docs and you have a competitive advantage. This is so immensely true. I’ve been so enthusiastically loyal to intel gpus in the last ten years or so (avoiding laptops with any kind of discrete gpus) because dealing with closed drivers is so much pain. I’m still skeptical about amd gpus, even though i hear good stuff about it. I kist want hardware i can trust, knowing I won’t have dumb driver issues.
WesolyKubeczek 3y ago
I knew for years that AMD’s development model leaves a lot to be desired. They like to throw big balls of source over the wall, and very soon after the bugs that keep haunting the previous generation of hardware just stop getting fixed. You’re SOL unless Dave Airlie himself runs into identical problems on his personal gear and gets angry enough about it to make a fix.
- iforgotpassword 3y ago
  Yeah, I've ranted about this here in the past. I'm glad someone high-profile enough is finally doing the same, as that might actually lead somewhere. Their closed development process is so moronic. It means their code is always ahead and out of sync with what is in public. There have been user provided fixes and improvements to ROCm on GitHub with no reaction from AMD. Probably because it wouldn't apply cleanly to whatever they currently have. It's sad to see your customers having to fix your drivers. It's even sadder to see you ignore it.
  - ethbr0 3y ago
    This is the greatest travesty of closed/open models, IMHO. I get exactly what happens -- devs are shipping their sprint pipeline internally, and pull requests or issues from the current open source head never make it onto their radar. This gatekeeps bugs visibility behind whatever PM is running sprint planning, which always leads to delivering what management wants instead of what users are dealing with. In complex, diverse environments, you're always going to have a myriad of bugs that are caused by a specific configuration. Absent a path to actually fix that, you end up with an enterprise-only product that's only stable in extremely specific configurations. And the greatest evil... you're also ignoring helpful reports of configurations where it's broken!
Roark66 3y ago
You can say a lot about nVidia, but for me all their products mostly just work on Linux(I use cuda a lot). I don't understand why AMD is having such trouble doing the same. Likewise with CPUs. It is ironic I have to be using Intel's math libraries to get good performance out of my AMD CPU.
- xbeuxhedovksb 3y ago
  Having a Nvidia Linux gaming setup, from my experience it doesn't work well at all : broken wayland, no hw video enc, etc. Sure some of those could be solved by sending weeks trying hacks things like VA-API over NVDEC emulation, but at the end of the day this is waste of time, tl:dr everything works on Nvidia Windows, many things broken on Nvidia Linux. NVidia on Linux is only good for non desktop use : cuda/compute/ml over ssh. AMD is the exact opposite, desktop things work better except there is no cuda/compute/ml So if you want something that works over the whole spectrum : desktop, wayland, compute/ml, video enc/dec, screen sharing, etc. it doesn't exist on Linux
- shmerl 3y ago
  For gaming and desktop usage, situation is completely the opposite. Nvidia is plagued by having not upstreamed drivers, and AMD just works.
  - amlib 3y ago
    really on linux you would be better off having a primary amd or intel gpu and then a secondary nvidia gpu just for compute/cuda tasks. But alas, many new am5 motherboards don't even have a secondary 16x pci-e slot capable of even pcie-4 4x bandwidth anymore. I guess we should thank this cloud compute craze for that...
    - shmerl 3y ago
      Some like Asrock reduced the number of PCIe slots and increased the number of USB 4 ports. So there is an option at least to use an external GPU now, since that's where they are re-allocating available bandwidth it seems.
aliljet 3y ago
I'm so thoroughly confused about why AMD wouldn't be falling over themselves to enable geohot and his followers to build an alternative to CUDA and NVIDIA. This feels like a conversation that geohot is attempting with feckless product and software managers who certainly can't make bold decisions. Has the CEO of AMD effectively spoken about this problem?
- izacus 3y ago
  Why would they spend limited resources on a fight they don't want? Part of leading a company is knowing which markets to pass up.
  - fragmede 3y ago
    You say that like CEOs are infallible. Thorsten Heins and Antonio M. Perez would like to have a word with you.
ddtaylor 3y ago
ROCm is a bit of a mess. I have used it successfully on a 6700 XT without issue but it was confusing because I thought I would need a "pro" driver but I did not.
codedokode 3y ago
Knowledgable people, please explain. Vulkan allows executing code on GPU. Can it be used for typical ML tasks (e.g. large matrix multiplication)? Why do we need drivers like CUDA or ROCm then?
- raphlinus 3y ago
  I think we'll get there but it will take years to arrive at a robust solution, because of the complexity of the ecosystem. The cooperative matrix extension, the portable and standard way to do what is usually marketed as "tensor cores," just landed in the spec and drivers a couple months ago. It will take people time to figure out how to use them effectively, partly because you have to query which options are supported at runtime. There's still a driver and you're still at mercy of it (bugs and all), it's just much more likely to be installed on your Windows or Linux machine than a ROCm stack.
- boustrophedon 3y ago
  To answer your second question directly: CUDA predates Vulkan. So if your question then becomes: Why did we need Vulkan if we have CUDA? OpenGL, DirectX, Metal, and Vulkan all have lots of graphics-specific concepts built in to their APIs. While you could probably implement most of this in CUDA, at some level you need hardware support for your graphics APIs because in the end the data in your framebuffer goes from the GPU's memory to your monitor via HDMI or DisplayPort. CUDA has no interface for that graphics-specific stuff (technically there are CUDA-OpenGL interop APIs inside CUDA) because it has to be able to run on AI acceleration hardware in datacenters (although that hardware didn't exist when CUDA was invented either, to be fair)

153 points by tikkun 3y ago

sarakasimova 3y ago

[flagged]

dur-randir 3y ago

That's why "ROCm support" is pure marketing and FUD. And people rightfully won't buy into it.

tikkun 3y ago

A possible prescription for AMD regarding AI and CUDA: 1) Open-source driver development as mentioned in this post 2) Set up 24/7 free software tech support on discord. Maybe for all use cases, maybe only AI use cases. Do the tech support via screen sharing, and have a writer join for all calls, so that every issue once solved gets a blog post 3) Have employees run all popular AI tools and get them working on AMD hardware, publish written guides and videos showing how to do it.

zirgs 3y ago
4) Release a consumer GPU with 32-80 GB of VRAM.
- ciupicri 3y ago
  What are you planning to play on that thing?
  - freedomben 3y ago
    I think Hogwart's Legacy could benefit from it. It crashes on my 16GB system due to being out of video memory regularly. It might be lying about the cause but that's what it claims.
  - Filligree 3y ago
    Llama 2.
- Tuna-Fish 3y ago
  It sounds silly, but people would endure a lot of pain to fit significantly larger models into ram.
- fbdab103 3y ago
  It could even be 2-3 generations behind and that thing would still sell.
- latchkey 3y ago
  Or at least make it so that you could rent the larger gpus by the hour. As far as I know, there isn't a single service that offers bare metal access to MI210/MI250's.
- codedokode 3y ago
  AMD has CPUs with built-in GPU with hundreds of ALUs and unified memory (GPU uses system memory). Cannot such CPU be used for ML tasks without purchasing expensive graphic cards? 64 Gb of ordinary RAM will be cheaper than 64 Gb of VRAM. Currently such built-in GPUs seem to be optimized for games only.
  - dur-randir 3y ago
    VRAM speed is a major factor. Even ddr5 with 4 channels would be slow compared to current GDDRs.
    - codedokode 3y ago
      That's correct, but if you are, for example, multiplying two 1024x1024 matrices (typical for ML loads) then you need to perform roughly 1B operations on 1M numbers. There is no need for loading lot of data. So we can do without fast memory in this case, can't we? Hopefully the cache and registers of GPU can handle this.
hedgehog 3y ago
The problem is not that people within the company don't have good ideas of how to improve deep learning research end user experience, it's just not a priority for AMD. It's annoying as a potential customer but arguably whatever their overall strategy is it's working.
- deleted 3y ago
  [deleted]
lul_open 3y ago
Yes, let's build an open source community on top of a closed platform. It's not like twitter, reddit, Facebook etc have taught us anything. Open a mailing list like every project that survives more than 5 years. Hell the barrier to entry will ensure you get people who can use a text editor and save you half the question of 'how do I install this on on an Intel a2 Mac???'
- branon 3y ago
  Yes, absolutely do not use Discord for this, Matrix would be better

oldgradstudent 3y ago

> A note culturally, I do sadly feel like what they responded to was george is upset and saying bad things about us which is bad for our brand and not holy shit we have a broken driver released panicing thousands of people’s kernels and crashing their GPUs. But it’s a start.

LargeTomato 3y ago
George is right. But He has a history of being an arrogant prick. That will color people's perceptions of you even if it's not relevant to the immediate interaction.
- brucethemoose2 3y ago
  I didn't even know who Hotz was, but that YouTube rant on this very issue made me extremely skeptical. Should someone who will just drop a GPU vendor if they get flustered really be leading a ML framework?
  - vbezhenar 3y ago
    You're welcome to fork and find another leader, I guess.
    - brucethemoose2 3y ago
      I don't think some stability or a formal decision making structure is too much to ask. TVM, MLIR based projects, GGML seem to have stable leadership.
  - BearOso 3y ago
    Hotz is known for hacking the iPhone and PS3. But you should look deeper into that before giving him any credit. His post is mostly hot air. AMD driver development is very open, excepting only ROCm. Not knowing where things are discussed or how the various driver systems, excluding Nvidia, interact on Linux is not an excuse to rant. The corporate amd kernel development starts amd-staging-drm-next, with internal AMD pull requests available on the mailing list or here: https://patchwork.freedesktop.org/project/amd-xorg-ddx/series/?ordering=-last_updated https://patchwork.freedesktop.org/project/amd-xorg-ddx/serie..., before going through airlied, the maintainer of drm in the main kernel, then to Linus. Everything user-space regarding OpenGL/Vulkan and some rusty OpenCL, aside from the proprietary amdgpu-pro driver which should almost never be used, is in Mesa. ROCm is the only thing with huge code dumps, obviously because it's a new effort, and it's said outright that it's unsupported for consumer GPUs. Yes, it's a buggy mess. Did his issue warrant the corporate response he got? No.

znpy 3y ago

> And in order to beat them, you must be playing to win, not just playing not to lose. Combine the driver openness with public hardware docs and you have a competitive advantage. This is so immensely true. I’ve been so enthusiastically loyal to intel gpus in the last ten years or so (avoiding laptops with any kind of discrete gpus) because dealing with closed drivers is so much pain. I’m still skeptical about amd gpus, even though i hear good stuff about it. I kist want hardware i can trust, knowing I won’t have dumb driver issues.

WesolyKubeczek 3y ago

I knew for years that AMD’s development model leaves a lot to be desired. They like to throw big balls of source over the wall, and very soon after the bugs that keep haunting the previous generation of hardware just stop getting fixed. You’re SOL unless Dave Airlie himself runs into identical problems on his personal gear and gets angry enough about it to make a fix.

iforgotpassword 3y ago
Yeah, I've ranted about this here in the past. I'm glad someone high-profile enough is finally doing the same, as that might actually lead somewhere. Their closed development process is so moronic. It means their code is always ahead and out of sync with what is in public. There have been user provided fixes and improvements to ROCm on GitHub with no reaction from AMD. Probably because it wouldn't apply cleanly to whatever they currently have. It's sad to see your customers having to fix your drivers. It's even sadder to see you ignore it.
- ethbr0 3y ago
  This is the greatest travesty of closed/open models, IMHO. I get exactly what happens -- devs are shipping their sprint pipeline internally, and pull requests or issues from the current open source head never make it onto their radar. This gatekeeps bugs visibility behind whatever PM is running sprint planning, which always leads to delivering what management wants instead of what users are dealing with. In complex, diverse environments, you're always going to have a myriad of bugs that are caused by a specific configuration. Absent a path to actually fix that, you end up with an enterprise-only product that's only stable in extremely specific configurations. And the greatest evil... you're also ignoring helpful reports of configurations where it's broken!

Roark66 3y ago

You can say a lot about nVidia, but for me all their products mostly just work on Linux(I use cuda a lot). I don't understand why AMD is having such trouble doing the same. Likewise with CPUs. It is ironic I have to be using Intel's math libraries to get good performance out of my AMD CPU.

xbeuxhedovksb 3y ago
Having a Nvidia Linux gaming setup, from my experience it doesn't work well at all : broken wayland, no hw video enc, etc. Sure some of those could be solved by sending weeks trying hacks things like VA-API over NVDEC emulation, but at the end of the day this is waste of time, tl:dr everything works on Nvidia Windows, many things broken on Nvidia Linux. NVidia on Linux is only good for non desktop use : cuda/compute/ml over ssh. AMD is the exact opposite, desktop things work better except there is no cuda/compute/ml So if you want something that works over the whole spectrum : desktop, wayland, compute/ml, video enc/dec, screen sharing, etc. it doesn't exist on Linux
shmerl 3y ago
For gaming and desktop usage, situation is completely the opposite. Nvidia is plagued by having not upstreamed drivers, and AMD just works.
- amlib 3y ago
  really on linux you would be better off having a primary amd or intel gpu and then a secondary nvidia gpu just for compute/cuda tasks. But alas, many new am5 motherboards don't even have a secondary 16x pci-e slot capable of even pcie-4 4x bandwidth anymore. I guess we should thank this cloud compute craze for that...
  - shmerl 3y ago
    Some like Asrock reduced the number of PCIe slots and increased the number of USB 4 ports. So there is an option at least to use an external GPU now, since that's where they are re-allocating available bandwidth it seems.

aliljet 3y ago

I'm so thoroughly confused about why AMD wouldn't be falling over themselves to enable geohot and his followers to build an alternative to CUDA and NVIDIA. This feels like a conversation that geohot is attempting with feckless product and software managers who certainly can't make bold decisions. Has the CEO of AMD effectively spoken about this problem?

izacus 3y ago
Why would they spend limited resources on a fight they don't want? Part of leading a company is knowing which markets to pass up.
- fragmede 3y ago
  You say that like CEOs are infallible. Thorsten Heins and Antonio M. Perez would like to have a word with you.

ddtaylor 3y ago

ROCm is a bit of a mess. I have used it successfully on a 6700 XT without issue but it was confusing because I thought I would need a "pro" driver but I did not.

codedokode 3y ago

Knowledgable people, please explain. Vulkan allows executing code on GPU. Can it be used for typical ML tasks (e.g. large matrix multiplication)? Why do we need drivers like CUDA or ROCm then?

raphlinus 3y ago
I think we'll get there but it will take years to arrive at a robust solution, because of the complexity of the ecosystem. The cooperative matrix extension, the portable and standard way to do what is usually marketed as "tensor cores," just landed in the spec and drivers a couple months ago. It will take people time to figure out how to use them effectively, partly because you have to query which options are supported at runtime. There's still a driver and you're still at mercy of it (bugs and all), it's just much more likely to be installed on your Windows or Linux machine than a ROCm stack.
boustrophedon 3y ago
To answer your second question directly: CUDA predates Vulkan. So if your question then becomes: Why did we need Vulkan if we have CUDA? OpenGL, DirectX, Metal, and Vulkan all have lots of graphics-specific concepts built in to their APIs. While you could probably implement most of this in CUDA, at some level you need hardware support for your graphics APIs because in the end the data in your framebuffer goes from the GPU's memory to your monitor via HDMI or DisplayPort. CUDA has no interface for that graphics-specific stuff (technically there are CUDA-OpenGL interop APIs inside CUDA) because it has to be able to run on AI acceleration hardware in datacenters (although that hardware didn't exist when CUDA was invented either, to be fair)