8 ms·

AMD Publishes Open-Source Driver for GPU Virtualization, Radeon "In the Roadmap"

196 points by davidlt 1y ago

janpmz 1y ago
This article is almost unreadable for me. The ads change in size and make the text jump. I'm adding it to NotebookLM now.
- Mountain_Skies 1y ago
  The article is extremely light on details anyway. The most important thing in it is the link to the repo at https://github.com/amd/MxGPU-Virtualization https://github.com/amd/MxGPU-Virtualization
- DistractionRect 1y ago
  Seems like using a sledge hammer to pound a nail. Why not just use an ad blocker like ublock origin?
proxysna 1y ago
That's pretty sick. Nice to see such things trickle down to consumer GPU's.
seanhunter 1y ago
It blows my mind how reliably AMD shoots itself in the foot. What we want isn’t that hard: 1) Support your graphics cards on linux using kernel drivers that you upstream. All of them. Not just a handful - all the ones you sell from say 18 months ago till today. 2) Make GPU acceleration actually work out of the box for pytorch and tensorflow. Not some special fork, patched version that you “maintain” on your website, the tip of the main branch for both of those libraries should just compile out of the box and give people gpu-accelerated ML. This is table stakes but it blows my mind that they keep making press releases and promises like this that things are on the roadmap without doing thing one and unfucking the basic dev experience so people can actually use their GPUs for real work. How it actually is: 1) Some cards work with rocm, some cards work with one of the other variations of BS libraries they have come up with over the years. Some cards work with amdgpu but many only work with proprietary kernel drivers which means if you don’t use precisely one of the distributions and kernel versions that they maintain you are sool. 2) Nothing whatsoever builds out of the box and when you get it to build almost nothing runs gpu accelerated. For me, pytorch requires a special downgrade, a python downgrade and a switch to a fork that AMD supposedly maintain although it doesn’t compile for me and when I managed to beat it into a shape where it compiled it wouldn’t run GPU accelerated even though games use the GPU just fine. I have a GPU that is supposedly current, so they are actively selling it, but can I use it? Can I bollocks. Ollama won’t talk to my GPU even though it supposedly works with ROCm. It only works with ROCm with some graphics cards. Tensorflow similar story when I last tried it although admittedly I didn’t try as hard as pytorch. Just make your shit work so that people can use it. It really shouldn’t be that hard. The dev experience with NVidia is a million times better.
- faust201 1y ago
  IIRC there was only one AMD employee that was working to integrate linux based things. Often, the response was - things are stuck in Intellectual property, or project managers etc. So even specs were not available.
- logicchains 1y ago
  SemiAnalysis had a good article on this recently, basically the reason AMD still sucks on the ML software side is that their compensation for devs is significantly worse than competitors like NVidia, Google and OpenAI, so most of the most competent devs go elsewhere.
- bayindirh 1y ago
  AMD has two driver teams at this point. One of Linux/Open Source, one for Catalyst/Closed source, and they are not allowed to interact. Because, there are tons of IP and trade secrets involved in driver development and optimization. Sometimes game related, sometimes for patching a rogue application which developers can't or don't fix, etc. etc. GPU drivers are ought to be easy, but in reality, they are not. The open source drivers are "vanilla" drivers without all these case-dependent patching and optimization. Actually, they really work well out of the box for normal desktop applications. I don't think there are any cards which do (or will) not work with the open source kernel drivers as long as you use a sufficiently recent version. ...and you mention ROCm. I'm not sure how ROCm's intellectual underpinnings are but, claiming lack of effort is a bit unfair to AMD. Yes, software was never their strong suit, but they're way better when compared to 20 years earlier. They have a proper open source driver which works, and a whole fleet of open source ROCm packages, which is very rigorously CI/CD tested by their maintainers now. Do not forget that some of the world's most powerful supercomputers run on Instinct cards, and AMD is getting tons of experience from these big players. If you think the underpinnings of GPGPU libraries are easy, I can only say that the reality is very different. The simple things people do with PyTorch and other very high level libraries pull enormous tricks under the hood, and you're really pushing the boundaries of the hardware performance and capability-wise. NVIDIA is not selling a tray full of switches and GPUs and require OEMs to integrate it as-is for no reason. On the other hand, the same NVIDIA acts very slowly to enable an open source ecosystem. So, yes, AMD is not in an ideal position right now, but calling them incompetent doesn't help either. P.S.: The company which fought for a completely open source HDMI 2.1 capable display driver is AMD, not NVIDIA.
throwaway48476 1y ago
If AMD does deliver on client dGPU virtualization it would be amazing.
- transpute 1y ago
  Some old AMD workstation GPUs supported SR-IOV, that repo was just archived. https://open-iov.org/index.php/GPU_Support#AMD https://open-iov.org/index.php/GPU_Support#AMD https://github.com/GPUOpen-LibrariesAndSDKs/MxGPU-Virtualization https://github.com/GPUOpen-LibrariesAndSDKs/MxGPU-Virtualiza... As "AI" use cases mature, NPU/AIE-ML virtualization will also be needed.
  - mtillman 1y ago
    The pro vii (a wonderfully spec’d card for my purposes) had a bunch of awesome features-even remote access-but then they pulled the marketing pages from their website and stopped shipping the link bridge (non mpx) to water down the value of the card. I have no idea how AMD works.
AbuAssar 1y ago
related: (AMD 2.0 – New Sense of Urgency) https://news.ycombinator.com/item?id=43780972 https://news.ycombinator.com/item?id=43780972
- bryanlarsen 1y ago
  Wow, that's a very substantive article.
- okucu 1y ago
  [flagged]
deleted 1y ago
[deleted]
curtisszmania 1y ago
[dead]
404human 1y ago
Anyone else find it funny that AMD releases new GPU features while people still can't get basic ML stuff working? It's like building a fancy garage before fixing the broken car.
- Nullabillity 1y ago
  Graphics work just fine. Y'know, the G in the name.
  - Redoubts 1y ago
    Shame that's not where the money is
- creata 1y ago
  Maybe ROCm is a bit of a mess, but AMD makes amazing graphics cards. And they open sourced their GPU driver, which is a massive plus in my book. If Radeon got virtualization support, it would make it the perfect GPU for running video games (or other GPU applications) in virtual machines - you wouldn't need to mess with PCIe passthrough anymore.
latchkey 1y ago
Previously: https://news.ycombinator.com/item?id=43759350 https://news.ycombinator.com/item?id=43759350
latchkey 1y ago
Hi, this is actually pretty important for my business. I've been waiting for the driver to be released for a year now. We got the binary a few months ago under NDA, but open sourcing it, is next level for us. What I wrote about this on twitter: https://x.com/HotAisle/status/1914549886185611627 https://x.com/HotAisle/status/1914549886185611627

196 points by davidlt 1y ago

This article is almost unreadable for me. The ads change in size and make the text jump. I'm adding it to NotebookLM now.

Mountain_Skies 1y ago
The article is extremely light on details anyway. The most important thing in it is the link to the repo at https://github.com/amd/MxGPU-Virtualization https://github.com/amd/MxGPU-Virtualization
DistractionRect 1y ago
Seems like using a sledge hammer to pound a nail. Why not just use an ad blocker like ublock origin?

proxysna 1y ago

That's pretty sick. Nice to see such things trickle down to consumer GPU's.

seanhunter 1y ago

It blows my mind how reliably AMD shoots itself in the foot. What we want isn’t that hard: 1) Support your graphics cards on linux using kernel drivers that you upstream. All of them. Not just a handful - all the ones you sell from say 18 months ago till today. 2) Make GPU acceleration actually work out of the box for pytorch and tensorflow. Not some special fork, patched version that you “maintain” on your website, the tip of the main branch for both of those libraries should just compile out of the box and give people gpu-accelerated ML. This is table stakes but it blows my mind that they keep making press releases and promises like this that things are on the roadmap without doing thing one and unfucking the basic dev experience so people can actually use their GPUs for real work. How it actually is: 1) Some cards work with rocm, some cards work with one of the other variations of BS libraries they have come up with over the years. Some cards work with amdgpu but many only work with proprietary kernel drivers which means if you don’t use precisely one of the distributions and kernel versions that they maintain you are sool. 2) Nothing whatsoever builds out of the box and when you get it to build almost nothing runs gpu accelerated. For me, pytorch requires a special downgrade, a python downgrade and a switch to a fork that AMD supposedly maintain although it doesn’t compile for me and when I managed to beat it into a shape where it compiled it wouldn’t run GPU accelerated even though games use the GPU just fine. I have a GPU that is supposedly current, so they are actively selling it, but can I use it? Can I bollocks. Ollama won’t talk to my GPU even though it supposedly works with ROCm. It only works with ROCm with some graphics cards. Tensorflow similar story when I last tried it although admittedly I didn’t try as hard as pytorch. Just make your shit work so that people can use it. It really shouldn’t be that hard. The dev experience with NVidia is a million times better.

faust201 1y ago
IIRC there was only one AMD employee that was working to integrate linux based things. Often, the response was - things are stuck in Intellectual property, or project managers etc. So even specs were not available.
logicchains 1y ago
SemiAnalysis had a good article on this recently, basically the reason AMD still sucks on the ML software side is that their compensation for devs is significantly worse than competitors like NVidia, Google and OpenAI, so most of the most competent devs go elsewhere.
bayindirh 1y ago
AMD has two driver teams at this point. One of Linux/Open Source, one for Catalyst/Closed source, and they are not allowed to interact. Because, there are tons of IP and trade secrets involved in driver development and optimization. Sometimes game related, sometimes for patching a rogue application which developers can't or don't fix, etc. etc. GPU drivers are ought to be easy, but in reality, they are not. The open source drivers are "vanilla" drivers without all these case-dependent patching and optimization. Actually, they really work well out of the box for normal desktop applications. I don't think there are any cards which do (or will) not work with the open source kernel drivers as long as you use a sufficiently recent version. ...and you mention ROCm. I'm not sure how ROCm's intellectual underpinnings are but, claiming lack of effort is a bit unfair to AMD. Yes, software was never their strong suit, but they're way better when compared to 20 years earlier. They have a proper open source driver which works, and a whole fleet of open source ROCm packages, which is very rigorously CI/CD tested by their maintainers now. Do not forget that some of the world's most powerful supercomputers run on Instinct cards, and AMD is getting tons of experience from these big players. If you think the underpinnings of GPGPU libraries are easy, I can only say that the reality is very different. The simple things people do with PyTorch and other very high level libraries pull enormous tricks under the hood, and you're really pushing the boundaries of the hardware performance and capability-wise. NVIDIA is not selling a tray full of switches and GPUs and require OEMs to integrate it as-is for no reason. On the other hand, the same NVIDIA acts very slowly to enable an open source ecosystem. So, yes, AMD is not in an ideal position right now, but calling them incompetent doesn't help either. P.S.: The company which fought for a completely open source HDMI 2.1 capable display driver is AMD, not NVIDIA.

throwaway48476 1y ago

If AMD does deliver on client dGPU virtualization it would be amazing.

transpute 1y ago
Some old AMD workstation GPUs supported SR-IOV, that repo was just archived. https://open-iov.org/index.php/GPU_Support#AMD https://open-iov.org/index.php/GPU_Support#AMD https://github.com/GPUOpen-LibrariesAndSDKs/MxGPU-Virtualization https://github.com/GPUOpen-LibrariesAndSDKs/MxGPU-Virtualiza... As "AI" use cases mature, NPU/AIE-ML virtualization will also be needed.
- mtillman 1y ago
  The pro vii (a wonderfully spec’d card for my purposes) had a bunch of awesome features-even remote access-but then they pulled the marketing pages from their website and stopped shipping the link bridge (non mpx) to water down the value of the card. I have no idea how AMD works.

AbuAssar 1y ago

related: (AMD 2.0 – New Sense of Urgency) https://news.ycombinator.com/item?id=43780972 https://news.ycombinator.com/item?id=43780972

bryanlarsen 1y ago
Wow, that's a very substantive article.
okucu 1y ago
[flagged]

deleted 1y ago

[deleted]

curtisszmania 1y ago

[dead]

404human 1y ago

Anyone else find it funny that AMD releases new GPU features while people still can't get basic ML stuff working? It's like building a fancy garage before fixing the broken car.

Nullabillity 1y ago
Graphics work just fine. Y'know, the G in the name.
- Redoubts 1y ago
  Shame that's not where the money is
creata 1y ago
Maybe ROCm is a bit of a mess, but AMD makes amazing graphics cards. And they open sourced their GPU driver, which is a massive plus in my book. If Radeon got virtualization support, it would make it the perfect GPU for running video games (or other GPU applications) in virtual machines - you wouldn't need to mess with PCIe passthrough anymore.

latchkey 1y ago

Previously: https://news.ycombinator.com/item?id=43759350 https://news.ycombinator.com/item?id=43759350

latchkey 1y ago

Hi, this is actually pretty important for my business. I've been waiting for the driver to be released for a year now. We got the binary a few months ago under NDA, but open sourcing it, is next level for us. What I wrote about this on twitter: https://x.com/HotAisle/status/1914549886185611627 https://x.com/HotAisle/status/1914549886185611627