10 ms·

ROCm is AMD's priority, executive says

293 points by mindcrime 3y ago

halJordan 3y ago
The first step is admitting there's a problem. So... that's nice.
- ethbr1 3y ago
  Exactly. People might trust AMD if they continue to invest in this for the next 10 years. It's clear it wasn't a corporate priority. Convince people it is via sustained action and investment, and eventually they might change their minds.
HideousKojima 3y ago
Only 16 years after Nvidia released CUDA
- grubbs 3y ago
  I remember chatting with some Nvidia rep at CES 2008. He showed me how cuda could be used to accelerate video upscale and encoding. I was 19 at the time and just a hobbyist. I thought that was the coolest thing in the world. (And yes I "snuck" in to CES using a fake business card to get my badge)
  - gdiamos 3y ago
    Back in the day, using CUDA was really hard. It got better as more people built on it and it got battle tested.
    - hyperbovine 3y ago
      It's still not exactly easy, and the API has not changed much since the aughts except than to become richer and more complicated. But almost nobody writes raw CUDA anymore. It's abstracted away beneath many layers of libraries, e.g. Flax -> Jax -> lax -> XLA -> CUDA.
- Dah00n 3y ago
  [flagged]
  - HideousKojima 3y ago
    The hilarious thing is I'm actually an AMD fanboy, I've made a point to only get their GPUs (and CPUs) for the last decade or so. But I'm still annoyed and frustrated that it's taken them so long to get their act together on this.
  - jacquesm 3y ago
    What a useless comment. It is you that drives the fire, I would be more than happy with a bit more competition. The sad reality is that right now if you want to focus on your job and not on the intermediary layers that NV is pretty much the only game in town. The 'Team Green' bs came out of the gaming world where people with zero qualifications were facing off with other people with zero qualifications about whose HW was 'the best' when 'the best' meant: I can play games. But this is entirely different, it is about long and deep support of a complex hardware/software combo where whole empires are built upon that support. Those are not decisions made lightly and unfortunately AMD has done very poorly so far. This announcement is great but the proof of the pudding will be in the eating, so let's see how many engineers they dedicate to delivering top notch software.
gravypod 3y ago
If this turns around it will be amazing but ROCm isnt the only issue. The entire driver stack is important. If they came out with virtualization support for their gpus (even if everyone paid a 10% perf hit) they'd take over the cheap hosted gpu space which is a huge market.
- mindcrime 3y ago
  Getting proper (and official) ROCm support across their consumer GPU line will be big as well. Hobbyists aren't buying MI300's and their ilk. And surely AMD is better off if a would be hobbyist (or low budget academic/industrial researcher) chooses a Radeon card over something from NVIDIA! I'm about to buy a high-end Radeon card myself, gambling that AMD is serious about this and will get it right, and that it won't be a wasted purchase. So yeah, if I seem like an AMD fan-boy (I am, somewhat) at least I'm putting my money where my mouth is. :-) AMD’s software stacks for each class of product are separate: ROCm (short for Radeon Open Compute platform) targets its Instinct data center GPU lines (and, soon, its Radeon consumer GPUs), They've been saying this for a while, and I'm encouraged by reports that people "out there" in the wild have actually gotten this to work with some cards, even in advance of the official support shipping. So here's hoping they are really serious about this point and make this real.
  - auggierose 3y ago
    Yeah, don't. Buy an Nvidia and get shit done.
    - iforgotpassword 3y ago
      A bit harsh but I agree in that I only believe it when I see it. Have been burned by empty promises by AMD before.
    - capableweb 3y ago
      For some people, it's not just about getting results or "get shit done" but about the journey and learning on the way there. Also, AMDs approach to openness tends to be a bit better than NVIDIA, so there's that too. And since we're on HackerNews after all, an AMD GPU for the hacker betting on the future seems pretty fitting.
alex21212 3y ago
Rocm and amd drives me nuts. The lack of support for consumer cards and the hassle of getting basic things in pytorch to just work was too much. I was burned by support that never came for my 6800xt. Recently went back to NVIDIA with a 4070 for pytorch. I hope amd gets their act together with rocm but I'm not going to buy an AMD GPU until they do fix it rather than just vaguely promise to add support some day ...
- zucker42 3y ago
  Exactly. I recently started a NN side project. The process for setting up PyTorch was to run `pacman -S cuda` and `pip install torch`. I was using a GTX 1060. If it was a project with a bigger budget, I could have rented servers from AWS with all the software preinstalled in no time. I don't even know if it would have been possible for me to do it with AMD, even if I owned an AMD graphics card. People like me are small potatoes to AMD, but surely it's hard to make significant inroads when it's impossible for anyone to learn or do small projects on ROCM, and big projects can't rely on ROCM just working.
  - jacquesm 3y ago
    People like you are small potatoes until you have some measure of success and then suddenly you're burning up GPU hours by the truckload and whatever you're used to you will continue using.
    - frognumber 3y ago
      I'm building a major open source stack on top of NVidia because of how bad my experience with AMD was. - I bought a ROCm-supported card. Said so on the box. Paid out-of-pocket. An NVidia vendor had sent me a free card, for comparison. - It never worked well, and a bit more than month after I bought it, AMD dropped support. Money down the drain. - AMD itself was a black hole for any sort of contact or support. I'm pretty sure this was a legal violation, as the card wasn't fit for the advertised purpose, but no one took responsibility, and small claims isn't worth it. I'm very supportive of open, but there's enough wrong at AMD that I'm not hitching myself to that wagon, probably ever.
Tsiklon 3y ago
I think AMD need to do something BIG in the enterprise space. It seems Nvidia have the Lion's Share of the Market, but Intel have been making good strides there with their DC GPUs. The software stack is the key here. If the drivers aren't there it doesn't matter what paper capabilities your product has if you can't use it. AMD have on paper done well with performance in recent generations of consumer cards but their drivers universally seem to be the let down to making the most of their architecture.
- therealmarv 3y ago
  they have! On one of the last keynotes in Summer they announced direct competitor to chips from Nvidia AI chips for enterprises: MI300X https://www.anandtech.com/show/18915/amd-expands-mi300-family-with-mi300x-gpu-only-192gb-memory https://www.anandtech.com/show/18915/amd-expands-mi300-famil... Software stack is crucial of course but if you buy this kind of chips (means you have a lot of money) you probably can also optimise your stack for it for some extra bucks to not rely on Nvidia's supply.
pixelpoet 3y ago
Oh man, this is exactly what I want to see on HN frontpage! I commented on another article about an AMD chip that had no OpenCL support that it made it dead in the water for me, and was downvoted; surely everyone understands how important CUDA is, and everyone should understand how important open standards are (e.g. FreeSync vs Nvidia's GSync), so I can't understand why more people don't share my zeal for OpenCL. I've shipped two commercial products based on it which still works perfectly today on all 3 desktop platforms from all GPU vendors... what's not to love?
- jjoonathan 3y ago
  For a long time, AMD promoted OpenCL as viable without it actually being viable. This leaves scars and resentment. Mine come from about 10 years ago. They run deep. I'm glad to hear your experience was better, but I'm fresh out of trust. This time, I need to see major projects in my application areas working on AMD before I buy, because AMD has taught me that "trust us" and "just around the corner" can mean "10 years later and it still hasn't happened." I'm pretty sure that this time is different, but the green tax is dirt cheap compared to learning this lesson the hard way, so I'm letting others jump first this time.
- Vvector 3y ago
  But is this just more BS from AMD? https://www.bit-tech.net/reviews/tech/cpus/amd-betting-everything-on-opencl/1/ https://www.bit-tech.net/reviews/tech/cpus/amd-betting-every... AMD Betting Everything on OpenCL (2011)
  - jjoonathan 3y ago
    I'm pretty sure the NVDA pump finally convinced the AMD board / C-Suite to prioritize this, but it takes time to steer a big ship. I'm hopeful, but there are still bad incentives to jump the gun on announcements so I'll let others take the plunge first.
- Conscat 3y ago
  OpenCL isn't very useful now that we have Vulkan. Its biggest advantage is that there exist C++ compilers for its kernels. But AMD's OpenCL runtime inserts excessive memory barriers not required by the spec (they won't fix this due to Hyrum's Law) and Vulkan gives you more control over the memory allocation and synchronization anyways. If we had better Vulkan shader compilers, OpenCL would serve basically no purpose, at least for AMD hardware.
gdiamos 3y ago
Relevant, we deployed Lamini on hundreds of MI200 GPUs. Lisa tweet: https://x.com/LisaSu/status/1706707561809105331?s=20 https://x.com/LisaSu/status/1706707561809105331?s=20 Lamini tweet: https://x.com/realSharonZhou/status/1706701693684154766?s=20 https://x.com/realSharonZhou/status/1706701693684154766?s=20 Blog: https://www.lamini.ai/blog/lamini-amd-paving-the-road-to-gpu-rich-enterprise-llms https://www.lamini.ai/blog/lamini-amd-paving-the-road-to-gpu... Register: https://www.theregister.com/2023/09/26/amd_instinct_ai_lamini/ https://www.theregister.com/2023/09/26/amd_instinct_ai_lamin... CRN: https://www.crn.com/news/components-peripherals/llm-startup-embraces-amd-gpus-says-rocm-has-parity-with-nvidia-s-cuda-platform https://www.crn.com/news/components-peripherals/llm-startup-... The hard part about using any AI Chips other than NVIDIA has been software. ROCm is finally at the point where it can train and deploy LLMs like Llama 2 in production. If you want to try this out, one big issue is that software support is hugely different on Instinct vs Radeon. I think AMD will fix this eventually, but today you need to use Instinct. We will post more information explaining how this works in the next few weeks. The middle section of the blog post above includes some details including GEMM/memcpy performance, and some of the software layers that we needed to write to run on AMD.
- mardifoufs 3y ago
  What's the cost benefit vs. Nvidia? Is it cheaper?
  - gdiamos 3y ago
    Available in orders of up to 10,000 GPUs today - no shortage More than 10x cheaper than allocating machines on a tier 1 cloud - AWS, Azure, GCP, Oracle, etc More memory - 128GB HBM per GPU - means bigger models fit for training/inference without the nightmare of model parallelism over MPI/infiniband/etc Longer term - finetuning optimizations
    - mardifoufs 3y ago
      Ah! The memory sounds interesting. How would that compare to similar Nvidia hardware w.r.t cost assuming the hardware was available? Does AMD provide something similar to nvlink, and even libraries like cudnn? Also, last I checked none of the public clouds offered any of the latest gens MI GPUs, so I wasn't aware that it had good availability! Azure had a preview but I'll look more into it now. Thank you for your answer btw!
01100011 3y ago
As far as I understand it, AMD basically has to do this because games are going to increasingly rely on LLMs & generative AI operating simultaneously with the graphics pipeline.
- imbusy111 3y ago
  It has nothing to do with games. The market outside of games for compute is much bigger at the moment with the AI hype, and AMD is positioned to take a good slice of it, if they get their software stack in order.
  - earthling8118 3y ago
    You've missed the point of their message. I think they're saying: Sure, the market is bigger. They could choose to continue to focus on gaming despite that. Except it doesn't seem like even that is an option.
capableweb 3y ago
"senior VP of the AI group at AMD", said at a "AI Hardware Summit" that "My area is AMDs No. 1 Priority". Tell me when the rest of the company aligns with you and has started to show any results in providing a good experience for people to do machine learning with AMD. As it stands right now, there is so much tooling missing, and the tooling that's there is severely lacking. But, I have a faith. They've reinvented themselves with CPUs, multiple times, so why not with GPUs, again?
- mindcrime 3y ago
  Tell me when the rest of the company aligns with you More or less the same message has been promulgated[1][2] by no less than Lisa Su[3], FWIW. [1]: https://www.phoronix.com/news/Lisa-Su-ROCm-Commitment https://www.phoronix.com/news/Lisa-Su-ROCm-Commitment [2]: https://www.forbes.com/sites/iainmartin/2023/05/31/lisa-su-saved-amd-now-she-wants-nvidias-ai-crown/ https://www.forbes.com/sites/iainmartin/2023/05/31/lisa-su-s... [3]: https://en.wikipedia.org/wiki/Lisa_Su https://en.wikipedia.org/wiki/Lisa_Su
mgaunard 3y ago
AMD has a history of providing sub-par software, and their strategy of (partially) opening up their specifications and have other people write it for free didn't work either. Nvidia has huge software teams, and so does Intel.
- mindcrime 3y ago
  I don't know if they'll ultimately succeed or not, but they at least seem to be putting genuine effort into this. ROCm releases are coming out at a relatively nice clip[1], including a new release just a week or two ago[2]. [1]: https://github.com/RadeonOpenCompute/ROCm/releases https://github.com/RadeonOpenCompute/ROCm/releases [2]: https://www.phoronix.com/news/AMD-ROCm-5.7-Released https://www.phoronix.com/news/AMD-ROCm-5.7-Released
  - Vvector 3y ago
    Yeah, AMD is doing more with ROCm. But are they catching up to Nvidia, or just not falling behind as fast as before? Only time will tell
    - mindcrime 3y ago
      It's a fair question. And I agree, all we can do is wait and see how things play out. I am definitely rooting for AMD here though, for multiple reasons.
- dagw 3y ago
  Not only sub-par software, but sub-par software that they drop support for after a couple of years. People can work around the problems with sub-par software if they believe that it will benefit them long term. They will absolutely not put in the effort if they fear it will be completely useless in 2 years time.
raphlinus 3y ago
ROCm makes me sad, as it reminds me of how much better GPUs could be than they are today. I've lately been exploring the idea of a "Good Parallel Computer," which combines most of the agility of a CPU with the efficient parallel throughput of a GPU. The central concept is that the decision to launch a workgroup is made by a programmable controller, rather than just being a cube of (x, y, z) or downstream of triangles. A particular workload it would likely excel at is sparse matrix multiplication, including multiple quantization levels like SpQR[1]. I'm hopeful that it could be an advance in execution model, but also a simplification, as I believe a lot of the complexity of the current GPU model is because of lots of workarounds for the weak execution model. I'm not optimistic about this being built any time soon, as it requires rethinking the software stack. But it's fun to think about. I might blog about it at some point, but I'm also interested in connecting with people who have been thinking along similar lines. [1]: https://arxiv.org/abs/2306.03078 https://arxiv.org/abs/2306.03078
- johncolanduoni 3y ago
  How does this differ from CUDA’s dynamic parallelism, which lets you launch kernels from within a kernel?
  - raphlinus 3y ago
    There are a lot of similarities, but the granularity is finer. The idea is that you make a decision to launch one workgroup (typically 1024 threads) when the input is available, which would typically be driven by queues, and potentially with joins as well, which is something the new work graph stuff can't quite do. Otherwise the idea of stages running in parallel, connected by queues, is similar. But I did an analysis of work graphs and came to the conclusion that it wouldn't help with the Vello (2d vector graphics) workload at all.
    - shaklee3 3y ago
      You can do device graph launch in cuda https://developer.nvidia.com/blog/enabling-dynamic-control-flow-in-cuda-graphs-with-device-graph-launch/ https://developer.nvidia.com/blog/enabling-dynamic-control-f...
Havoc 3y ago
I've concluded they're just allergic to money. Even after it became very clear that this is going to be big they're still slow off the block as if they're not even trying. e.g. Why not make a list of the top 500 people in AI field and send them cards no strings attached plus as good of low level documentation as you can muster. Insignificant cost to AMD but could move the mindshare needle if even 20 of the 500 experiment and make some noise about it in their circles. The Icewhale guys did exactly that best as I can tell. 350k USD hardware kickstarter so really lean. Yet all the youtubers even vaguely in their niche seem to have one of their boards. It's a good board don't get me wrong, but there is no way that was organic. Some sharp marketeer made sure the right people have the gear to influence mindshare. https://www.youtube.com/results?search_query=zimaboard https://www.youtube.com/results?search_query=zimaboard
- jjoonathan 3y ago
  My suspicion is that the GPGPU hardware in shipped cards has known problems / severe limitations due to neglect of that side of the architecture for the last ~10 years. Shipping a bunch of cards only to burn the next generation of AMD compute fans as badly as they burned the last generation of AMD compute fans would not be wise. It's painful to wait, but it may well be for the best.
  - gdiamos 3y ago
    Instinct has much better SW support today than Radeon, so you would need to send MI210s/etc . I think it's at the point where if you are comfortable with GEMM kernels, setting up SLURM, etc it is usable. But if you want to stay at the huggingface layer or higher, you will run into issues. Many AI researchers are higher level than that these days, but some are still of us willing to go lower level.
  - spacecadet 3y ago
    Yeah, this. I tried to do some computing with AMD server grade cards 2 years ago and found all of the API so out of fate and the documentation equally out of date... Went CUDA and didnt look back. Sad, cause Im an AMD fanboy of old.
  - 3y ago
martinald 3y ago
It's absolutely mindboggling to me that AMD is still struggling so badly on this. There is an absolutely enormous market for AMD GPUs for this, but they seem to be completely stuck on how to build a developer ecosystem. Why aren't AMD throwing as many developers as possible submitting PRs for the open source LLM effort adding ROCm support, for example? It would give AMD real world insights to the problems with their drivers and SDKs as well, which are incredibly numerous. People would be willing to overlook a huge amount of jank for cheap(er) cards with large VRAM configurations. I don't think they when need to be particularly fast, just have the VRAM needed, which I'm sure AMD could put specialist cards together for.
- hedgehog 3y ago
  Historically they believed that "the community" would address broader ML software support. I think the idea was they could assign dedicated engineers for bigger customers and together that was a sort of Pareto-goodish solution given their constraints as a company. Even in retrospect I'm not sure if that was a good call or not.
  - Almondsetat 3y ago
    I mean, they would be right if all their cards, both consumer and enterprises, supported the same programming interface. You cannot trust the community to do the work for you but then only make the software available for $Xk dollar cards
    - hedgehog 3y ago
      That's not necessary or sufficient. Going back to 2017 or so when I was working in the area their OpenCL support was good enough, the missing parts were an equivalent to cuDNN and upstreamed support in TensorFlow etc. That work does not subdivide in a way amenable to being a community effort and it's way too big for a hobby project. Today the technical landscape is different but from what I can tell the basic problems are the same.
      - deleted 3y ago
no_wizard 3y ago
The inevitable fight here is between ROCm which may have, 100s of AMD engineers working on it and related verticals, at best, without significant changes at the company, plus whatever contributions they can muster from the community. I think at least headcount check, CUDA had thousands of engineers working on it and related verticals. I know there's a philosophy that states, eventually, open source eats everything, however, this one seems like there is so much catch up that AMD will need to spend big and fast to get off the ground competitively.
- deleted 3y ago
  [deleted]
clhodapp 3y ago
If they were serious, they would start something like drm/mesa but for compute and it would just work out of the box with a stock Linux kernel.
- JonChesterfield 3y ago
  The amdkfd driver is in stock Linux kernels. ROCm is mostly userspace, if you don't install the kernel module that comes with it, code still runs.
- imtringued 3y ago
  Rusticl is the latest attempt at developing an OpenCL implementation for mesa and that is exactly the goal.
dauertewigkeit 3y ago
With all this hype about CUDA, I have recently started looking into programming CUDA as a job as I love that kind of challenge, but to my dismay I found that these tasks are very niche. So it is not even that people are routinely writing new CUDA code. It's just that the current corpus is too big and comprehensive for alternatives to compete with.
- jacquesm 3y ago
  That and a massive amount of experience already out there on how to optimize for that particular architecture. NVidia has done well for itself on the back of four sequential very good bets coupled with dedication unmatched by any other vendor, both on the hardware and on the software side. It also was one of the few times that I didn't care if I ran the vendor supplied closed source stuff because it seemed to work just fine and I never had the feeling they would suddenly drop support for my platform.
- coder543 3y ago
  Specialized skills can have a fairly small job market sometimes. I think a lot of CUDA code ends up being foundational as part of popular libraries, supporting tons of applications that never need to write a single line of CUDA themselves.
ckastner 3y ago
The Debian ROCm Team [1] has made quite a bit of progress in getting the ROCm stack into the official Archive. Most components are already packaged, the next big target is adding support to the PyTorch package. Many of the packages are older versions; this is because getting broad coverage was prioritized. The other next big target that is currently being worked on is getting full ROCm 5.7 support. I fully expect Debian 13 (trixie) to come with full ROCm support out-of-the-box, and as a consequence, also derivatives to have support (Ubuntu above all). In fact, there will almost certainly be backports of ROCm 5.7 to Debian 12 (bookworm) within the next few months, so one will be able to just $ sudo apt-get install pytorch-rocm One current obstacle is infrastructure: the Debian build and CI infrastructures (both hardware and software) were not designed with GPUs in mind. This is also being worked on. Edit: forgot to say that the CI infra that the Team is setting up here tests all of these packages on consumer cards, too. So while there may not be official support for most of these, upstream tests passing on the cards within the infra should be a good indication for practical support. [1] https://salsa.debian.org/rocm-team/ https://salsa.debian.org/rocm-team/
- avcxz 3y ago
  I'd also like to point out that ROCm has been packaged for Arch Linux since the beginning of 2023, with efforts starting since March 2020 [1]. Currently on Arch Linux you can run the following successfully: $ sudo pacman -S python-pytorch-rocm Arch Linux even has ROCm support with blender. [1] https://github.com/rocm-arch https://github.com/rocm-arch
  - alright2565 3y ago
    Hope you don't mind, but I have a rant I need to get out. I decided to give this another try now that you've mentioned it. Let's get things started the way the arch wiki suggests: $ sudo pacman -S rocm-hip-sdk $ /opt/rocm/bin/clinfo ERROR: clGetPlatformIDs(-1001) $ sudo /opt/rocm/bin/clinfo ... Board name: AMD Radeon RX 6600 XT ... Ok, I wonder what's wrong. maybe it's this? https://stackoverflow.com/questions/4959621/error-1001-in-clgetplatformids-call https://stackoverflow.com/questions/4959621/error-1001-in-cl... Nope. Anything about this on the arch wiki? Nope This bug report[2] from 2021? Maybe I need to update my groups. [2]: https://github.com/RadeonOpenCompute/ROCm/issues/1411 https://github.com/RadeonOpenCompute/ROCm/issues/1411 $ ls -la /dev/kfd crw-rw-rw- 1 root render 237, 0 Sep 26 20:33 /dev/kfd $ sudo usermod -aG render $(whoami) $ # relogin $ /opt/rocm/bin/clinfo ERROR: clGetPlatformIDs(-1001) Ok, I'm a pretty advanced linux user, I'll just jump right in: $ strace /opt/rocm/bin/clinfo ... openat(AT_FDCWD, "rusticl.icd", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = -1 ENOENT (No such file or directory) Apparently I have some leftover environment variables (OCL_ICD_VENDORS) from last time I spent half a day trying to get this to work. I can fix that. After all, it'd be entirely unreasonable to expect rocm to give me a better error, like "Could not open opencl icd `rusticl.icd`". Success: $ /opt/rocm/bin/clinfo Number of platforms: 1 ... Board name: AMD Radeon RX 6600 XT Well, let's run some apps! $ darktable -d opencl ... [dt_opencl_device_init] DEVICE: 0: 'gfx1032' PLATFORM NAME & VENDOR: AMD Accelerated Parallel Processing, Advanced Micro Devices, Inc. ... PHI node has multiple entries for the same basic block with different incoming values! %967 = phi float [ %largephi.extractslice0, %sw.default ], [ %largephi.extractslice055, %sw.bb667 ], [ %largephi.extractslice059, %sw.bb663 ], [ %largephi.extractslice063, %sw.bb659 ], [ %largephi.extractslice067, %sw.bb655 ], [ %largephi.extractslice071, %sw.bb646 ], [ %largephi.extractslice075, %_Z4fmodff.exit16 ], [ %largephi.extractslice079, %_Z4fmodff.exit13 ], [ %largephi.extractslice083, %_Z4fmodff.exit ], [ %largephi.extractslice087, %sw.bb562 ], [ %largephi.extractslice091, %sw.bb555 ], [ %largephi.extractslice095, %sw.bb533 ], [ %largephi.extractslice099, %if.then502 ], [ %largephi.extractslice0103, %if.else517 ], [ %largephi.extractslice0107, %if.then456 ], [ %largephi.extractslice0111, %if.else471 ], [ %largephi.extractslice0115, %if.then393 ], [ %largephi.extractslice0119, %if.else408 ], [ %largephi.extractslice0123, %if.then338 ], [ %largephi.extractslice0127, %if.else353 ], [ %largephi.extractslice0131, %if.then283 ], [ %largephi.extractslice0135, %if.else298 ], [ %largephi.extractslice0139, %if.then224 ], [ %largephi.extractslice0143, %if.else241 ], [ %largephi.extractslice0147, %sw.bb193 ], [ %largephi.extractslice0151, %sw.bb180 ], [ %largephi.extractslice0155, %sw.bb168 ], [ %largephi.extractslice0159, %sw.bb158 ], [ %largephi.extractslice0163, %sw.bb147 ], [ %largephi.extractslice0167, %if.then116 ], [ %largephi.extractslice0171, %if.else131 ], [ %largephi.extractslice0175, %sw.bb71 ], [ %largephi.extractslice0179, %sw.bb ], [ %largephi.extractslice0183, %if.end ], [ %largephi.extractslice0187, %if.end ], [ %largephi.extractslice0191, %if.end ], [ %largephi.extractslice0195, %if.end ], [ %largephi.extractslice0199, %if.end ] label %if.end %largephi.extractslice0183 = extractelement <4 x float> %div, i64 0 %largephi.extractslice0191 = extractelement <4 x float> %div, i64 0 in function blendop_Lab LLVM ERROR: Broken function found, compilation aborted! [1] 27586 IOT instruction (core dumped) darktable -d opencl uh that's great. Maybe blender? It worked! Not too bad for 2 minutes render: https://i.imgur.com/FD1SsQG.png https://i.imgur.com/FD1SsQG.png What about pytorch? It prompted this whole thing anyway: $ sudo pacman -S python-pytorch-rocm python-torchvision $ python neural_style/neural_style.py eval --content-image ../../2min.png --model ./saved_models/mosaic.pth --output-image out.png --cuda 1 [1] 32471 segmentation fault (core dumped) python neural_style/neural_style.py eval --content-image ../../2min.png $ sudo dmesg --follow [ 2467.536713] python[33309]: segfault at 68 ip 00007f12c5504d5d sp 00007ffc8f539c20 error 4 in libamdhip64.so.5.6.31062[7f12c541e000+357000] likely on CPU 14 (core 7, socket 0) [ 2467.536727] Code: ec 78 48 89 bd 78 ff ff ff 64 48 8b 04 25 28 00 00 00 48 89 45 c8 31 c0 85 f6 0f 88 09 03 00 00 48 8b 85 78 ff ff ff 48 63 de <48> 8b 50 68 48 8b 40 70 48 89 85 70 ff ff ff 48 29 d0 48 c1 f8 03 uh oh. Maybe I can crack some passwords? $ hashcat -m 0 -a 0 -o cracked.txt target_hashes.txt /usr/share/dict/american-english ... hiprtcCompileProgram(): HIPRTC_ERROR_COMPILATION error: unknown argument: '-flegacy-pass-manager' 1 error generated when compiling for gfx1032. * Device #1: Kernel /usr/share/hashcat/OpenCL/shared.cl build failed. Well, so much for that. Best I can get to work with rocm is 1/4 apps.
ryukoposting 3y ago
s/OpenCL/ROCm/g
vegabook 3y ago
With all due respect this is an insult to those of us who have loyally purchased AMD for numerous years, trying our very best to do compute with days, nay weeks, of attempts. Now 5 years too late we get told its suddenly their number one priority. Too late. Not only has all goodwill gone, but it's in deep negative territory. Even 50% lower performance stacks like Intel / Apple are much more appealing than AMD will ever be at this stage.
screcth 3y ago
What's stopping AMD from implementing CUDA? Just like Wine implemented Windows APIs
- cowmix 3y ago
  Why is this not upvoted more? Very good question.
- dotnet00 3y ago
  That is effectively what HIP is supposed to be (while sidesteppingsome copyright gray areas). They have a very close copy of the CUDA API and it can compile either for AMD GPUs or map onto the associated CUDA call for NVIDIA.
- AstralStorm 3y ago
  Nothing, HIP is essentially API compatible. That gets you nothing because CUDA nVidia optimized code will perform quite abysmally on a Radeon/Instinct. And furthermore nVidia has a bunch of proprietary libraries AMD has not cloned either. Normal people use Tensorflow, Keras or PyTorch anyway, not raw CUDA or even its libraries. The one place that is the stronghold of raw CUDA is molecular dynamics simulations because it's been written ages ago by some researcher who has never heard of Tensorflow etc. And probably uses cublas and/or cufft for which the AMD replacement is a joke and incompatible API. Situation there is slowly improving finally with Magma.
dingi 3y ago
[flagged]
justinclift 3y ago
Words versus actions. People don't really care about what the executive says. Especially when the same executive is also quoted with patently dishonest bullshit: > If you think about the product portfolio that AMD has, it’s arguably the broadest in the industry in terms of AI compute What AMD does is what people will pay attention to.
sorenjan 3y ago
Maybe I'll believe them when a consumer on Windows and Linux can download a binary from something like Meshlab or Automatic1111 and it just works on their gaming computer. If all they're interested in is selling CDNA to data centers I don't think they'll get enough mind share to be a realistic option. Also, is it really a good idea for various projects to add another proprietary platform? We should move away from Cuda and Rocm and towards open standards like Sycl. I don't want to have to care about who made my GPU, just as I don't have to care about who made my CPU.
- mmis1000 3y ago
  They did just start porting support ROCm to windows a few month ago(more specifically, ROCm 5.5.1 released a few months ago). And yea, ROCm for windows specifically supports rdna2 and rdna3 instead of cdna like ROCm for linux. So at least the title isn't a total lie. But ROCm for windows still have a few components missing. Will they finish the porting? Who knows? You may try to guess it.
ElectronBadger 3y ago
Really? Than you might explain why this list is so pitifully short: https://rocm.docs.amd.com/en/latest/release/gpu_os_support.html#linux-supported-gpus https://rocm.docs.amd.com/en/latest/release/gpu_os_support.h... and will get even shorter: https://rocm.docs.amd.com/en/latest/CHANGELOG.html#amd-instinct-mi50-end-of-support-notice https://rocm.docs.amd.com/en/latest/CHANGELOG.html#amd-insti... I'm so for AMD, but in terms of easily-accessible GPU computing ROCm is way behind CUDA.
eachro 3y ago
Not particularly relevant but the name "ROCm" is kind of terrible. Hard to pronounce, doesnt look good (the caps and then lower case is quite jarring). Minor details but I feel like these things do have a bit of downstream impact.

293 points by mindcrime 3y ago

halJordan 3y ago

The first step is admitting there's a problem. So... that's nice.

ethbr1 3y ago
Exactly. People might trust AMD if they continue to invest in this for the next 10 years. It's clear it wasn't a corporate priority. Convince people it is via sustained action and investment, and eventually they might change their minds.

HideousKojima 3y ago

Only 16 years after Nvidia released CUDA

grubbs 3y ago
I remember chatting with some Nvidia rep at CES 2008. He showed me how cuda could be used to accelerate video upscale and encoding. I was 19 at the time and just a hobbyist. I thought that was the coolest thing in the world. (And yes I "snuck" in to CES using a fake business card to get my badge)
- gdiamos 3y ago
  Back in the day, using CUDA was really hard. It got better as more people built on it and it got battle tested.
  - hyperbovine 3y ago
    It's still not exactly easy, and the API has not changed much since the aughts except than to become richer and more complicated. But almost nobody writes raw CUDA anymore. It's abstracted away beneath many layers of libraries, e.g. Flax -> Jax -> lax -> XLA -> CUDA.
Dah00n 3y ago
[flagged]
- HideousKojima 3y ago
  The hilarious thing is I'm actually an AMD fanboy, I've made a point to only get their GPUs (and CPUs) for the last decade or so. But I'm still annoyed and frustrated that it's taken them so long to get their act together on this.
- jacquesm 3y ago
  What a useless comment. It is you that drives the fire, I would be more than happy with a bit more competition. The sad reality is that right now if you want to focus on your job and not on the intermediary layers that NV is pretty much the only game in town. The 'Team Green' bs came out of the gaming world where people with zero qualifications were facing off with other people with zero qualifications about whose HW was 'the best' when 'the best' meant: I can play games. But this is entirely different, it is about long and deep support of a complex hardware/software combo where whole empires are built upon that support. Those are not decisions made lightly and unfortunately AMD has done very poorly so far. This announcement is great but the proof of the pudding will be in the eating, so let's see how many engineers they dedicate to delivering top notch software.

gravypod 3y ago

If this turns around it will be amazing but ROCm isnt the only issue. The entire driver stack is important. If they came out with virtualization support for their gpus (even if everyone paid a 10% perf hit) they'd take over the cheap hosted gpu space which is a huge market.

mindcrime 3y ago
Getting proper (and official) ROCm support across their consumer GPU line will be big as well. Hobbyists aren't buying MI300's and their ilk. And surely AMD is better off if a would be hobbyist (or low budget academic/industrial researcher) chooses a Radeon card over something from NVIDIA! I'm about to buy a high-end Radeon card myself, gambling that AMD is serious about this and will get it right, and that it won't be a wasted purchase. So yeah, if I seem like an AMD fan-boy (I am, somewhat) at least I'm putting my money where my mouth is. :-) AMD’s software stacks for each class of product are separate: ROCm (short for Radeon Open Compute platform) targets its Instinct data center GPU lines (and, soon, its Radeon consumer GPUs), They've been saying this for a while, and I'm encouraged by reports that people "out there" in the wild have actually gotten this to work with some cards, even in advance of the official support shipping. So here's hoping they are really serious about this point and make this real.
- auggierose 3y ago
  Yeah, don't. Buy an Nvidia and get shit done.
  - iforgotpassword 3y ago
    A bit harsh but I agree in that I only believe it when I see it. Have been burned by empty promises by AMD before.
  - capableweb 3y ago
    For some people, it's not just about getting results or "get shit done" but about the journey and learning on the way there. Also, AMDs approach to openness tends to be a bit better than NVIDIA, so there's that too. And since we're on HackerNews after all, an AMD GPU for the hacker betting on the future seems pretty fitting.

alex21212 3y ago

Rocm and amd drives me nuts. The lack of support for consumer cards and the hassle of getting basic things in pytorch to just work was too much. I was burned by support that never came for my 6800xt. Recently went back to NVIDIA with a 4070 for pytorch. I hope amd gets their act together with rocm but I'm not going to buy an AMD GPU until they do fix it rather than just vaguely promise to add support some day ...

zucker42 3y ago
Exactly. I recently started a NN side project. The process for setting up PyTorch was to run `pacman -S cuda` and `pip install torch`. I was using a GTX 1060. If it was a project with a bigger budget, I could have rented servers from AWS with all the software preinstalled in no time. I don't even know if it would have been possible for me to do it with AMD, even if I owned an AMD graphics card. People like me are small potatoes to AMD, but surely it's hard to make significant inroads when it's impossible for anyone to learn or do small projects on ROCM, and big projects can't rely on ROCM just working.
- jacquesm 3y ago
  People like you are small potatoes until you have some measure of success and then suddenly you're burning up GPU hours by the truckload and whatever you're used to you will continue using.
  - frognumber 3y ago
    I'm building a major open source stack on top of NVidia because of how bad my experience with AMD was. - I bought a ROCm-supported card. Said so on the box. Paid out-of-pocket. An NVidia vendor had sent me a free card, for comparison. - It never worked well, and a bit more than month after I bought it, AMD dropped support. Money down the drain. - AMD itself was a black hole for any sort of contact or support. I'm pretty sure this was a legal violation, as the card wasn't fit for the advertised purpose, but no one took responsibility, and small claims isn't worth it. I'm very supportive of open, but there's enough wrong at AMD that I'm not hitching myself to that wagon, probably ever.

Tsiklon 3y ago

I think AMD need to do something BIG in the enterprise space. It seems Nvidia have the Lion's Share of the Market, but Intel have been making good strides there with their DC GPUs. The software stack is the key here. If the drivers aren't there it doesn't matter what paper capabilities your product has if you can't use it. AMD have on paper done well with performance in recent generations of consumer cards but their drivers universally seem to be the let down to making the most of their architecture.

therealmarv 3y ago
they have! On one of the last keynotes in Summer they announced direct competitor to chips from Nvidia AI chips for enterprises: MI300X https://www.anandtech.com/show/18915/amd-expands-mi300-family-with-mi300x-gpu-only-192gb-memory https://www.anandtech.com/show/18915/amd-expands-mi300-famil... Software stack is crucial of course but if you buy this kind of chips (means you have a lot of money) you probably can also optimise your stack for it for some extra bucks to not rely on Nvidia's supply.

pixelpoet 3y ago

Oh man, this is exactly what I want to see on HN frontpage! I commented on another article about an AMD chip that had no OpenCL support that it made it dead in the water for me, and was downvoted; surely everyone understands how important CUDA is, and everyone should understand how important open standards are (e.g. FreeSync vs Nvidia's GSync), so I can't understand why more people don't share my zeal for OpenCL. I've shipped two commercial products based on it which still works perfectly today on all 3 desktop platforms from all GPU vendors... what's not to love?

jjoonathan 3y ago
For a long time, AMD promoted OpenCL as viable without it actually being viable. This leaves scars and resentment. Mine come from about 10 years ago. They run deep. I'm glad to hear your experience was better, but I'm fresh out of trust. This time, I need to see major projects in my application areas working on AMD before I buy, because AMD has taught me that "trust us" and "just around the corner" can mean "10 years later and it still hasn't happened." I'm pretty sure that this time is different, but the green tax is dirt cheap compared to learning this lesson the hard way, so I'm letting others jump first this time.
Vvector 3y ago
But is this just more BS from AMD? https://www.bit-tech.net/reviews/tech/cpus/amd-betting-everything-on-opencl/1/ https://www.bit-tech.net/reviews/tech/cpus/amd-betting-every... AMD Betting Everything on OpenCL (2011)
- jjoonathan 3y ago
  I'm pretty sure the NVDA pump finally convinced the AMD board / C-Suite to prioritize this, but it takes time to steer a big ship. I'm hopeful, but there are still bad incentives to jump the gun on announcements so I'll let others take the plunge first.
Conscat 3y ago
OpenCL isn't very useful now that we have Vulkan. Its biggest advantage is that there exist C++ compilers for its kernels. But AMD's OpenCL runtime inserts excessive memory barriers not required by the spec (they won't fix this due to Hyrum's Law) and Vulkan gives you more control over the memory allocation and synchronization anyways. If we had better Vulkan shader compilers, OpenCL would serve basically no purpose, at least for AMD hardware.

gdiamos 3y ago

Relevant, we deployed Lamini on hundreds of MI200 GPUs. Lisa tweet: https://x.com/LisaSu/status/1706707561809105331?s=20 https://x.com/LisaSu/status/1706707561809105331?s=20 Lamini tweet: https://x.com/realSharonZhou/status/1706701693684154766?s=20 https://x.com/realSharonZhou/status/1706701693684154766?s=20 Blog: https://www.lamini.ai/blog/lamini-amd-paving-the-road-to-gpu-rich-enterprise-llms https://www.lamini.ai/blog/lamini-amd-paving-the-road-to-gpu... Register: https://www.theregister.com/2023/09/26/amd_instinct_ai_lamini/ https://www.theregister.com/2023/09/26/amd_instinct_ai_lamin... CRN: https://www.crn.com/news/components-peripherals/llm-startup-embraces-amd-gpus-says-rocm-has-parity-with-nvidia-s-cuda-platform https://www.crn.com/news/components-peripherals/llm-startup-... The hard part about using any AI Chips other than NVIDIA has been software. ROCm is finally at the point where it can train and deploy LLMs like Llama 2 in production. If you want to try this out, one big issue is that software support is hugely different on Instinct vs Radeon. I think AMD will fix this eventually, but today you need to use Instinct. We will post more information explaining how this works in the next few weeks. The middle section of the blog post above includes some details including GEMM/memcpy performance, and some of the software layers that we needed to write to run on AMD.

mardifoufs 3y ago
What's the cost benefit vs. Nvidia? Is it cheaper?
- gdiamos 3y ago
  Available in orders of up to 10,000 GPUs today - no shortage More than 10x cheaper than allocating machines on a tier 1 cloud - AWS, Azure, GCP, Oracle, etc More memory - 128GB HBM per GPU - means bigger models fit for training/inference without the nightmare of model parallelism over MPI/infiniband/etc Longer term - finetuning optimizations
  - mardifoufs 3y ago
    Ah! The memory sounds interesting. How would that compare to similar Nvidia hardware w.r.t cost assuming the hardware was available? Does AMD provide something similar to nvlink, and even libraries like cudnn? Also, last I checked none of the public clouds offered any of the latest gens MI GPUs, so I wasn't aware that it had good availability! Azure had a preview but I'll look more into it now. Thank you for your answer btw!

01100011 3y ago

As far as I understand it, AMD basically has to do this because games are going to increasingly rely on LLMs & generative AI operating simultaneously with the graphics pipeline.

imbusy111 3y ago
It has nothing to do with games. The market outside of games for compute is much bigger at the moment with the AI hype, and AMD is positioned to take a good slice of it, if they get their software stack in order.
- earthling8118 3y ago
  You've missed the point of their message. I think they're saying: Sure, the market is bigger. They could choose to continue to focus on gaming despite that. Except it doesn't seem like even that is an option.

capableweb 3y ago

"senior VP of the AI group at AMD", said at a "AI Hardware Summit" that "My area is AMDs No. 1 Priority". Tell me when the rest of the company aligns with you and has started to show any results in providing a good experience for people to do machine learning with AMD. As it stands right now, there is so much tooling missing, and the tooling that's there is severely lacking. But, I have a faith. They've reinvented themselves with CPUs, multiple times, so why not with GPUs, again?

mindcrime 3y ago
Tell me when the rest of the company aligns with you More or less the same message has been promulgated[1][2] by no less than Lisa Su[3], FWIW. [1]: https://www.phoronix.com/news/Lisa-Su-ROCm-Commitment https://www.phoronix.com/news/Lisa-Su-ROCm-Commitment [2]: https://www.forbes.com/sites/iainmartin/2023/05/31/lisa-su-saved-amd-now-she-wants-nvidias-ai-crown/ https://www.forbes.com/sites/iainmartin/2023/05/31/lisa-su-s... [3]: https://en.wikipedia.org/wiki/Lisa_Su https://en.wikipedia.org/wiki/Lisa_Su

mgaunard 3y ago

AMD has a history of providing sub-par software, and their strategy of (partially) opening up their specifications and have other people write it for free didn't work either. Nvidia has huge software teams, and so does Intel.

mindcrime 3y ago
I don't know if they'll ultimately succeed or not, but they at least seem to be putting genuine effort into this. ROCm releases are coming out at a relatively nice clip[1], including a new release just a week or two ago[2]. [1]: https://github.com/RadeonOpenCompute/ROCm/releases https://github.com/RadeonOpenCompute/ROCm/releases [2]: https://www.phoronix.com/news/AMD-ROCm-5.7-Released https://www.phoronix.com/news/AMD-ROCm-5.7-Released
- Vvector 3y ago
  Yeah, AMD is doing more with ROCm. But are they catching up to Nvidia, or just not falling behind as fast as before? Only time will tell
  - mindcrime 3y ago
    It's a fair question. And I agree, all we can do is wait and see how things play out. I am definitely rooting for AMD here though, for multiple reasons.
dagw 3y ago
Not only sub-par software, but sub-par software that they drop support for after a couple of years. People can work around the problems with sub-par software if they believe that it will benefit them long term. They will absolutely not put in the effort if they fear it will be completely useless in 2 years time.

raphlinus 3y ago

ROCm makes me sad, as it reminds me of how much better GPUs could be than they are today. I've lately been exploring the idea of a "Good Parallel Computer," which combines most of the agility of a CPU with the efficient parallel throughput of a GPU. The central concept is that the decision to launch a workgroup is made by a programmable controller, rather than just being a cube of (x, y, z) or downstream of triangles. A particular workload it would likely excel at is sparse matrix multiplication, including multiple quantization levels like SpQR[1]. I'm hopeful that it could be an advance in execution model, but also a simplification, as I believe a lot of the complexity of the current GPU model is because of lots of workarounds for the weak execution model. I'm not optimistic about this being built any time soon, as it requires rethinking the software stack. But it's fun to think about. I might blog about it at some point, but I'm also interested in connecting with people who have been thinking along similar lines. [1]: https://arxiv.org/abs/2306.03078 https://arxiv.org/abs/2306.03078

johncolanduoni 3y ago
How does this differ from CUDA’s dynamic parallelism, which lets you launch kernels from within a kernel?
- raphlinus 3y ago
  There are a lot of similarities, but the granularity is finer. The idea is that you make a decision to launch one workgroup (typically 1024 threads) when the input is available, which would typically be driven by queues, and potentially with joins as well, which is something the new work graph stuff can't quite do. Otherwise the idea of stages running in parallel, connected by queues, is similar. But I did an analysis of work graphs and came to the conclusion that it wouldn't help with the Vello (2d vector graphics) workload at all.
  - shaklee3 3y ago
    You can do device graph launch in cuda https://developer.nvidia.com/blog/enabling-dynamic-control-flow-in-cuda-graphs-with-device-graph-launch/ https://developer.nvidia.com/blog/enabling-dynamic-control-f...

Havoc 3y ago

I've concluded they're just allergic to money. Even after it became very clear that this is going to be big they're still slow off the block as if they're not even trying. e.g. Why not make a list of the top 500 people in AI field and send them cards no strings attached plus as good of low level documentation as you can muster. Insignificant cost to AMD but could move the mindshare needle if even 20 of the 500 experiment and make some noise about it in their circles. The Icewhale guys did exactly that best as I can tell. 350k USD hardware kickstarter so really lean. Yet all the youtubers even vaguely in their niche seem to have one of their boards. It's a good board don't get me wrong, but there is no way that was organic. Some sharp marketeer made sure the right people have the gear to influence mindshare. https://www.youtube.com/results?search_query=zimaboard https://www.youtube.com/results?search_query=zimaboard

jjoonathan 3y ago
My suspicion is that the GPGPU hardware in shipped cards has known problems / severe limitations due to neglect of that side of the architecture for the last ~10 years. Shipping a bunch of cards only to burn the next generation of AMD compute fans as badly as they burned the last generation of AMD compute fans would not be wise. It's painful to wait, but it may well be for the best.
- gdiamos 3y ago
  Instinct has much better SW support today than Radeon, so you would need to send MI210s/etc . I think it's at the point where if you are comfortable with GEMM kernels, setting up SLURM, etc it is usable. But if you want to stay at the huggingface layer or higher, you will run into issues. Many AI researchers are higher level than that these days, but some are still of us willing to go lower level.
- spacecadet 3y ago
  Yeah, this. I tried to do some computing with AMD server grade cards 2 years ago and found all of the API so out of fate and the documentation equally out of date... Went CUDA and didnt look back. Sad, cause Im an AMD fanboy of old.
- 3y ago

martinald 3y ago

It's absolutely mindboggling to me that AMD is still struggling so badly on this. There is an absolutely enormous market for AMD GPUs for this, but they seem to be completely stuck on how to build a developer ecosystem. Why aren't AMD throwing as many developers as possible submitting PRs for the open source LLM effort adding ROCm support, for example? It would give AMD real world insights to the problems with their drivers and SDKs as well, which are incredibly numerous. People would be willing to overlook a huge amount of jank for cheap(er) cards with large VRAM configurations. I don't think they when need to be particularly fast, just have the VRAM needed, which I'm sure AMD could put specialist cards together for.

hedgehog 3y ago
Historically they believed that "the community" would address broader ML software support. I think the idea was they could assign dedicated engineers for bigger customers and together that was a sort of Pareto-goodish solution given their constraints as a company. Even in retrospect I'm not sure if that was a good call or not.
- Almondsetat 3y ago
  I mean, they would be right if all their cards, both consumer and enterprises, supported the same programming interface. You cannot trust the community to do the work for you but then only make the software available for $Xk dollar cards
  - hedgehog 3y ago
    That's not necessary or sufficient. Going back to 2017 or so when I was working in the area their OpenCL support was good enough, the missing parts were an equivalent to cuDNN and upstreamed support in TensorFlow etc. That work does not subdivide in a way amenable to being a community effort and it's way too big for a hobby project. Today the technical landscape is different but from what I can tell the basic problems are the same.
    - deleted 3y ago

no_wizard 3y ago

The inevitable fight here is between ROCm which may have, 100s of AMD engineers working on it and related verticals, at best, without significant changes at the company, plus whatever contributions they can muster from the community. I think at least headcount check, CUDA had thousands of engineers working on it and related verticals. I know there's a philosophy that states, eventually, open source eats everything, however, this one seems like there is so much catch up that AMD will need to spend big and fast to get off the ground competitively.

deleted 3y ago
[deleted]

clhodapp 3y ago

If they were serious, they would start something like drm/mesa but for compute and it would just work out of the box with a stock Linux kernel.

JonChesterfield 3y ago
The amdkfd driver is in stock Linux kernels. ROCm is mostly userspace, if you don't install the kernel module that comes with it, code still runs.
imtringued 3y ago
Rusticl is the latest attempt at developing an OpenCL implementation for mesa and that is exactly the goal.

dauertewigkeit 3y ago

With all this hype about CUDA, I have recently started looking into programming CUDA as a job as I love that kind of challenge, but to my dismay I found that these tasks are very niche. So it is not even that people are routinely writing new CUDA code. It's just that the current corpus is too big and comprehensive for alternatives to compete with.

jacquesm 3y ago
That and a massive amount of experience already out there on how to optimize for that particular architecture. NVidia has done well for itself on the back of four sequential very good bets coupled with dedication unmatched by any other vendor, both on the hardware and on the software side. It also was one of the few times that I didn't care if I ran the vendor supplied closed source stuff because it seemed to work just fine and I never had the feeling they would suddenly drop support for my platform.
coder543 3y ago
Specialized skills can have a fairly small job market sometimes. I think a lot of CUDA code ends up being foundational as part of popular libraries, supporting tons of applications that never need to write a single line of CUDA themselves.

ckastner 3y ago

The Debian ROCm Team [1] has made quite a bit of progress in getting the ROCm stack into the official Archive. Most components are already packaged, the next big target is adding support to the PyTorch package. Many of the packages are older versions; this is because getting broad coverage was prioritized. The other next big target that is currently being worked on is getting full ROCm 5.7 support. I fully expect Debian 13 (trixie) to come with full ROCm support out-of-the-box, and as a consequence, also derivatives to have support (Ubuntu above all). In fact, there will almost certainly be backports of ROCm 5.7 to Debian 12 (bookworm) within the next few months, so one will be able to just $ sudo apt-get install pytorch-rocm One current obstacle is infrastructure: the Debian build and CI infrastructures (both hardware and software) were not designed with GPUs in mind. This is also being worked on. Edit: forgot to say that the CI infra that the Team is setting up here tests all of these packages on consumer cards, too. So while there may not be official support for most of these, upstream tests passing on the cards within the infra should be a good indication for practical support. [1] https://salsa.debian.org/rocm-team/ https://salsa.debian.org/rocm-team/

avcxz 3y ago
I'd also like to point out that ROCm has been packaged for Arch Linux since the beginning of 2023, with efforts starting since March 2020 [1]. Currently on Arch Linux you can run the following successfully: $ sudo pacman -S python-pytorch-rocm Arch Linux even has ROCm support with blender. [1] https://github.com/rocm-arch https://github.com/rocm-arch
- alright2565 3y ago
  Hope you don't mind, but I have a rant I need to get out. I decided to give this another try now that you've mentioned it. Let's get things started the way the arch wiki suggests: $ sudo pacman -S rocm-hip-sdk $ /opt/rocm/bin/clinfo ERROR: clGetPlatformIDs(-1001) $ sudo /opt/rocm/bin/clinfo ... Board name: AMD Radeon RX 6600 XT ... Ok, I wonder what's wrong. maybe it's this? https://stackoverflow.com/questions/4959621/error-1001-in-clgetplatformids-call https://stackoverflow.com/questions/4959621/error-1001-in-cl... Nope. Anything about this on the arch wiki? Nope This bug report[2] from 2021? Maybe I need to update my groups. [2]: https://github.com/RadeonOpenCompute/ROCm/issues/1411 https://github.com/RadeonOpenCompute/ROCm/issues/1411 $ ls -la /dev/kfd crw-rw-rw- 1 root render 237, 0 Sep 26 20:33 /dev/kfd $ sudo usermod -aG render $(whoami) $ # relogin $ /opt/rocm/bin/clinfo ERROR: clGetPlatformIDs(-1001) Ok, I'm a pretty advanced linux user, I'll just jump right in: $ strace /opt/rocm/bin/clinfo ... openat(AT_FDCWD, "rusticl.icd", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = -1 ENOENT (No such file or directory) Apparently I have some leftover environment variables (OCL_ICD_VENDORS) from last time I spent half a day trying to get this to work. I can fix that. After all, it'd be entirely unreasonable to expect rocm to give me a better error, like "Could not open opencl icd `rusticl.icd`". Success: $ /opt/rocm/bin/clinfo Number of platforms: 1 ... Board name: AMD Radeon RX 6600 XT Well, let's run some apps! $ darktable -d opencl ... [dt_opencl_device_init] DEVICE: 0: 'gfx1032' PLATFORM NAME & VENDOR: AMD Accelerated Parallel Processing, Advanced Micro Devices, Inc. ... PHI node has multiple entries for the same basic block with different incoming values! %967 = phi float [ %largephi.extractslice0, %sw.default ], [ %largephi.extractslice055, %sw.bb667 ], [ %largephi.extractslice059, %sw.bb663 ], [ %largephi.extractslice063, %sw.bb659 ], [ %largephi.extractslice067, %sw.bb655 ], [ %largephi.extractslice071, %sw.bb646 ], [ %largephi.extractslice075, %_Z4fmodff.exit16 ], [ %largephi.extractslice079, %_Z4fmodff.exit13 ], [ %largephi.extractslice083, %_Z4fmodff.exit ], [ %largephi.extractslice087, %sw.bb562 ], [ %largephi.extractslice091, %sw.bb555 ], [ %largephi.extractslice095, %sw.bb533 ], [ %largephi.extractslice099, %if.then502 ], [ %largephi.extractslice0103, %if.else517 ], [ %largephi.extractslice0107, %if.then456 ], [ %largephi.extractslice0111, %if.else471 ], [ %largephi.extractslice0115, %if.then393 ], [ %largephi.extractslice0119, %if.else408 ], [ %largephi.extractslice0123, %if.then338 ], [ %largephi.extractslice0127, %if.else353 ], [ %largephi.extractslice0131, %if.then283 ], [ %largephi.extractslice0135, %if.else298 ], [ %largephi.extractslice0139, %if.then224 ], [ %largephi.extractslice0143, %if.else241 ], [ %largephi.extractslice0147, %sw.bb193 ], [ %largephi.extractslice0151, %sw.bb180 ], [ %largephi.extractslice0155, %sw.bb168 ], [ %largephi.extractslice0159, %sw.bb158 ], [ %largephi.extractslice0163, %sw.bb147 ], [ %largephi.extractslice0167, %if.then116 ], [ %largephi.extractslice0171, %if.else131 ], [ %largephi.extractslice0175, %sw.bb71 ], [ %largephi.extractslice0179, %sw.bb ], [ %largephi.extractslice0183, %if.end ], [ %largephi.extractslice0187, %if.end ], [ %largephi.extractslice0191, %if.end ], [ %largephi.extractslice0195, %if.end ], [ %largephi.extractslice0199, %if.end ] label %if.end %largephi.extractslice0183 = extractelement <4 x float> %div, i64 0 %largephi.extractslice0191 = extractelement <4 x float> %div, i64 0 in function blendop_Lab LLVM ERROR: Broken function found, compilation aborted! [1] 27586 IOT instruction (core dumped) darktable -d opencl uh that's great. Maybe blender? It worked! Not too bad for 2 minutes render: https://i.imgur.com/FD1SsQG.png https://i.imgur.com/FD1SsQG.png What about pytorch? It prompted this whole thing anyway: $ sudo pacman -S python-pytorch-rocm python-torchvision $ python neural_style/neural_style.py eval --content-image ../../2min.png --model ./saved_models/mosaic.pth --output-image out.png --cuda 1 [1] 32471 segmentation fault (core dumped) python neural_style/neural_style.py eval --content-image ../../2min.png $ sudo dmesg --follow [ 2467.536713] python[33309]: segfault at 68 ip 00007f12c5504d5d sp 00007ffc8f539c20 error 4 in libamdhip64.so.5.6.31062[7f12c541e000+357000] likely on CPU 14 (core 7, socket 0) [ 2467.536727] Code: ec 78 48 89 bd 78 ff ff ff 64 48 8b 04 25 28 00 00 00 48 89 45 c8 31 c0 85 f6 0f 88 09 03 00 00 48 8b 85 78 ff ff ff 48 63 de <48> 8b 50 68 48 8b 40 70 48 89 85 70 ff ff ff 48 29 d0 48 c1 f8 03 uh oh. Maybe I can crack some passwords? $ hashcat -m 0 -a 0 -o cracked.txt target_hashes.txt /usr/share/dict/american-english ... hiprtcCompileProgram(): HIPRTC_ERROR_COMPILATION error: unknown argument: '-flegacy-pass-manager' 1 error generated when compiling for gfx1032. * Device #1: Kernel /usr/share/hashcat/OpenCL/shared.cl build failed. Well, so much for that. Best I can get to work with rocm is 1/4 apps.

ryukoposting 3y ago

s/OpenCL/ROCm/g

vegabook 3y ago

With all due respect this is an insult to those of us who have loyally purchased AMD for numerous years, trying our very best to do compute with days, nay weeks, of attempts. Now 5 years too late we get told its suddenly their number one priority. Too late. Not only has all goodwill gone, but it's in deep negative territory. Even 50% lower performance stacks like Intel / Apple are much more appealing than AMD will ever be at this stage.

screcth 3y ago

What's stopping AMD from implementing CUDA? Just like Wine implemented Windows APIs

cowmix 3y ago
Why is this not upvoted more? Very good question.
dotnet00 3y ago
That is effectively what HIP is supposed to be (while sidesteppingsome copyright gray areas). They have a very close copy of the CUDA API and it can compile either for AMD GPUs or map onto the associated CUDA call for NVIDIA.
AstralStorm 3y ago
Nothing, HIP is essentially API compatible. That gets you nothing because CUDA nVidia optimized code will perform quite abysmally on a Radeon/Instinct. And furthermore nVidia has a bunch of proprietary libraries AMD has not cloned either. Normal people use Tensorflow, Keras or PyTorch anyway, not raw CUDA or even its libraries. The one place that is the stronghold of raw CUDA is molecular dynamics simulations because it's been written ages ago by some researcher who has never heard of Tensorflow etc. And probably uses cublas and/or cufft for which the AMD replacement is a joke and incompatible API. Situation there is slowly improving finally with Magma.

dingi 3y ago

[flagged]

justinclift 3y ago

Words versus actions. People don't really care about what the executive says. Especially when the same executive is also quoted with patently dishonest bullshit: > If you think about the product portfolio that AMD has, it’s arguably the broadest in the industry in terms of AI compute What AMD does is what people will pay attention to.

sorenjan 3y ago

Maybe I'll believe them when a consumer on Windows and Linux can download a binary from something like Meshlab or Automatic1111 and it just works on their gaming computer. If all they're interested in is selling CDNA to data centers I don't think they'll get enough mind share to be a realistic option. Also, is it really a good idea for various projects to add another proprietary platform? We should move away from Cuda and Rocm and towards open standards like Sycl. I don't want to have to care about who made my GPU, just as I don't have to care about who made my CPU.

mmis1000 3y ago
They did just start porting support ROCm to windows a few month ago(more specifically, ROCm 5.5.1 released a few months ago). And yea, ROCm for windows specifically supports rdna2 and rdna3 instead of cdna like ROCm for linux. So at least the title isn't a total lie. But ROCm for windows still have a few components missing. Will they finish the porting? Who knows? You may try to guess it.

ElectronBadger 3y ago

Really? Than you might explain why this list is so pitifully short: https://rocm.docs.amd.com/en/latest/release/gpu_os_support.html#linux-supported-gpus https://rocm.docs.amd.com/en/latest/release/gpu_os_support.h... and will get even shorter: https://rocm.docs.amd.com/en/latest/CHANGELOG.html#amd-instinct-mi50-end-of-support-notice https://rocm.docs.amd.com/en/latest/CHANGELOG.html#amd-insti... I'm so for AMD, but in terms of easily-accessible GPU computing ROCm is way behind CUDA.

eachro 3y ago

Not particularly relevant but the name "ROCm" is kind of terrible. Hard to pronounce, doesnt look good (the caps and then lower case is quite jarring). Minor details but I feel like these things do have a bit of downstream impact.