12 ms·
CUDA for AMD on Windows
- chiassedu80 2mo agoCUDA for AMD on Windows I’ve been working on a Windows setup that lets CUDA-targeted applications run on AMD GPUs using ZLUDA + ROCm/HIP. Repo: https://github.com/Speedstu/CUDA-for-AMD-Windows https://github.com/Speedstu/CUDA-for-AMD-Windows So far, it has only been tested on my RX 9060 XT (gfx1200), where I’ve used it with CUDA-enabled LibTorch workloads, including long ai training and use. I also added a GPU scanner / auto-detection system that detects: AMD GPU model gfxXXXX architecture ROCm/HIP installation driver info whether the GPU has already been validated by the project Example: RX 9060 XT → gfx1200 → RDNA4 → HIP detected → validated The goal now is to test it on more hardware, especially RX 6000 / 7000 / 9000 cards. If you have an AMD GPU on Windows and want to try it, I’d really appreciate compatibility reports working or broken. There’s a dedicated GPU compatibility issue template in the repo. If this is useful to you, a star would also help the project get more testers.
- nine_k 2mo ago(As a side note, I love the name ZLUDA; it very aptly means "delusion" or "deception" in Polish.)
- swerner 2mo agoTIL, I didn’t know that. I always assumed it came from level zero”, the Intel computer layer that ZLUDA was translating to before its developer was hired by AMD to target HIP.
- harhargange 2mo agoHow about Linux support? I have 7900xt with linux
- system2 2mo agoI wish there were a way to use RDNA1 cards with CUDA for AMD. My 5700XTs are sitting in a drawer.
- monster_truck 2mo agoRDNA1 isn't good for a whole lot, even flagship RDNA2 cards are a stretch for many things. The lack of WMMA/matrix multiply/BF16 is too severe of a penalty. The FP16 throughput on RDNA1 is both shader reliant and requires everything to be packed first. Even with 2 or 4 or 1000 cards, you would be consuming all of the available memory and memory bandwidth just packing and unpacking values, and if you really want to dump a hundred billion tokens into making it work anyways, you're only going to find out that even if you bother to sit there ferrying packed values to ram or disk before then issuing the instructions, paying that already severe penalty again when the values then have to be unpacked is so steep of a cost that the 256 BF16 flops/cu/clock's effective throughput is outright lower than simply doing it on a Zen 2 processor. You also don't have INT8 (or really INT4) on RDNA1 so the other RNS/CRT tricks aren't viable. Sadly RDNA1's VCN2 also lacks actually good x264 bframe encoding support, or even P010 for 10 bit color, so what I'm saying is you should sell them. Used Radeon VII's are like $260, you'll go a lot further with those especially if you throw in a 7900XTX, and then augment that further with a 9070 CRE (you only want it for its int8 cores), and of course 128GB of ram. E: And sure, that's 3, or ideally 4 GPUs, and a good bit of extra work. But that gets you up to more than halfway to the naive performance of a $15,000 MI300x in a surprising amount of cases, with additional strengths that it lacks. For far less than half of the cost
- system2 2mo agoI agree, I should've sold them last year when they hit $500 each. I have like 8 of them from my mining days. What a silly mistake I made.
- dracotomes 2mo agoI still have a 5700XT I bought in 2019 (i think) for 300€ in my gaming rig. Crazy that they were going for $500 6 years later.
- system2 2mo agoI just checked, they are going for $150-180 on eBay U.S. I guess I will let them go now.
- lulzx 2mo agoI made cuda-metal btw (for mac kek), https://github.com/lulzx/cuda-metal https://github.com/lulzx/cuda-metal
- linuxhansl 2mo agoOff-topic and somewhat of a rant, but I'd far prefer us all focusing on open standards like HIP, SYCL, OpenCL, etc. It's unbearable that most LLM inference happens on closed H/W, closed drivers, and closed SDKs.
- bigyabai 2mo agoI would too, but sadly that's Khronos' job to organize, and they've had trouble getting American vendors to work together. It's likely that CUDA will continue dominating until they put aside their differences. The current MLX/MPS/ROCm ecosystems are too fractured to threaten Nvidia.
- boredatoms 2mo agoMaybe a not-Khronos org should try
- high_na_euv 2mo agoWdym American Vendors? Intel uses SPIRV iirc
- bigyabai 2mo agoI'm talking about holistic efforts like OpenCL, and standards that would be equivalent to Nvidia's "Compute Capability" versioning. The basic underlying tech can be agreed on, but Apple/AMD/Intel all have different GPU priorities that limit their ability to agree on a CUDA-adjacent hardware platform.
- swerner 2mo agoWhat do you mean by holistic? SYCL is an open versioned standard that allows for vendor specific extensions. The problem is not that there isn’t a proper standard, the problem is that many hardware vendors - or software developers simply don’t want to adopt it. Intel (via Codeplay) was handing it out on a silver platter - Nvidia on SYCL, full top chain, and people still wouldn’t want it.
- swerner 2mo agoAI will take down Nvidia’s moat. When it becomes trivial to translate CUDA/PTX to HIP, SYCL or Metal, CUDA is no longer the moat, it becomes the intermediate representation.
- bayindirh 2mo ago> When it becomes trivial to translate CUDA/PTX to HIP,... ZLUDA is already doing that, no?
- swerner 2mo agoI don't think we're at a point yet where anyone would trust ZLUDA enough to ship commercial products that rely on it. I would be delighted though, if anyone can prove me wrong.
- bayindirh 2mo agoNo, but we can go there. This is an open source project. Anyone can put some more effort behind it and push it further. It's improving, AFAICS. Src: https://github.com/vosen/ZLUDA https://github.com/vosen/ZLUDA
- swerner 2mo agoUnfortunately, many of very good ideas end with “it’s open source, anyone can contribute” because very few actually do.
- mrheosuper 2mo agoWhat about spinning up a thousand agent and let them run wild ? Like many projects are doin
- bayindirh 2mo agoFor CRUD, that can work. For something like that, it's almost impossible. It's niche, performance and code-quality sensitive, needs deep and wide knowledge, and hardware dependent.
- Nurysso 2mo agoman i can't say how much i used to like cuda when i had a nvidia gpu it made ml so much fun and on amd its a war especially on rdna 2 cards which i have. hopefully one day we will be able to properly translate cuda for its amd counter parts
- latchkey 2mo agoThere are also interesting efforts like: https://github.com/Zaneham/Booth https://github.com/Zaneham/Booth https://scale-lang.com/ https://scale-lang.com/
- KennyBlanken 2mo agoTo save everyone a click: No cuDNN and based off an ancient version of ROCm for windows (7.1 has been out for ages, 7.2 is current.)
- woctordho 2mo agoNo, 10.1 is current
- triwats 2mo agoInteresting option for CDNA architecture chips. I wonder if this moves to an open standard? AMD GPUs build for AI specs for reference: https://flopper.io/gpus?vendor=AMD&page=1 https://flopper.io/gpus?vendor=AMD&page=1
- vivzkestrel 2mo agostupid question - what exactly is CUDA? - is it patented hardware tech that AMD cannot replicate? - is it a library that ll also work on AMD while it currently works on nvidia - can AMD cook an equivalent of CUDA, if so why havent they done it?
- pjmlp 2mo agoIt is an ecosystem. At the botton you have the PTX bytecode. Followed by (from NVidia) compilers for C, C++, Fortran, Python JIT and more recently Julia JIT. Additionally there are third party support for .NET, Java, Haskell tooling to also target CUDA. Then you have the libraries ready to plug into the official NVidia compilers to reduce the need to explicitly write CUDA code manually. Finally you have the graphical debugging and profiling tools that make targeting GPUs as easy as regular CPU programming, ignoring the hardware differences for a moment. All wannabe replacements fail, because they focus only on the CUDA C++ compiler and nothing else.
- harhargange 2mo agoIf anyone has used this, how good is this? And will it allow running AMD GPU with MATLAB? I have a 7900xt
- maxlin 2mo agoI can only hope NVIDIA won't send an army of lawyers against this.
- qalmakka 2mo ago> CUDA for AMD on Windows This gives me strong "Bash on Ubuntu on Windows" kind of vibes
- Shocka1 2mo agoRight, it reminds me of this website back in the day, when people really were coding these things as well instead of just generating 100% of it. Being sarcastic here, but stuff like "How I coded Java with Rust" were a common thing hah!
- imnotr0b0t 2mo ago[dead]
- atsetilam 2mo agoI think projects like this one are a clear indication that CUDA *is* the de facto standard and that aspiring open standards are competing for their niche. The rest of the world is still living in the CUDA ecosystem. Asking accelerated HPC devs to port every project to a different language (and possibly maintain two codebases) is something we haven't asked of anyone else for decades now. And yes, LLMs can help (although CUDA makes up the overwhelming majority of their training data, with the hilarious result that you need to give them a disambiguation table like "you'll want to write $CUDA, but write $else"), but AI is clearly not the correct tool for the job. A compiler is. ZLUDA is JIT (it's working with the binary, not the source, which usually means you've lost some information) and it doesn't support CDNA, which is a showstopper for enterprises, tho it's a great project and it's awesome seeing it work on Windows (I wonder whether MS Defender complains about it due to library injection? IIRC, it used to. That'd be another enterprise showstopper) Someone here mentioned scale-lang.com already as a portability tool - it also beats HIP in the performance game, on average. scale-lang.com/s/discord is where its community lives. disclaimer: I am part of SCALE's team, Spectral Compute