8 ms·
VUDA: A Vulkan Implementation of CUDA
- empyrrhicist 3y agoDoesn't look very active, does it still work?
- saboot 3y agoThat is quite interesting .. so I should be able to run my cuda accelerated programs on AMD and Intel devices then, correct?
- josalhor 3y agoAs someone who has never programmed directly for a GPU, how does this compare to HIP? Can this be an efficient abstraction over Nvidia and AMD GPUs?
- westurner 3y agoFrom https://news.ycombinator.com/item?id=34399633 https://news.ycombinator.com/item?id=34399633 : >>> hipify-clang is a clang-based tool for translating CUDA sources into HIP sources. It translates CUDA source into an abstract syntax tree, which is traversed by transformation matchers. After applying all the matchers, the output HIP source is produced. [...] (Edit) CUDA APIs supported by hipify-clang: https://rocm.docs.amd.com/projects/HIPIFY/en/latest/supported_apis.html https://rocm.docs.amd.com/projects/HIPIFY/en/latest/supporte...
- nightowl_games 3y agoHow does this relate to the goals outlined by George Hotz to bring ML to AMD chips and break the Nvidia dominance? I'm not an expert here but this approach seems powerful and important. But this system seems complex enough to doubt the ability of an individual to build. It seems like this would need a corporate sponsor to get off the ground. Perhaps AMD itself would be interested in paying engineers to iterate on this?
- jitl 3y agoHoltz is talking about drivers too, not just user space libraries. > The software is terrible! There’s kernel panics in the driver. You have to run a newer kernel than the Ubuntu default to make it remotely stable. I’m still not sure if the driver supports putting two cards in one machine, or if there’s some poorly written global state. When I put the second card in and run an OpenCL program, half the time it kernel panics and you have to reboot. He also talks about user space stuff but clearly he thinks the whole stack, above and below this kind of library also needs a lot of work.
- cwillu 3y agoPar for the course with new kernel things: it's unusual for something new in the kernel to be stable in the distro kernels unless they've devoted a great deal of effort to backport things.
- arthur2e5 3y agoThe way conservative distros define "stable" is part of the problem. For things less than 3 years old, going for stale versions often runs counter to "stable".
- iforgotpassword 3y agoThis is still so mind-boggling to me. AMD should be in a good financial position now that Zen was such a success, and that their GPUs are catching up too. Why are their drivers still a Clusterfuck across the board after all these years? Why not throw more manpower at the problem? I'm sure even if their GPUs were twice as fast as Nvidia's, everybody would still buy team green because it's better to have a card that works than a broken piece of garbage. We tried to get an MI50 to work reliably at work with KVM, but that thing was a complete dumpster fire. A colleague just bought a 7900XTX for gaming and spent days getting it to work. This included three Windows reinstalls. And that use case is gaming on Windows, which supposedly is the best supported case. It only gets worse from there. Compute on Linux? Lol. Now last time this topic came up, someone claimed that AMD is pretty much at their limits production wise, and there are a few unnamed large companies buying loads of their cards for compute and cloud gaming, and AMD basically has engineers dedicated to making sure things work exactly for their use case, so they don't have to really care about the rest. Sounds pretty wild, but not completely unrealistic...
- DrNosferatu 3y agoSounds great! How far is it actually compatible right now? Are there any tests / benchmarks? Can this be used to run CUDA-accelerated LLMs?
- johndough 3y agoThe Vulkan spec does not enforce strict IEEE 754 floating point semantics, so perfect compatibility with CUDA is impossible. https://registry.khronos.org/vulkan/specs/1.3-khr-extensions/html/chap43.html#spirvenv-op-prec https://registry.khronos.org/vulkan/specs/1.3-khr-extensions... However, the deep learning field does currently not pay much attention to reproducibility, so this might not be a big issue.
- my123 3y agoMemory management in Vulkan is _very_ restricted. Nothing remotely like UVM. CUDA on Vulkan for these reasons will always stay a pet project at best, with no shot at usable quality whatsoever.
- syntaxers 3y ago> > VUDA only takes kernels in SPIR-V format. VUDA does not provide any support for compiling CUDA C kernels directly to SPIR-V (yet). However, it does not know or care how the SPIR-V source was created - may it be GLSL, HLSL, OpenCL. So the answer is no, it can't be used with kernels that use cublas or cudnn, which excludes almost all ML use-cases.
- AndrewKemendo 3y agoThis could be a big deal if we have an actual alternative to CUDA I can’t see NVIDIA letting this just exist
- Ballas 3y agoMy opinion is that CUDA is not the mote keeping the others out - it's the CUDNN (and CUBLAS), more specifically the level to which they are optimized.
- roenxi 3y agoI'm not even certain optimisation matters. I can crash my machine (AMD graphics) with a stock Debian install by letting something attempt BLAS on the GPU. The situation is starting to improve though. Installed a bunch of libraries from https://repo.radeon.com/rocm/apt/5.4 https://repo.radeon.com/rocm/apt/5.4 jammy main and the crashes got less frequent. I don't have a lot of faith in AMD to deliver reliable BLAS libraries at this point, but it could happen. The hardware is there, I just don't think they're prioritising supporting the right places in the distribution chain or supporting consumer-level graphics.
- metal_am 3y agoHow about something like MAGMA?
- roenxi 3y agoNo idea. But if it goes through OpenCl it will likely expose the same bugs.
- Ballas 3y agoI do find it strange that AMD is not allocating more resources to ROCM, given that that seems to be where the money is, at least from my viewpoint. I guess they have been able to sell more cards than they could manufacture, but that seems to be changing.
- deleted 3y ago
- gymbeaux 3y agoThings like this pop up relatively often but they never pick up steam and I am still using Nvidia GPUs. I would imagine this is no different.
- wtcactus 3y agoSeems dead. Last commit was February 2022.
- qwertox 3y agoAnd that was only one line added. Most of the code is 3 and 5 years old.
- fancyfredbot 3y agoIt's not an implementation of CUDA, it's an implementation of the CUDA runtime API. The API is used to configure the card, allocate and copy memory, and run kernels. Importantly you cannot use this to write the actual kernels which run on the GPU!
- sheepscreek 3y agoI was half hoping this meant running CUDA code on AMD GPUs. Thanks for clarifying.
- empyrrhicist 3y agoI know AMD has a whole bunch of (related?) projects for GPU compute, but man - if they could just provide an interop layer that Just Works they'd get immediate access to so much more market share.
- heyoni 3y agoThen they too could call themselves an AI company!
- empyrrhicist 3y agoI've been hoping for it for so long - I wonder if there's enough interest that someone could do a GoFundMe to hire at least one full time dev lol.
- sanxiyn 3y agoEh well, it is very close to just working. From "Training LLMs with AMD MI250 GPUs and MosaicML": > It all just works. No code changes were needed. https://www.mosaicml.com/blog/amd-mi250 https://www.mosaicml.com/blog/amd-mi250
- paulmd 3y ago“Just works” in this context means executing the compiled CUDA or the PTX bytecode without recompiling. Nobody is ever going to utilize ROCm if it requires distributing as source and recompiling. To make it even more insulting, even simply installing ROCm itself is a massive burden, even on an ostensibly-supported (as geohot discovered) and even just “it works out of the box if you distribute and compile it locally” is ignoring that whole massive “draw the rest of the owl” stage of getting ROCm installed and building properly in your environment.
- netheril96 3y agoJust in case other people who have AMD GPU and run Windows have the same needs as I have, that is, to train or run machine learning models, please checkout torch-directml and tensorflow-directml.
- bornfreddy 3y agoDoes that work? I might be in market for a new GPU if AMD had something that beats NVidia for ML (for sane price)... I can't really justify buying NVidia GPU, anything decent is too expensive.
- netheril96 3y agoIt works for me. I have no issues training a WGAN with it. But I don’t know how much slower it is compared to CUDA on a similar priced NVIDIA card.
- skocznymroczny 3y agoHow does it work? Last time I tried DirectML it wasn't well supposed and there was little software which supported it. Also the performance seemed to be not too great. I am currently using a Linux install because with ROCm I can use popular tools like Automatic111 webui and oobabooga.
- netheril96 3y agoI trained a WGAN on torch-directml with no issues so the software seems quite supported. But I can’t speak of performance because I have nothing to compare against.
- skocznymroczny 3y agoI gave it a try. Yeah seems like it works well on the functional side, but the performance was at least 4x slower than what ROCm on Linux gives me.
- HarHarVeryFunny 3y ago
- einpoklum 3y ago1. This implements the clunky C-ish API; there's also the Modern-C++ API wrappers, with automatic error checking, RAII resource control etc.; see: https://github.com/eyalroz/cuda-api-wrappers https://github.com/eyalroz/cuda-api-wrappers (due disclosure: I'm the author) 2. Implementing the _runtime_ API is not the right choice; it's important to implement the _driver_ API, otherwise you can't isolate contexts, dynamically add newly-compiled JIT kernels via modules etc. 3. This is less than 3000 lines of code. Wrapping all of the core CUDA APIs (driver, runtime, NVTX, JIT compilation of CUDA-C++ and of PTX) took me > 14,000 LoC.
- xeonmc 3y agoHuge missed opportunity to call it "Vuudoo"
- Conscat 3y agoMight be confusing with Voodoo https://github.com/cogciprocate/voodoo https://github.com/cogciprocate/voodoo
- userbinator 3y agoThat's already confusing enough with the 3dfx GPUs of the same name: https://en.wikipedia.org/wiki/3dfx_Interactive#Product_development_history https://en.wikipedia.org/wiki/3dfx_Interactive#Product_devel...
- panzi 3y agoThere needs to be a 3rd implementation called SHUDA.
- jaimex2 3y agoI have no hope for AMD. They should have made compatibility tools yesterday.