9 ms·
I find it odd that given the billions of dollars involved, no competitor has managed to replicate the functions of CUDA. Is it that hard to do, or is the softw
by RachelF 1y ago
I find it odd that given the billions of dollars involved, no competitor has managed to replicate the functions of CUDA.
Is it that hard to do, or is the software lock-in so great?
- deleted 1y ago[deleted]
- ronsor 1y agoThe problem is that CUDA is tightly integrated with NVIDIA hardware. You don't just have to replicate CUDA (which is a lot of tedious work at best), but you also need the hardware to run your "I can't believe it's not CUDA"
- shmerl 1y agoA better question is why there is no stronger push for a nicer GPU language that's not tied to any particular GPU and serves any purpose of GPU usage (whether it's compute or graphics). I mean efforts like rust-gpu: https://github.com/Rust-GPU/rust-gpu/ https://github.com/Rust-GPU/rust-gpu/ Combine such language with Vulkan (using Rust as well) and why would you need CUDA?
- bee_rider 1y agoI think Intel Fortran has some ability to offload to their GPUs now. And Nvidia has some stuff to run(?) CUDA from Fortran. Probably just needs a couple short decades of refinement…
- pjmlp 1y agoOne of the reasons CUDA won over OpenCL, was that NVidia, contrary to Khronos, saw a value in helping those HPC researchers move their Fortran code into the GPU. Hence they bought PGI, and improved their compiler. Intel eventually did the same with Open API (which isn't plain OpenCL, rather an extension with Intel goodies). I was on a Khronos webminar where the panel showed disbelief why anyone would care about Fortran, oh well.
- kristianp 1y agoIt's insane how big the NVidia dev kit is. They've got a library for everything. It seems like they have as broad software support as possible.
- bee_rider 1y agoThat’s actually pretty surprising to me. Of course, there are always jokes about Fortran being some language that people don’t realize is still kicking. But I’d expect a standards group that is at least parallel computing adjacent to know that it is still around.
- pjmlp 1y agoYet not only they joked about Fortran, it took CUDA adoption success, for them to take C++ seriously and come up with SPIR as counterpoint to PTX. Which in the end was worthless because both Intel and AMD botched all OpenCL 2.x efforts. Hence why OpenCL 3.0 is basically OpenCL 1.0 rebranded, and SYSCL went its own way. It took a commercial company, Codeaplay a former compiler vendor for games consoles, to actually come up with a good tooling for SYSCL. Which Intel in the middle of extending SYSCL with their Data Paralell C++, eventually acquired. Those products are in the foundation of One API, and naturally go beyond what barebones OpenCL happens to be. The mismanagement Khronos has done with OpenCL is one of the reasons Apple lost ties with Khronos.
- JonChesterfield 1y agoI like Julia for this. Pretty language, layered on LLVM like most things. Modular are doing interesting things with Mojo too. People seem to like cuda though.
- shmerl 1y agoCUDA is just DOA as a nice language being Nvidia only (not counting efforts like ZLUDA).
- JonChesterfield 1y agoThat's a compiler problem. Once could start from clang -xcuda and hack onwards. Or work in the intersection of CUDA and HIP which is relatively broad if a bit of a porting nuisance.
- shmerl 1y agoMay be, but who is working on that compiler? And the whole ecosystem is controlled by a nasty company. You don't want to deal with that. Besides, I'd say Rust is a nicer language than CUDA dialects.
- JonChesterfield 1y agoChris and Nick originally, a few more of us these days. Spectral compute. We might have a nicer world if people had backed opencl instead of cuda but whatever. Likewise rust has a serious edge over c++. But to the compiler hacker, this is all obfuscated SSA form anyway, it's hard to get too emotional about the variations.
- shmerl 1y agoLanguage would matter more for those who actually would want to write some programs in it. So I'd say rust-gpu is something that should get more backing.
- 1y ago
- pjmlp 1y agoTooling and ecosystem, that is why.
- shmerl 1y agoRust has great tooling and ecosystem. The point here is more of interest of those who want better alternatives to CUDA. AMD would be an obvious beneficiary to back the above, so I'm surprised about some lack of interest from their likes.
- pjmlp 1y agoIt has zero CUDA tooling, that is what is relevant when positioning itself as alternative to C, C++, Fortran, Python JIT, PTX based compilers, compute libraries, Visual Studio and Eclipse integration, graphical debugger. Cross compiling Rust into PTX is not enough to make researchers leave CUDA.
- shmerl 1y agoAnd CUDA has zero non CUDA tooling. That's a pointless circular argument which doesn't mean anything. Rust has Rust tooling and it's very good. Being language agnostic is also not the task of the language, but task of IR. There is already a bunch of languages, such as Slang. The point is to use Rust itself for it.
- pjmlp 1y agoWhere is the graphical debugging experience for Rust, given that it so great tooling? Slang belongs to NVidia, and was nicely given to Khronos, because almost everyone started relying on HLSL, given that Khronos decided not to spend any additional resources on GLSL. Just like with Mantle and Vulkan, Khronos seems that without external help they aren't able to produce anything meaningful since Long Peak days.
- fnands 1y agoMojo might be what you are looking for: https://docs.modular.com/mojo/manual/gpu/intro-tutorial/ https://docs.modular.com/mojo/manual/gpu/intro-tutorial/ The language is general, but the current focus is really on programming GPUs.
- chickenzzzzu 1y agoVulkan is at 95% of CUDA performance already. The remaining 5% is CUDA's small dispatch logic. The reason why people continue to use CUDA and Pytorch and so on is because they are literally too stupid and too lazy to do it any other way
- pjmlp 1y agoWith zero tooling, hence why no one cares about Vulkan, other than Valve and Google.
- chickenzzzzu 1y agoWhat tooling do you need? I'll make it for you for free
- pjmlp 1y agoGreat, lets start with a Fortran compiler like CUDA has. When you're done, you can create IDE plugins, and a graphical debugger with feature parity to NInsights.
- chickenzzzzu 1y agoOk, that's a good retort. How many months of work do those things save you, compared to actually solving the problem you want to solve without those tools? The argument you are making sounds to me like, "well good luck making a Vulkan application without cmake, ninja, meson, git, visual studio, clion" etc, when in reality a 5 line bash script to gcc works just fine
- pjmlp 1y agoFirst step is to understand why proprietary technology gets adoption. Lacking understanding is doomed to failure.
- triknomeister 1y ago
- JonChesterfield 1y agoI'm pretty sure it's a political limitation, not a technical one. Implementing it is definitely a pain - it's a mix of hardcore backwards compatibility (i.e. cruft) and a rapidly moving target - but it's also obviously just a lot of carefully chosen ascii written down in text files. The non-nvidia hardware vendors really don't want cuda to win. AMD went for open source + collaborative in a big way, opencl then hsa. Both broadly ignored. I'm not sure what Intel are playing at with spirv - that stack doesn't make any sense to me whatsoever. Cuda is alright though, in a kind of crufty obfuscation over SSA sense. Way less annoying than opencl certainly. You can run it on amdgpu hardware if you want to - https://docs.scale-lang.com/stable/ https://docs.scale-lang.com/stable/ and https://github.com/vosen/ZLUDA https://github.com/vosen/ZLUDA already exist. I'm hacking on scale these days.
- joe_the_user 1y agoThe thing that's also worth saying is that everyone speaks vaguely about CUDA's "institutional memory" and investment and so forth. But the concrete qualities of CUDA and Nvidia's offerings generally is a move toward general purpose parallel computing. Parallel processing is "the future" and approach of just do loop and have each iteration be parallel is dead simple. Which is to say Nvidia has invested a lot in making "easy things easy along with hard things no harder". In contrast, other chip makers seem to be acculturated to the natural lock-in of having a dumb, convoluted interface to compensate for a given chip being high performance.
- joe_the_user 1y agoCUDA does involve a massive investment for Nvidia. It's not that it's impossible to replicate the functionality. But once a company has replicated that functionality, that company basically is going to be selling at competitive prices, which isn't a formula for high profits. Notably, AMD funded a CUDA clone, ZLUDA, and then quashed it[1]. Comments at the time here involved a lot of "they would always be playing catch up". I think the mentality of chip makers generally is that they'd rather control a small slice of a market than fight competitively for a large slice. It makes sense in that they invest years in advance and expect those investments to pay high profits. [1] https://www.tomshardware.com/pc-components/gpus/amd-asks-developer-to-take-down-open-source-zluda-dev-vows-to-rebuild-his-project https://www.tomshardware.com/pc-components/gpus/amd-asks-dev...
- noosphr 1y agoCuda isn't a massive investment, it's 20 years worth of institutional knowledge with a stable external api. There are very few companies outside of 00s Microsoft who have managed to support 20 years worth of backward compatibility along with the bleeding edge.
- Izikiel43 1y ago> Cuda isn't a massive investmen > it's 20 years worth of institutional knowledge with a stable external api > There are very few companies outside of 00s Microsoft who have managed to support 20 years worth of backward compatibility along with the bleeding edge. To me that sounds like massive investment
- triknomeister 1y agoZLUDA was quashed due to concerns about infringement /violating terms of use.
- joe_the_user 1y agoThat was the story but the legality of cloning an API/ABI/etc is well established by, for example Google vs Oracle (though with gotchas that might Nvidia to put a legal fight).
- pjmlp 1y agoBecause most fail to understand what makes CUDA great, and keep only trying to replicate C++ API. They overlook CUDA is a polyglot ecosystem composed by C, C++ and Fortran as main languages, with Python JIT DSL since this year, compiler infrastructure for any compiler backend that wishes to target it of which there are a few including strange stuff like Haskell, IDE integration with Eclipse and Visual Studio, graphical debugging just like on the CPU. It is like when Khronos puts out those spaghetti riddled standards, expecting each vendor/open source community, to create some kind of SDK, versus the vertical integration of console devkits and proprietary APIs, and then asking why professional studios have no qualms with proprietary tooling.
- oivey 1y agoSlight correction: CUDA Python JIT has existed for a very long time. Warp is a late comer.
- pjmlp 1y agoKind of, none of those are at the integration level of CUTLASS 4, and the new cu tile architecture, introduced at GTC 2025. But you're right there was already something in place.
- oivey 1y agoI took a closer look at some of that and it’s pretty cool. Definitely neat to have some good higher level abstractions than the old C-style CUDA syntax that Numba was built on.
- melodyogonna 1y agoCuda is so many things I'm not sure it is even possible to replicate it.
- triknomeister 1y agoIt is hard to do in the sense that it requires a very good taste about programming languages, which in turn requires really listening to the customers, and that requires huge number of people who are skilled. And no one has really invested that much money into their software ecosystem yet.