5 ms·
Certainly impressive that this is possible! However, for my use cases (running on arbitrary client hardware) I generally distrust any abstractions over the GPU
by vouwfietsman 1y ago
Certainly impressive that this is possible!
However, for my use cases (running on arbitrary client hardware) I generally distrust any abstractions over the GPU api, as the entire point is to leverage the low level details of the gpu. Treating those details as a nuisance leads to bugs and performance loss, because each target is meaningfully different.
To overcome this, a similar system should be brought forward by the vendors. However, since they failed to settle their arguments, I imagine the platform differences are significant. There are exceptions to this (e.g Angle), but they only arrive at stability by limiting the feature set (and so performance).
Its good that this approach at least allows conditional compilation, that helps for sure.
- theknarf 1y agoIf only everyone could have agreed upon spir-v
- kookamamie 1y agoExactly. Not sure why it would be better to run Rust on Nvidia GPUs compared to actual CUDA code. I get the idea of added abstraction, but do think it becomes a bit jack-of-all-tradesey.
- MuffinFlavored 1y ago> Exactly. Not sure why it would be better to run Rust on Nvidia GPUs compared to actual CUDA code. You get to pull no_std Rust crates and they go to GPU instead of having to convert them to C++
- rbanffy 1y agoI think the idea is to allow developers to write a single implementation and have a portable binary that can run on any kind of hardware. We do that all the time - there are lots of code that chooses optimal code paths depending on runtime environment or which ISA extensions are available.
- kookamamie 1y agoSure. The performance-purist in me would be very doubtful about the result's optimality, though.
- littlestymaar 1y agoThe performance purist don't use Cuda either though (that's why Deepseek used PTX directly). Everything is an abstraction and choosing the right level of abstraction for your usecase is a tradeoff between your engineering capacities and your performance needs.
- LowLevelMahn 1y agothis Rust demo also uses PTX directly During the build, build.rs uses rustc_codegen_nvvm to compile the GPU kernel to PTX. The resulting PTX is embedded into the CPU binary as static data. The host code is compiled normally.
- LegNeato 1y agoTo be more technically correct, we compile to NVVM IR and then use NVIDIA's NVVM to convert it to PTX.
- saagarjha 1y agoThat’s not really the same thing; it compiles through PTX rather than using inline assembly.
- LegNeato 1y agoFYI, you can drop down into ptx if need be: https://github.com/Rust-GPU/Rust-CUDA/blob/aa7e61512788cc70200005eb318b7441a97fad6d/crates/cuda_std/src/mem.rs#L40 https://github.com/Rust-GPU/Rust-CUDA/blob/aa7e61512788cc702...
- brandonpelfrey 1y ago
- the__alchemist 1y agoI think the sweet spot is: If your program is written in rust, use an abstraction like Cudarc to send and receive data from the GPU. Write normal CUDA kernels.
- Ar-Curunir 1y agoBecause folks like to program in Rust, not CUDA
- tucnak 1y ago"Folks" as-in Rust stans, whom know very little about CUDA and what makes it nice in the first place, sure, but is there demand for Rust ports amongst actual CUDA programmers? I think not.
- LegNeato 1y agoFYI, rust-cuda outputs nvvm so it can integrate with the existing cuda ecosystem. We aren't suggesting rewriting everything in Rust. Check the repo for crates that allow using existing stuff like cudnn and cuBLAS.
- tucnak 1y agoI take it you're the maintainer. Firstly, congrats on the work done, for the open source people are a small crowd, and determination of Rust teams here is commendable. On the other hand, I'm struggling to see the unique value proposition. What is your motivation with Rust-GPU? Graphics or general-purpose computing? If it's the latter, at least from my POV, I would struggle to justify going up against a daunting umbrella project like this; in view of it likely culminating in layers upon layers of abstraction. Is the long-term goal here to have fun writing a bit of Rust, or upsetting the entrenched status quo of highly-concurrent GPU programming? There's a saying that goes along like "pleasing all is a lot like pleasing none," and intuitively I would guess it should apply here.
- LegNeato 1y agoThanks! I'm personally focused on compute, while other contributors are focused on graphics. I believe GPUs are the future of computing. I think the tooling, languages, and ecosystems of GPUs are very bad compared to CPUs. Partially because they are newer, partially because they are different, and partially because for some reason the expectations are so low. So I intend to upset the status quo.
- JayEquilibria 1y agoGood stuff. I have been thinking of learning Rust because of people here even though CUDA is what I care about. My abstractions though are probably best served by Pytorch and Julia so Rust is just a waste of time, FOR ME.
- diabllicseagull 1y agosame here. I'm always hesitant to build anything commercial over abstractions, adapter or translation layers that may or may not have sufficient support in the future. sadly in 2025, we are still in desparate need for an open standard that's supported by all vendors and that allows programming for the full feature set of current gpu hardware. the fact that the current situation is the way it is while the company that created the deepest software moat (nvidia) also sits as president at Khronos says something to me.
- pjmlp 1y agoKhronos APIs are the C++ of graphics programming, there is a reason why professional game studios never do political wars on APIs. Decades of exerience building cross platform game engines since the days of raw assembly programming across heterogeneous computer architectures. What matters are game design and IP, that they eventually can turn into physical assets like toys, movies, collection assets. Hardware abstraction layers are done once per platform, can even leave an intern do it, at least the initial hello triangle. As for who seats as president at Khronos, so are elections on committee driven standards bodies.
- ducktective 1y agoI think you are very experienced in this subject. Can you explain what's wrong with WebGPU? Doesn't it utilize like 80% of the cool features of the modern GPUs? Games and ambitious graphics-hungry applications aside, why aren't we seeing more tech built on top of WebGPU like GUI stacks? Why aren't we seeing browsers and web apps using it? Do you recommended learning it (considering all the things worth learning nowadays and rise of LLMs)
- 3836293648 1y agoFirst of all WebGPU has only been supported in Chrome for a few months and Firefox in the next release. And that's just Windows. We haven't had enough time to develop anything really. Secondly, the WebGPU standard is like Vulkan 1.0 and is cumbersome to work with. But that part is hearsay, I don't have much experience with it.
- littlestymaar 1y agoEverything is an abstraction though, even Cuda abstracts away very difference pieces of hardware with totally different capabilities.
- LegNeato 1y agoRust is a system language, so you should have the control you need. We intend to bring GPU details and APIs into the language and core / std lib, and expose GPU and driver stuff to the `cfg()` system. (Author here)
- Voultapher 1y agoWho is we here? I'm curious to hear more about your ambitions here, since surly pulling in wgpu or something similar seems out-of-scope for the traditionally lean Rust stdlib.
- LegNeato 1y agoMany of us working on Rust + GPUs in various projects have discussed starting a GPU working group to explore some of these questions: https://gist.github.com/LegNeato/a1fb3e3a9795af05f22920709d92e456 https://gist.github.com/LegNeato/a1fb3e3a9795af05f22920709d9... Agreed, I don't think we'd ever pull in things like wgpu, but we might create APIs or traits wgpu could use to improve perf/safety/ergonomics/interoperability.
- Voultapher 1y agoCool, looking forward to that. It's certainly a good fit for the Rust story overall, given the increasingly heterogenous nature of systems.
- junon 1y agoI'm surprised there isn't already a Rust GPU WG. That'd be incredible.
- jpc0 1y agoHears an idea, Get Nvidia, AMD, Intel and whoever else you can get into a room. Get LLVMs boys into the same room. Compile LLVMIR directly into hardware instructions fed into the GPU, get them to open up. Having to target an API is part of the problem, get them to allow you to write Rust that directly compiles into the code that will run on the GPU, not something that becomes something else, that becomes spirv that controls a driver that will eventually run on the GPU.
- ants_everywhere 1y agoGenuine question since you seem to care about the performance: As an outsider, where we are with GPUs looks a lot like where we were with CPUs many years ago. And (AFAIK), the solution there was three-part compilers where optimizations happen on a middle layer and the third layer transforms the optimized code to run directly on the hardware. A major upside is that the compilers get smarter over time because the abstractions are more evergreen than the hardware targets. Is that sort of thing possible for GPUs? Or is there too much diversity in GPUs to make it feasible/economical? Or is that obviously where we're going and we just don't have it working yet?
- nicoburns 1y agoThe status quo in GPU-land seems to be that the compiler lives in the GPU driver and is largely opaque to everyone other than the OS/GPU vendors. Sometimes there is an additional layer of compiler in user land that compilers into the language that the driver-compiler understands. I think a lot of people would love to move to the CPU model where the actual hardware instructions are documented and relatively stable between different GPUs. But that's impossible to do unless the GPU vendors commit to it.
- sim7c00 1y agoi think intel and amd provide ISA docs for their hw. not sure about nvidia didnt check it in forever
- pornel 1y agoI would like CPUs to move to the GPU model, because in the CPU land adoption of wider SIMD instructions (without manual dispatch/multiversioning faff) takes over a decade, while in the GPU land it's a driver update. To be clear, I'm talking about the PTX -> SASS compilation (which is something like LLVM bitcode to x86-64 microcode compilation). The fragmented and messy high-level shader language compilers are a different thing, in the higher abstraction layers.
- rowanG077 1y agoSo what do you use? CUDA abstracts over the GPU hardware, opencl does, vulkan does. I guess you can write raw PTX?
- hyperbolablabla 1y agoWhat we really need is a consistent GPU ISA. If it wasn't for the fairly recent proliferation of ARM CPUs, we more or less would've rallied around x86 as the de facto ISA for general purpose compute. I'm not sure why we couldn't do the same for GPUs as well.