4 ms·
Since NVIDIA owns huggingface now and huggingface has the excellent Candle [1] crate for inference on Rust, this seems like a good step towards nice native Rust
by dllu 17d ago
Since NVIDIA owns huggingface now and huggingface has the excellent Candle [1] crate for inference on Rust, this seems like a good step towards nice native Rust kernels.
[1] https://github.com/huggingface/candle https://github.com/huggingface/candle
- jacobgorm 17d agoNobody cares if kernels are written in Rust. Kernels were meant to be written in C, but if you want to go more high-level try Triton or a similar DSL that nicely abstract tile sizes etc.
- deleted 17d ago[deleted]
- cpill 17d agooh no no no, this is going to break the CPP hold on AI and game dev.
- pjmlp 16d agoNah, Rust compiler still needs C++ to be built in first place, and everyone on AI uses LLVM as infrastructure.
- keithnz 17d agokernels aren't meant to be written by any defined language. C is just a traditionally good default language that took over from assembly. No particular reason we have to stick with C.
- chadcmulligan 17d agoAnd quite a few reasons that something better than C should be used. Rust seems a good candidate.
- jacobgorm 16d agoWhat reasons would you have to prefer Rust over C for compute kernels? I am a great fan of Rust, but I don't see any benefit for kernels, due to their relatively simple nature.
- chadcmulligan 15d agoI'm not sure why being relatively simple would mean C over Rust? Rust still has the safety advantages.
- pjmlp 16d agoThat is exactly why OpenCL failed adoption, focusing on C, instead of being polyglot like CUDA.
- zozbot234 16d agoSYCL is the natively polyglot counterpart, with practical implementations of it compiling down to the same sort of SPIR-V kernels as OpenCL. (OTOH, much of the current adoption on the open standards side seems to target the more widely supported SPIR-V compute shaders, via Vulkan compute.)
- pjmlp 16d agoNot really, first of all it is for C++, not the range of languages supported by CUDA. Before SPIR was a thing in OpenCL, Khronos could not understand why anyone would care about anything else other than C99, or why supporting Fortran on GPUs was at all relevant. Secondly, from the competition only Intel cares about SYCL with their own sugar on top, OpenAPI. AMD hasn't cared one second about it. You may mention Codeplay, which is anyway an Intel owned company since 2022. As for Vulkan, it doesn't have neither the features, nor the tooling that CUDA enjoys, it is the usual putting up with using LEGOs from different brands, with various pin sizes, that is so common with Khronos.
- Anoian 16d agoI have never seen a comment this gray
- instagraham 16d agonoob here - what's the benefit of this? Will using Rust lead to more optimal LLMs or code or both?
- jvanderbot 16d agoI view it more of supporting an expanding use case. If rust gets popular then you'll want to support it.
- the__alchemist 16d agoI will give you an outsider's perspective on an analogy in this case. It is easy to see Candle as a ML crate to use for neural networks in rust. I have used it, and it works well. The analogy is Tensorflow 5-10 years ago. It is popular, and there are lots of material on it. You quickly learn from talking to people that due to whims, a collection of reasons, people's love of consensus that no one is recommending it; new people are not learning it. In this case, the Torch analogy is the Burn lib.
- laggui 16d agoAnd to tie this back into GPU programming, Burn's backends use CubeCL, which lets you write compute kernels in a Rust DSL using #[cube], with a JIT compiler and autotuning machinery. It targets CUDA, AMD, Metal, Vulkan and WebGPU. (disclosure: I am a contributor)
- LtdJorge 16d agoIt's very cool. If Rust had comptime, apart from macros, it would be unmatched in capabilities.
- melihelibol 16d agoIntegration with candle is already available! https://github.com/huggingface/candle/pull/3934/ https://github.com/huggingface/candle/pull/3934/