4 ms·
At least with the more common modern ML (LLMs etc) operations, no one drops down to Rust. It's really only good for IO/CPU work, but that's rarely a spot for n
by brrrrrm 3y ago
At least with the more common modern ML (LLMs etc) operations, no one drops down to Rust. It's really only good for IO/CPU work, but that's rarely a spot for new operations, just for data loading and high-latency network stuff.
People typically use Triton or CUDA (usually the C++ flavor) to implement operations. It'd be cool if Rust had a CUDA dialect.
- sanderjd 3y agoThe point is that someone dropped down to implement those operations, at some point. It seems like you can target CUDA with Rust, because this project seems to be doing that?
- brrrrrm 3y agohttps://github.com/huggingface/candle/blob/main/candle-kernels/src/conv.cu https://github.com/huggingface/candle/blob/main/candle-kerne... That’s the C++ variant of cuda, not Rust. They’re just binding it as one might in Python. At which point, what’s the value add?
- sanderjd 3y agoAha! Thanks for digging that up!