3 ms·
Sure, but any language can provide that kind of pre-built block in a library. (Indeed, I suspect the library we're discussing has this operation, though I didn'
by sanderjd 3y ago
Sure, but any language can provide that kind of pre-built block in a library. (Indeed, I suspect the library we're discussing has this operation, though I didn't check.) But if you want to implement a new operation that performs well, not every language can do that. In python, people tend to drop down to C or C++ or increasingly Rust to do that. But it seems (to me) like it would be nice to be able to use a single language for both things.
- brrrrrm 3y agoAt least with the more common modern ML (LLMs etc) operations, no one drops down to Rust. It's really only good for IO/CPU work, but that's rarely a spot for new operations, just for data loading and high-latency network stuff. People typically use Triton or CUDA (usually the C++ flavor) to implement operations. It'd be cool if Rust had a CUDA dialect.
- sanderjd 3y agoThe point is that someone dropped down to implement those operations, at some point. It seems like you can target CUDA with Rust, because this project seems to be doing that?
- brrrrrm 3y agohttps://github.com/huggingface/candle/blob/main/candle-kernels/src/conv.cu https://github.com/huggingface/candle/blob/main/candle-kerne... That’s the C++ variant of cuda, not Rust. They’re just binding it as one might in Python. At which point, what’s the value add?
- sanderjd 3y agoAha! Thanks for digging that up!