3 ms·
> "Basically nobody writes CUDA," Quite a thing to say as head of an AI hardware startup. Most of the machine learning libraries we use/test in production (an
by brucethemoose2 3y ago
> "Basically nobody writes CUDA,"
Quite a thing to say as head of an AI hardware startup.
Most of the machine learning libraries we use/test in production (and that I use personally on my desktop) have hand written, extensively optimized CUDA kernels. It would be problem #1 switching to Tenstorrent hardware.
Not that I like that one bit, or that I really understand the optimization of these libraries. I do see triton code as well, but it still seems to be the suboptimal path.
- qwertox 3y agoBut what kind of CUDA kernels have you written? Kernels for AI training and inferencing? Because I think that's what he's trying to say. I doubt he's saying that in the context of CFD or other similar ones.
- brucethemoose2 3y agoIf you are implying we are at the mercy of a few CUDA experts because we can't efficiently write it ourselves, you would be correct. Triton does seem way more approachable as something I'd like to pick up.
- pjmlp 3y agoHence why NVidia is also investing into GPUs JITs for Python. One advantage of CUDA, that Khronos/AMD/Intel realized too late, was the polyglot support that NVidia started to push and support since CUDA 3.0.
- imtringued 3y agoI hear you say this over and over again, but honestly AMDs OpenCL implementations used to be unusable garbage.
- pjmlp 3y agoThat in top of everything else, with OpenCL stuck in 1990's tooling.
- PeterisP 3y agoBut why do I need to write CUDA kernels? The CUDA kernels I need already exist, and I don't particularly care if writing them was so horrible that it required scratching your eyes out and sacrificing kittens to demons - I would care about the (re)writing effort only if I was considering other hardware that doesn't yet have the support that CUDA has as then I'd actually have to do that myself.