3 ms·
PyTorch is only part of it. There is still a huge amount of CUDA that isn’t just wrapped by PyTorch and isn’t easily portable.
by qeternity 11mo ago
PyTorch is only part of it. There is still a huge amount of CUDA that isn’t just wrapped by PyTorch and isn’t easily portable.
- svara 11mo ago... but not in deep learning or am I missing something important here?
- qeternity 11mo agoYes, absolutely in deep learning. Custom fused CUDA kernels everywhere.
- Scene_Cast2 11mo agoYep. MoE, FlashAttention, or sparse retrieval architectures for example.