5 ms·
I think the software they were referring to is CUDA and the developer experience around the nvidia stack.
by jjice 1y ago
I think the software they were referring to is CUDA and the developer experience around the nvidia stack.
- moralestapia 1y ago??? Know any LLMs that are implemented in CUDA?
- wmf 1y agoUltimately all of them except Gemini.
- moralestapia 1y agoWrong. Show me one single CUDA kernel on Llama's source code. (and that's a really easy one, if one knows a bit about it)
- rnrn 1y agoremoving comment since I regret attempting to engage in this thread
- moralestapia 1y agoWrong. It is the same PyTorch whether it runs on an AMD or an NVIDIA GPU. The exact same PyTorch, actually. Are you're trying to suggest that the machine code that runs on the GPU is the one that is different? If you knew a bit more, you would know that this is the case even between different generations of GPUs of the same vendor; making that argument completely absurd.
- rnrn 1y agoremoving comment since I regret attempting to engage in this thread
- imtringued 1y agoThe average consumer uses llama.cpp. So here is your list of kernels: https://github.com/ggml-org/llama.cpp/tree/master/ggml/src/ggml-cuda https://github.com/ggml-org/llama.cpp/tree/master/ggml/src/g... And here is pretty damning evidence that you're full of shit: https://github.com/ggml-org/llama.cpp/blob/master/ggml/src/ggml-hip/CMakeLists.txt https://github.com/ggml-org/llama.cpp/blob/master/ggml/src/g... The ggml-hip backend references the ggml-cuda kernels. The "software is the same" (as in, it is CUDA) and yet AMD is still behind.
- moralestapia 1y ago@tomhow personal attack