4 ms·
Chip speed isn't as important as good software
by Vvector 1y ago
Chip speed isn't as important as good software
- moralestapia 1y agoThe software is the same, AMD is not doing its own LLMs.
- jjice 1y agoI think the software they were referring to is CUDA and the developer experience around the nvidia stack.
- moralestapia 1y ago??? Know any LLMs that are implemented in CUDA?
- wmf 1y agoUltimately all of them except Gemini.
- moralestapia 1y agoWrong. Show me one single CUDA kernel on Llama's source code. (and that's a really easy one, if one knows a bit about it)
- rnrn 1y agoremoving comment since I regret attempting to engage in this thread
- moralestapia 1y agoWrong. It is the same PyTorch whether it runs on an AMD or an NVIDIA GPU. The exact same PyTorch, actually. Are you're trying to suggest that the machine code that runs on the GPU is the one that is different? If you knew a bit more, you would know that this is the case even between different generations of GPUs of the same vendor; making that argument completely absurd.
- rnrn 1y agoremoving comment since I regret attempting to engage in this thread
- imtringued 1y agoThe average consumer uses llama.cpp. So here is your list of kernels: https://github.com/ggml-org/llama.cpp/tree/master/ggml/src/ggml-cuda https://github.com/ggml-org/llama.cpp/tree/master/ggml/src/g... And here is pretty damning evidence that you're full of shit: https://github.com/ggml-org/llama.cpp/blob/master/ggml/src/ggml-hip/CMakeLists.txt https://github.com/ggml-org/llama.cpp/blob/master/ggml/src/g... The ggml-hip backend references the ggml-cuda kernels. The "software is the same" (as in, it is CUDA) and yet AMD is still behind.
- moralestapia 1y ago@tomhow personal attack