3 ms·
If hope AMD can produce a chip that matches H100 in training workloads.
by kombine 1y ago
If hope AMD can produce a chip that matches H100 in training workloads.
- moralestapia 1y agoYou mean a slower chip? Their MI300s already beat them, 400s coming soon.
- Vvector 1y agoChip speed isn't as important as good software
- moralestapia 1y agoThe software is the same, AMD is not doing its own LLMs.
- jjice 1y agoI think the software they were referring to is CUDA and the developer experience around the nvidia stack.
- moralestapia 1y ago??? Know any LLMs that are implemented in CUDA?
- wmf 1y agoUltimately all of them except Gemini.
- moralestapia 1y agoWrong. Show me one single CUDA kernel on Llama's source code. (and that's a really easy one, if one knows a bit about it)
- rnrn 1y agoremoving comment since I regret attempting to engage in this thread
- moralestapia 1y agoWrong. It is the same PyTorch whether it runs on an AMD or an NVIDIA GPU. The exact same PyTorch, actually. Are you're trying to suggest that the machine code that runs on the GPU is the one that is different? If you knew a bit more, you would know that this is the case even between different generations of GPUs of the same vendor; making that argument completely absurd.
- rnrn 1y agoremoving comment since I regret attempting to engage in this thread
- imtringued 1y agoThe average consumer uses llama.cpp. So here is your list of kernels: https://github.com/ggml-org/llama.cpp/tree/master/ggml/src/ggml-cuda https://github.com/ggml-org/llama.cpp/tree/master/ggml/src/g... And here is pretty damning evidence that you're full of shit: https://github.com/ggml-org/llama.cpp/blob/master/ggml/src/ggml-hip/CMakeLists.txt https://github.com/ggml-org/llama.cpp/blob/master/ggml/src/g... The ggml-hip backend references the ggml-cuda kernels. The "software is the same" (as in, it is CUDA) and yet AMD is still behind.
- moralestapia 1y ago@tomhow personal attack
- lhl 1y agoLast year I had issues using MI300X for training, and when it did work, was about 20-30% slower than H100, but I'm doing some OpenRLHF (transformers/DeepSpeed-based) DPO training atm w/ latest ROCm and PyTorch and it seems to be doing OK, roughly matching GPU-hour perf w/ an H200 for small ~12h runs. Note: previous testing I did was on a single (8x) MI300X node, currently I'm doing testing on just a single MI300X GPU, so not quite apples-to-apples, multi-GPU/multi-node training is still a question mark, just a single data point.
- fooker 1y agoIt gets even more jarring that H100 is about three years old now.