4 ms·

> It's the only credible competition NVidia has This is wrong, both AMD and Intel (through Habana) have GPUs comparable to H100s in performance.

by ipsum2 3y ago

nl 3y ago
Yes, but they don't have the custom kernels that CUDA has. TPUs do have some!
- rurban 3y ago
  They have Vulcan, which is cross-compatible. And AMD has ROCm. pytorch is standard and pytorch has ROCm support. And the Google TPU v5 also has pytorch support. We do have a couple of H100's, but I'd love to replace them with AMD's
  - nl 3y ago
    Vulcan is a driver-level API. It competes with DirectX and OpenGL. CUDA is a language you write kernels. It competes with OpenAI's Triton language. Here's what CUDA looks like: https://github.com/tspeterkim/flash-attention-minimal/blob/main/flash.cu https://github.com/tspeterkim/flash-attention-minimal/blob/m... This is what Triton looks like: https://triton-lang.org/main/getting-started/tutorials/06-fused-attention.html#sphx-glr-download-getting-started-tutorials-06-fused-attention-py https://triton-lang.org/main/getting-started/tutorials/06-fu... By contrast Vulcan looks like this: https://github.com/KhronosGroup/Vulkan-Samples/blob/main/samples/api/hello_triangle/hello_triangle.cpp https://github.com/KhronosGroup/Vulkan-Samples/blob/main/sam... (It's true to some extent that maybe you could use Vulcan shaders to write deep learning kernels, maybe? I'm not aware of anyone doing it though)
  - Kelteseth 3y ago
    If AMD fixes or open sources their proprietary firmware blob[0]. Geohot streamed all weekend on Twitch, reverse engineering the AMD firmware. It was quite entertaining learning about how that low level hardware firmware works[1] and his rants about AMD of course. [0] https://www.phoronix.com/news/Tinybox-Radeon-Again-UMR https://www.phoronix.com/news/Tinybox-Radeon-Again-UMR [1] https://www.twitch.tv/georgehotz https://www.twitch.tv/georgehotz
    - KeplerBoy 3y ago
      Geohot is wrangling with unsupported consumer hardware. The datacenter stuff is on a different architecture and driver stack. The number one supercomputer on the top500 list (frontier at ORNL) is based on AMD GPUs and AMD is probably more invested in supporting that.
      - QuadmasterXLII 3y ago
        I mean, that's kinda nvidia's whole shtick: anyone can play around synthesizing cat pictures on their gaming GPU and if they make a breakthrough, the same software will transfer to X million dollar supercomputers.
      - kkielhofner 3y ago
        I work with Frontier and ORNL/OLCF. They have had and continue to have issues with AMD/ROCm but yes, they do of course get excellent support from AMD. The entire team at OLCF is incredible as well (obviously) and they do amazing work. Frontier certainly has some unique quirks but the documentation is online[0] and most of these quirks are inherent to the kinds of fundamental issues you'll see on any system in the space (SLURM, etc). However, most of the issues are fundamentally ROCm and you'll run into them on any MIxxx anywhere. I run into them frequently with supported and unsupported consumer gear all the way up. [0] - https://docs.olcf.ornl.gov/systems/frontier_user_guide.html https://docs.olcf.ornl.gov/systems/frontier_user_guide.html
    - immibis 3y ago
      Subscriber only videos, so nobody can confirm that he did that, nor archive whatever valuable information he released. At least not without paying some money in the next 7-14 days before they're deleted.
      - ionelaipatioaei 3y ago
        https://www.youtube.com/@geohotarchive https://www.youtube.com/@geohotarchive
    - 3y ago
- ipsum2 3y ago
  Huh? The reason why they're competitive with Nvidia is because they have custom kernels for all the popular models.
kettleballroll 3y ago
But they're far behind in adoption in the AI space, while TPUs have both adoption (inside Google and on top) and a very strong software offering (Jax and TF)
HarHarVeryFunny 3y ago
There's also Amazon's AWS "Trainium" chips, which is what Anthropic will be using going forward. If you're talking about training LLMs, involving 10's of thousands of processors, then the specifics of one processor vs another isn't the most important thing - it's the overall architecture and infrastructure in place to manage it.

> It's the only credible competition NVidia has This is wrong, both AMD and Intel (through Habana) have GPUs comparable to H100s in performance.

by ipsum2 3y ago

> It's the only credible competition NVidia has This is wrong, both AMD and Intel (through Habana) have GPUs comparable to H100s in performance.

nl 3y ago

Yes, but they don't have the custom kernels that CUDA has. TPUs do have some!

rurban 3y ago
They have Vulcan, which is cross-compatible. And AMD has ROCm. pytorch is standard and pytorch has ROCm support. And the Google TPU v5 also has pytorch support. We do have a couple of H100's, but I'd love to replace them with AMD's
- nl 3y ago
  Vulcan is a driver-level API. It competes with DirectX and OpenGL. CUDA is a language you write kernels. It competes with OpenAI's Triton language. Here's what CUDA looks like: https://github.com/tspeterkim/flash-attention-minimal/blob/main/flash.cu https://github.com/tspeterkim/flash-attention-minimal/blob/m... This is what Triton looks like: https://triton-lang.org/main/getting-started/tutorials/06-fused-attention.html#sphx-glr-download-getting-started-tutorials-06-fused-attention-py https://triton-lang.org/main/getting-started/tutorials/06-fu... By contrast Vulcan looks like this: https://github.com/KhronosGroup/Vulkan-Samples/blob/main/samples/api/hello_triangle/hello_triangle.cpp https://github.com/KhronosGroup/Vulkan-Samples/blob/main/sam... (It's true to some extent that maybe you could use Vulcan shaders to write deep learning kernels, maybe? I'm not aware of anyone doing it though)
- Kelteseth 3y ago
  If AMD fixes or open sources their proprietary firmware blob[0]. Geohot streamed all weekend on Twitch, reverse engineering the AMD firmware. It was quite entertaining learning about how that low level hardware firmware works[1] and his rants about AMD of course. [0] https://www.phoronix.com/news/Tinybox-Radeon-Again-UMR https://www.phoronix.com/news/Tinybox-Radeon-Again-UMR [1] https://www.twitch.tv/georgehotz https://www.twitch.tv/georgehotz
  - KeplerBoy 3y ago
    Geohot is wrangling with unsupported consumer hardware. The datacenter stuff is on a different architecture and driver stack. The number one supercomputer on the top500 list (frontier at ORNL) is based on AMD GPUs and AMD is probably more invested in supporting that.
    - QuadmasterXLII 3y ago
      I mean, that's kinda nvidia's whole shtick: anyone can play around synthesizing cat pictures on their gaming GPU and if they make a breakthrough, the same software will transfer to X million dollar supercomputers.
    - kkielhofner 3y ago
      I work with Frontier and ORNL/OLCF. They have had and continue to have issues with AMD/ROCm but yes, they do of course get excellent support from AMD. The entire team at OLCF is incredible as well (obviously) and they do amazing work. Frontier certainly has some unique quirks but the documentation is online[0] and most of these quirks are inherent to the kinds of fundamental issues you'll see on any system in the space (SLURM, etc). However, most of the issues are fundamentally ROCm and you'll run into them on any MIxxx anywhere. I run into them frequently with supported and unsupported consumer gear all the way up. [0] - https://docs.olcf.ornl.gov/systems/frontier_user_guide.html https://docs.olcf.ornl.gov/systems/frontier_user_guide.html
  - immibis 3y ago
    Subscriber only videos, so nobody can confirm that he did that, nor archive whatever valuable information he released. At least not without paying some money in the next 7-14 days before they're deleted.
    - ionelaipatioaei 3y ago
      https://www.youtube.com/@geohotarchive https://www.youtube.com/@geohotarchive
  - 3y ago
ipsum2 3y ago
Huh? The reason why they're competitive with Nvidia is because they have custom kernels for all the popular models.

kettleballroll 3y ago

But they're far behind in adoption in the AI space, while TPUs have both adoption (inside Google and on top) and a very strong software offering (Jax and TF)

HarHarVeryFunny 3y ago

There's also Amazon's AWS "Trainium" chips, which is what Anthropic will be using going forward. If you're talking about training LLMs, involving 10's of thousands of processors, then the specifics of one processor vs another isn't the most important thing - it's the overall architecture and infrastructure in place to manage it.