3 ms·

Today's MI300x announcement. Worth a watch before you comment about how their software sucks. https://www.youtube.com/watch?v=tfSZqjxsr0M https://www.youtube.c

Today's MI300x announcement. Worth a watch before you comment about how their software sucks. https://www.youtube.com/watch?v=tfSZqjxsr0M https://www.youtube.com/watch?v=tfSZqjxsr0M

VHRanger 3y ago
Koduri left so they have a chance to make the software not suck anymore
dhruvdh 3y ago
I'll add some highlights for those who don't have the time - Flash Attention 2 kernels, Paged Attention kernels, PyTorch, vLLM, transformers, ONNX runtime support, OpenAI triton support in v3, tensorflow, JAX, OpenXLA. Also, while this wasn't in the presentation, CUDA can be translated to HIP using automated tools. and - CuPBoPAMD is a CUDA translator that translates CUDA programs at NVVM IR level to HIP-compatible IR that can run on AMD GPUs (https://dl.acm.org/doi/pdf/10.1145/3624062.3624185 https://dl.acm.org/doi/pdf/10.1145/3624062.3624185)
- latchkey 3y ago
  > CUDA can be translated to HIP using automated tools. Not just automated, but also open source. https://github.com/ROCm-Developer-Tools/HIPIFY https://github.com/ROCm-Developer-Tools/HIPIFY
  - superkuh 3y ago
    Open source, but only covering some AMD hardware for 4 years. Don't trust the support to last.
    - slavik81 3y ago
      One of my coworkers uses a Vega 56 in their local workstation. That card is six years old. It's not officially supported, but it still works. I have no idea if the AI frameworks work on hardware that old, but the math libraries still do.
- viewtransform 3y ago
  They've gone so far to be code drop in compatible that an <import cuda> statement is still <import cuda> when running Rocm on AMD! AMD made a presentation on their AI software strategy at Microsoft Ignite two weeks ago. Worth a watch for the slides and live demo https://youtu.be/7jqZBTduhAQ?t=61 https://youtu.be/7jqZBTduhAQ?t=61
- dotnet00 3y ago
  >CuPBoPAMD They really couldn't come up with a better name than whatever that is?
incrudible 3y ago
All that buzzword bingo does not mean the software is actually robust and performs well. If you have hardware that has much better specs on paper, but only delivers maybe 20% better performance in their own benchmarks, I am still going to presume that their software is not up to par.
- daveguy 3y ago
  20% is a huge amount in terms of high end performance. For example, 20% more transistors / compute units / etc does not give 20% more performance at the top end unless your execution is excellent.
  - shaklee3 3y ago
    OP's point is that the hardware specs are 2-3x higher in many places, but all their benchmarks are 20-30% higher. The article mentions this as well. It means AMD couldn't even utilize their own hardware very well at this point.
    - zaptrem 3y ago
      Transformers are heavily memory bandwidth bound on modern hardware, and these chips only have 60% higher memory bandwidth.
      - shaklee3 3y ago
        Their own slides show a 20-30% speedup on attention tasks.
- latchkey 3y ago
  20% better performance, lower cost and most important... availability.

Today's MI300x announcement. Worth a watch before you comment about how their software sucks. https://www.youtube.com/watch?v=tfSZqjxsr0M https://www.youtube.com/watch?v=tfSZqjxsr0M

VHRanger 3y ago

Koduri left so they have a chance to make the software not suck anymore

dhruvdh 3y ago

I'll add some highlights for those who don't have the time - Flash Attention 2 kernels, Paged Attention kernels, PyTorch, vLLM, transformers, ONNX runtime support, OpenAI triton support in v3, tensorflow, JAX, OpenXLA. Also, while this wasn't in the presentation, CUDA can be translated to HIP using automated tools. and - CuPBoPAMD is a CUDA translator that translates CUDA programs at NVVM IR level to HIP-compatible IR that can run on AMD GPUs (https://dl.acm.org/doi/pdf/10.1145/3624062.3624185 https://dl.acm.org/doi/pdf/10.1145/3624062.3624185)

latchkey 3y ago
> CUDA can be translated to HIP using automated tools. Not just automated, but also open source. https://github.com/ROCm-Developer-Tools/HIPIFY https://github.com/ROCm-Developer-Tools/HIPIFY
- superkuh 3y ago
  Open source, but only covering some AMD hardware for 4 years. Don't trust the support to last.
  - slavik81 3y ago
    One of my coworkers uses a Vega 56 in their local workstation. That card is six years old. It's not officially supported, but it still works. I have no idea if the AI frameworks work on hardware that old, but the math libraries still do.
viewtransform 3y ago
They've gone so far to be code drop in compatible that an <import cuda> statement is still <import cuda> when running Rocm on AMD! AMD made a presentation on their AI software strategy at Microsoft Ignite two weeks ago. Worth a watch for the slides and live demo https://youtu.be/7jqZBTduhAQ?t=61 https://youtu.be/7jqZBTduhAQ?t=61
dotnet00 3y ago
>CuPBoPAMD They really couldn't come up with a better name than whatever that is?

incrudible 3y ago

All that buzzword bingo does not mean the software is actually robust and performs well. If you have hardware that has much better specs on paper, but only delivers maybe 20% better performance in their own benchmarks, I am still going to presume that their software is not up to par.

daveguy 3y ago
20% is a huge amount in terms of high end performance. For example, 20% more transistors / compute units / etc does not give 20% more performance at the top end unless your execution is excellent.
- shaklee3 3y ago
  OP's point is that the hardware specs are 2-3x higher in many places, but all their benchmarks are 20-30% higher. The article mentions this as well. It means AMD couldn't even utilize their own hardware very well at this point.
  - zaptrem 3y ago
    Transformers are heavily memory bandwidth bound on modern hardware, and these chips only have 60% higher memory bandwidth.
    - shaklee3 3y ago
      Their own slides show a 20-30% speedup on attention tasks.
latchkey 3y ago
20% better performance, lower cost and most important... availability.