3 ms·

I'll add some highlights for those who don't have the time - Flash Attention 2 kernels, Paged Attention kernels, PyTorch, vLLM, transformers, ONNX runtime suppo

I'll add some highlights for those who don't have the time - Flash Attention 2 kernels, Paged Attention kernels, PyTorch, vLLM, transformers, ONNX runtime support, OpenAI triton support in v3, tensorflow, JAX, OpenXLA. Also, while this wasn't in the presentation, CUDA can be translated to HIP using automated tools. and - CuPBoPAMD is a CUDA translator that translates CUDA programs at NVVM IR level to HIP-compatible IR that can run on AMD GPUs (https://dl.acm.org/doi/pdf/10.1145/3624062.3624185 https://dl.acm.org/doi/pdf/10.1145/3624062.3624185)

latchkey 3y ago
> CUDA can be translated to HIP using automated tools. Not just automated, but also open source. https://github.com/ROCm-Developer-Tools/HIPIFY https://github.com/ROCm-Developer-Tools/HIPIFY
- superkuh 3y ago
  Open source, but only covering some AMD hardware for 4 years. Don't trust the support to last.
  - slavik81 3y ago
    One of my coworkers uses a Vega 56 in their local workstation. That card is six years old. It's not officially supported, but it still works. I have no idea if the AI frameworks work on hardware that old, but the math libraries still do.
viewtransform 3y ago
They've gone so far to be code drop in compatible that an <import cuda> statement is still <import cuda> when running Rocm on AMD! AMD made a presentation on their AI software strategy at Microsoft Ignite two weeks ago. Worth a watch for the slides and live demo https://youtu.be/7jqZBTduhAQ?t=61 https://youtu.be/7jqZBTduhAQ?t=61
dotnet00 3y ago
>CuPBoPAMD They really couldn't come up with a better name than whatever that is?