4 ms·

Cuda is a proprietary Nvidia framework. You don't need cuda support to train deep learning models on TensorFlow. ROCm also works - and is built by AMD and is op

Cuda is a proprietary Nvidia framework. You don't need cuda support to train deep learning models on TensorFlow. ROCm also works - and is built by AMD and is open-source.

missosoup 7y ago
Tensorflow is only a small subset of the massive engineering investment that has been done into cuda by various orgs and open source projects. We have GPU databases, GPU accelerated analytics packages, GPU accelerated DSP, etc. All with hand-written cuda kernels. Unless AMD can provide a toolchain which takes cuda code and generates whatever it takes to run it with performance parity to Nvidia cards, it'll never take off. ROCm is a decade too late to simply coexist with cuda and battle for market/mind share. Unless they can offer a massive perf/cost advantage, there is no incentive for anyone to invest in porting their code.
- dekken_ 7y ago
  > Unless AMD can provide a toolchain which takes cuda code and generates whatever it takes to run it with performance parity to Nvidia cards, it'll never take off. They have this. It's called HIP.
  - missosoup 7y ago
    https://rocm-documentation.readthedocs.io/en/latest/Programming_Guides/HIP-porting-guide.html https://rocm-documentation.readthedocs.io/en/latest/Programm... "HIP is designed to ease the porting of existing CUDA code into the HIP environment." Not even close an drop-in swappable alternative.
    - tonypace 7y ago
      Of course not, and some things would never be ported. But if they had a massive cost advantage for a year or two in data centers, a lot of things would get ported, quickly.
      - deleted 7y ago
        [deleted]
  - pjmlp 7y ago
    Does HIP support all the programming languages that are able to target CUDA, like Julia, Numba, .NET, Java, Haskell, Fortran, C++? Khronos did a big mistake to keep pushing C as the only compute language for so long, and it remains to be seen how much adoption SPIR-V is able to actually take from PTX. So unless HIP actually matches PTX, it is a non-starter.
    - xiphias2 7y ago
      In theory HIP is a modified LLVM, but as far as I know Julia uses a modified LLVM as well, so unless they are upstreamed, we can't use them together :( I guess I'm staying with NVIDIA with developer laptops for a long time, and maybe using AMD for ultrabook.
      - pjmlp 7y ago
        Except that some of the languages that target CUDA do so via PTX, not by migrating their compiler backends into LLVM. So if HIP will only work with LLVM, that is also a non-starter.
dnautics 7y ago
Have you tried installing rocm? Last I checked two months ago you had to run a Ubuntu 16.04 container on Ubuntu 18.04 because the amd drivers were only compiled for bionic and the rocm software was only compiled for xenial.
- melbourne_mat 7y ago
  I've been running rocm directly on Ubuntu 19.04 for a while now. I didn't have to build from source either. It's odd that they haven't updated their docs about this yet.
  - dnautics 7y ago
    Cool, thanks!! I'll take a look at it
- danieldk 7y ago
  We have installed ROCm on vanilla Ubuntu 18.04. Though I use the libraries through Nix with the nixos-rocm repo: https://github.com/nixos-rocm/nixos-rocm https://github.com/nixos-rocm/nixos-rocm The last time we purchased GPUs, we bought two Radeon VIIs besides RTX 5000s. For training transformers in Tensorflow and PyTorch, they were a steal, because the training is only marginally slower than the RTXes at a fraction of the price. For LSTMs, we have found performance regressions compared to NVIDIA, but it's largely because it was hard to hit the optimized MIOpen code paths.

Cuda is a proprietary Nvidia framework. You don't need cuda support to train deep learning models on TensorFlow. ROCm also works - and is built by AMD and is open-source.

missosoup 7y ago

Tensorflow is only a small subset of the massive engineering investment that has been done into cuda by various orgs and open source projects. We have GPU databases, GPU accelerated analytics packages, GPU accelerated DSP, etc. All with hand-written cuda kernels. Unless AMD can provide a toolchain which takes cuda code and generates whatever it takes to run it with performance parity to Nvidia cards, it'll never take off. ROCm is a decade too late to simply coexist with cuda and battle for market/mind share. Unless they can offer a massive perf/cost advantage, there is no incentive for anyone to invest in porting their code.

dekken_ 7y ago
> Unless AMD can provide a toolchain which takes cuda code and generates whatever it takes to run it with performance parity to Nvidia cards, it'll never take off. They have this. It's called HIP.
- missosoup 7y ago
  https://rocm-documentation.readthedocs.io/en/latest/Programming_Guides/HIP-porting-guide.html https://rocm-documentation.readthedocs.io/en/latest/Programm... "HIP is designed to ease the porting of existing CUDA code into the HIP environment." Not even close an drop-in swappable alternative.
  - tonypace 7y ago
    Of course not, and some things would never be ported. But if they had a massive cost advantage for a year or two in data centers, a lot of things would get ported, quickly.
    - deleted 7y ago
      [deleted]
- pjmlp 7y ago
  Does HIP support all the programming languages that are able to target CUDA, like Julia, Numba, .NET, Java, Haskell, Fortran, C++? Khronos did a big mistake to keep pushing C as the only compute language for so long, and it remains to be seen how much adoption SPIR-V is able to actually take from PTX. So unless HIP actually matches PTX, it is a non-starter.
  - xiphias2 7y ago
    In theory HIP is a modified LLVM, but as far as I know Julia uses a modified LLVM as well, so unless they are upstreamed, we can't use them together :( I guess I'm staying with NVIDIA with developer laptops for a long time, and maybe using AMD for ultrabook.
    - pjmlp 7y ago
      Except that some of the languages that target CUDA do so via PTX, not by migrating their compiler backends into LLVM. So if HIP will only work with LLVM, that is also a non-starter.

dnautics 7y ago

Have you tried installing rocm? Last I checked two months ago you had to run a Ubuntu 16.04 container on Ubuntu 18.04 because the amd drivers were only compiled for bionic and the rocm software was only compiled for xenial.

melbourne_mat 7y ago
I've been running rocm directly on Ubuntu 19.04 for a while now. I didn't have to build from source either. It's odd that they haven't updated their docs about this yet.
- dnautics 7y ago
  Cool, thanks!! I'll take a look at it
danieldk 7y ago
We have installed ROCm on vanilla Ubuntu 18.04. Though I use the libraries through Nix with the nixos-rocm repo: https://github.com/nixos-rocm/nixos-rocm https://github.com/nixos-rocm/nixos-rocm The last time we purchased GPUs, we bought two Radeon VIIs besides RTX 5000s. For training transformers in Tensorflow and PyTorch, they were a steal, because the training is only marginally slower than the RTXes at a fraction of the price. For LSTMs, we have found performance regressions compared to NVIDIA, but it's largely because it was hard to hit the optimized MIOpen code paths.