8 ms·

Cuda.jl v3.3: union types, debug info, graph APIs

156 points by ViralBShah 5y ago

aardvarkr 5y ago
Who knew that Julia could do CUDA work too? Every day I grow more and more impressed with the language.
- version_five 5y ago
  I last experimented with CUDA.jl a year ago, and it was very useable then. This is a good reminder to re-evaluate the Julia deep learning ecosystem. If I were working for myself I would definitely try to do more with Julia (for machine learning). Realistically, python has such an established base that it will take some time to get orgs that are already all in on python to come over.
  - dnautics 5y ago
    I think it's not dumb to target Greenfield users: just installing python gpu wheels is often difficult enough that several companies exist (indirectly) because it's so difficult to do right (e.g. selling a gpu PC with that stuff preinstalled)
  - queuebert 5y ago
    I just finished setting up a new machine to run some Kaggle stuff. Both Tensorflow and PyTorch had issues with CUDA versions and dependencies that weren't immediately fixed with a clean virtualenv, while both Knet.jl and Flux.jl installed flawlessly.
    - wdroz 5y ago
      For Pytorch and Tensorflow, you can use conda to install them with the right CUDA and cudnn versions.
      - kwertzzz 5y ago
        For Pytorch, I had no issues with conda. But with Tensorflow from conda, the training process just hangs (consuming 100% of CPU but no GPU resources, despite my GPUs are recognized). I got more luck with installing Tensorflow with pip. Given the fact that Tensorflow documentation does not mention conda, I wondering how well this is supported.
        wdroz 5y ago
        You install cudatoolkit from conda then tensorflow with pip.
- eigenspace 5y ago
  It also happens to be one of the most easy and reliable ways I know of to install CUDA on your machine. Everything is handled through the artifact system so you don't have to mess with downloading it yourself and making sure you have the right versions and such. (Before someone complains, you can also opt out of this and direct the library to a version you installed yourself)
  - krastanov 5y ago
    Could you elaborate a bit on this? I know that a gigantic bunch of libraries are involved that is usually terrible to install but Julia does it for you in the equivalent of a python virtual-env. However, aren't there also Linux kernel components that are necessary? How are those installed?
    - ChrisRackauckas 5y ago
      CUDA.jl installs all of the CUDA drivers and associated libraries like cudnn for you if you don't have them. Those are all vendered via the Yggdrasil system so that users don't have to deal with it.
      - krastanov 5y ago
        CUDA.jl does not install the actual kernel driver, right? I do not really see how it can do that and the sibling comment does confirm that the kernel driver is not managed by Julia.
        kwertzzz 5y ago
        Yes, you would still need to the NVIDIA kernel driver (preferably the most current one). Desktop users typically have it already installed. But the main difficulty in my opinion is to install CUDA (with CuDNN,...). Even the TensorFlow documentation [0] is outdated in this regards as it covers only Ubuntu 18.04. The installation process of CUDA.jl is really quite good and reliable. Per default it downloads it own version of CUDA and CuDNN, or you can use a system-wide CUDA installation by setting some environment variables [1]. [0] https://www.tensorflow.org/install/gpu https://www.tensorflow.org/install/gpu [1] https://cuda.juliagpu.org/stable/installation/overview/ https://cuda.juliagpu.org/stable/installation/overview/
deleted 5y ago
[deleted]
up6w6 5y ago
A fun fact is that the GPUCompiler, which compiles the code to run in GPU's, is the current way to generate binaries without hiding the whole ~200mb of julia runtime in the binary. https://github.com/JuliaGPU/GPUCompiler.jl/ https://github.com/JuliaGPU/GPUCompiler.jl/ https://github.com/tshort/StaticCompiler.jl/ https://github.com/tshort/StaticCompiler.jl/
- snicker7 5y ago
  AKA Julia does GPU better than CPU.
Karrot_Kream 5y ago
Really looking forward to Turing.jl gaining CUDA support
pabs3 5y ago
Are there similar things for other types of GPUs? Edit: the site has one project per GPU type, shame there isn't one interface that works with every GPU type instead.
- krastanov 5y ago
  These libraries provide the same API, so from a user perspective, as long as you do not need low-level access, it does not matter what your GPU is. However, the low-level library for AMD GPUs is more of an alpha quality in Julia.
- eigenspace 5y ago
  https://github.com/JuliaGPU/AMDGPU.jl https://github.com/JuliaGPU/AMDGPU.jl https://github.com/JuliaGPU/oneAPI.jl https://github.com/JuliaGPU/oneAPI.jl These are both less mature than CUDA.jl, but are in active development. > Edit: the site has one project per GPU type, shame there isn't one interface that works with every GPU type instead. That would be https://juliagpu.github.io/KernelAbstractions.jl https://juliagpu.github.io/KernelAbstractions.jl
  - pabs3 5y ago
    https://github.com/JuliaGPU/KernelAbstractions.jl https://github.com/JuliaGPU/KernelAbstractions.jl
- jpsamaroo 5y ago
  For kernel programming, https://github.com/JuliaGPU/KernelAbstractions.jl https://github.com/JuliaGPU/KernelAbstractions.jl (shortened to KA) is what the JuliaGPU team has been developing as a unified programming interface for GPUs of any flavor. It's not significantly different from the (basically identical) interfaces exposed by CUDA.jl and AMDGPU.jl, so it's easy to transition to. I think the event system in KA is also far superior to CUDA's native synchronization system, since it allows one to easily express graphs of dependencies between kernels and data transfers.
xvilka 5y ago
I wish more attention would be towards open source alternatives for CUDA, such as AMD's ROCm[1][2] and Julia framework using it - AMDGPU.jl[3]. It's sad to see so many people praise NVIDIA which is the enemy of open source, openly hostile to anything except their oversized proprietary binary blobs. [1] https://rocmdocs.amd.com/en/latest/index.html https://rocmdocs.amd.com/en/latest/index.html [2] https://github.com/RadeonOpenCompute/ROCm https://github.com/RadeonOpenCompute/ROCm [3] https://github.com/JuliaGPU/AMDGPU.jl https://github.com/JuliaGPU/AMDGPU.jl
- mixedCase 5y ago
  Maybe if AMD starts caring about ROCm, users might. To this day Navi and newer cards are unsupported.
  - kwertzzz 5y ago
    This is really the main problem. As far as I know, ROCm requires a quite expensive GPU for data centers (if you want to have a current GPU) which makes it quite difficult to build a community around ROCm.
- krapht 5y ago
  Most people hack on this stuff for work, and time is money. OpenCL is just a lot less productive than CUDA for most tasks. The NVidia price premium isn't big enough to make people switch over.
  - andi999 5y ago
    Agree, been doing some cuda as part of my job since 2009, and it works very smooth, you could write non trivial programs after less than one week of learning (if you know C before). When opencl got a little hype, I had a look, I didn't manage to run a simple example, and I was asking myself: do I really want to have to use all this boilerplate code? Also at least a while back there is no good fft outside of cuda, maybe that changed though.
- pjmlp 5y ago

156 points by ViralBShah 5y ago

aardvarkr 5y ago

Who knew that Julia could do CUDA work too? Every day I grow more and more impressed with the language.

version_five 5y ago
I last experimented with CUDA.jl a year ago, and it was very useable then. This is a good reminder to re-evaluate the Julia deep learning ecosystem. If I were working for myself I would definitely try to do more with Julia (for machine learning). Realistically, python has such an established base that it will take some time to get orgs that are already all in on python to come over.
- dnautics 5y ago
  I think it's not dumb to target Greenfield users: just installing python gpu wheels is often difficult enough that several companies exist (indirectly) because it's so difficult to do right (e.g. selling a gpu PC with that stuff preinstalled)
- queuebert 5y ago
  I just finished setting up a new machine to run some Kaggle stuff. Both Tensorflow and PyTorch had issues with CUDA versions and dependencies that weren't immediately fixed with a clean virtualenv, while both Knet.jl and Flux.jl installed flawlessly.
  - wdroz 5y ago
    For Pytorch and Tensorflow, you can use conda to install them with the right CUDA and cudnn versions.
    - kwertzzz 5y ago
      For Pytorch, I had no issues with conda. But with Tensorflow from conda, the training process just hangs (consuming 100% of CPU but no GPU resources, despite my GPUs are recognized). I got more luck with installing Tensorflow with pip. Given the fact that Tensorflow documentation does not mention conda, I wondering how well this is supported.
      - wdroz 5y ago
        You install cudatoolkit from conda then tensorflow with pip.
eigenspace 5y ago
It also happens to be one of the most easy and reliable ways I know of to install CUDA on your machine. Everything is handled through the artifact system so you don't have to mess with downloading it yourself and making sure you have the right versions and such. (Before someone complains, you can also opt out of this and direct the library to a version you installed yourself)
- krastanov 5y ago
  Could you elaborate a bit on this? I know that a gigantic bunch of libraries are involved that is usually terrible to install but Julia does it for you in the equivalent of a python virtual-env. However, aren't there also Linux kernel components that are necessary? How are those installed?
  - ChrisRackauckas 5y ago
    CUDA.jl installs all of the CUDA drivers and associated libraries like cudnn for you if you don't have them. Those are all vendered via the Yggdrasil system so that users don't have to deal with it.
    - krastanov 5y ago
      CUDA.jl does not install the actual kernel driver, right? I do not really see how it can do that and the sibling comment does confirm that the kernel driver is not managed by Julia.
      - kwertzzz 5y ago
        Yes, you would still need to the NVIDIA kernel driver (preferably the most current one). Desktop users typically have it already installed. But the main difficulty in my opinion is to install CUDA (with CuDNN,...). Even the TensorFlow documentation [0] is outdated in this regards as it covers only Ubuntu 18.04. The installation process of CUDA.jl is really quite good and reliable. Per default it downloads it own version of CUDA and CuDNN, or you can use a system-wide CUDA installation by setting some environment variables [1]. [0] https://www.tensorflow.org/install/gpu https://www.tensorflow.org/install/gpu [1] https://cuda.juliagpu.org/stable/installation/overview/ https://cuda.juliagpu.org/stable/installation/overview/

deleted 5y ago

[deleted]

up6w6 5y ago

A fun fact is that the GPUCompiler, which compiles the code to run in GPU's, is the current way to generate binaries without hiding the whole ~200mb of julia runtime in the binary. https://github.com/JuliaGPU/GPUCompiler.jl/ https://github.com/JuliaGPU/GPUCompiler.jl/ https://github.com/tshort/StaticCompiler.jl/ https://github.com/tshort/StaticCompiler.jl/

snicker7 5y ago
AKA Julia does GPU better than CPU.

Karrot_Kream 5y ago

Really looking forward to Turing.jl gaining CUDA support

pabs3 5y ago

Are there similar things for other types of GPUs? Edit: the site has one project per GPU type, shame there isn't one interface that works with every GPU type instead.

krastanov 5y ago
These libraries provide the same API, so from a user perspective, as long as you do not need low-level access, it does not matter what your GPU is. However, the low-level library for AMD GPUs is more of an alpha quality in Julia.
eigenspace 5y ago
https://github.com/JuliaGPU/AMDGPU.jl https://github.com/JuliaGPU/AMDGPU.jl https://github.com/JuliaGPU/oneAPI.jl https://github.com/JuliaGPU/oneAPI.jl These are both less mature than CUDA.jl, but are in active development. > Edit: the site has one project per GPU type, shame there isn't one interface that works with every GPU type instead. That would be https://juliagpu.github.io/KernelAbstractions.jl https://juliagpu.github.io/KernelAbstractions.jl
- pabs3 5y ago
  https://github.com/JuliaGPU/KernelAbstractions.jl https://github.com/JuliaGPU/KernelAbstractions.jl
jpsamaroo 5y ago
For kernel programming, https://github.com/JuliaGPU/KernelAbstractions.jl https://github.com/JuliaGPU/KernelAbstractions.jl (shortened to KA) is what the JuliaGPU team has been developing as a unified programming interface for GPUs of any flavor. It's not significantly different from the (basically identical) interfaces exposed by CUDA.jl and AMDGPU.jl, so it's easy to transition to. I think the event system in KA is also far superior to CUDA's native synchronization system, since it allows one to easily express graphs of dependencies between kernels and data transfers.

xvilka 5y ago

I wish more attention would be towards open source alternatives for CUDA, such as AMD's ROCm[1][2] and Julia framework using it - AMDGPU.jl[3]. It's sad to see so many people praise NVIDIA which is the enemy of open source, openly hostile to anything except their oversized proprietary binary blobs. [1] https://rocmdocs.amd.com/en/latest/index.html https://rocmdocs.amd.com/en/latest/index.html [2] https://github.com/RadeonOpenCompute/ROCm https://github.com/RadeonOpenCompute/ROCm [3] https://github.com/JuliaGPU/AMDGPU.jl https://github.com/JuliaGPU/AMDGPU.jl

mixedCase 5y ago
Maybe if AMD starts caring about ROCm, users might. To this day Navi and newer cards are unsupported.
- kwertzzz 5y ago
  This is really the main problem. As far as I know, ROCm requires a quite expensive GPU for data centers (if you want to have a current GPU) which makes it quite difficult to build a community around ROCm.
krapht 5y ago
Most people hack on this stuff for work, and time is money. OpenCL is just a lot less productive than CUDA for most tasks. The NVidia price premium isn't big enough to make people switch over.
- andi999 5y ago
  Agree, been doing some cuda as part of my job since 2009, and it works very smooth, you could write non trivial programs after less than one week of learning (if you know C before). When opencl got a little hype, I had a look, I didn't manage to run a simple example, and I was asking myself: do I really want to have to use all this boilerplate code? Also at least a while back there is no good fft outside of cuda, maybe that changed though.
pjmlp 5y ago