4 ms·

Perhaps the fact that 99% of users never write kernels is partial due to the fact that you need to switch languages to do so. Maybe a lot more users might want

Perhaps the fact that 99% of users never write kernels is partial due to the fact that you need to switch languages to do so. Maybe a lot more users might want to write and modify GPU kernels if they had that as an easier option

p1esk 6y ago
Writing a fast CUDA kernel requires quite a bit of low level GPU knowledge. You typically do that when you really need it and it's not already available - like when you invented a new sparse GEMM algorithm, or want to speed up convolutions with binary weights/inputs. If you do need that, congrats, you're the 1%, and you're likely to not be afraid of mixing languages and getting your hands dirty with C++.
- pjmlp 6y ago
  One of the reasons CUDA wipedout OpenCL is due to its polyglot support, no need to get hands dirty with C++. Fortran, Java, .NET. Julia, D, Rust and anything else able to either generate PTX or integrate with CUDA LLVM backend can be used.
- amkkma 6y ago
  >Writing a fast CUDA kernel requires quite a bit of low level GPU knowledge False. Not in Julia: https://juliagpu.gitlab.io/KernelAbstractions.jl/ https://juliagpu.gitlab.io/KernelAbstractions.jl/ https://github.com/mcabbott/Tullio.jl https://github.com/mcabbott/Tullio.jl
  - ChrisRackauckas 6y ago
    To share the details, here's a post demonstrating Tullio.jl, a Julia-based higher order tensor library, is faster than OpenBLAS for gemm kernels: https://discourse.julialang.org/t/realistically-how-close-is-gaius-jl-to-becoming-a-full-replacement-for-blas-in-julia/42780/5 https://discourse.julialang.org/t/realistically-how-close-is... And PaddedMatrices.jl is another example: https://discourse.julialang.org/t/ann-paddedmatrices-jl-julia-blas-and-partially-sized-arrays/38215 https://discourse.julialang.org/t/ann-paddedmatrices-jl-juli... So there isn't just one example but multiples at this point. RecursiveFactorization's `lu` tends to perform better than MKL for small matrices and this is what we use in the differential equation solvers when there are <100 ODEs. That's all on CPUs though. KernelAbstractions.jl is the GPU version and that's still being constructed but moving strong.
  - amkkma 6y ago
    For the person who downvoted me, care to refute my examples?
    - p1esk 6y ago
      I haven't downvoted you, but from KernelAbstractions.jl docs it's not clear you will be able to write a fast kernel (emphasis on fast) without having as much GPU/CUDA knowledge as what's required to write the same fast kernel in CUDA. I might be wrong, and if so that would be great. Here's an example: not long ago I needed a fast GPU implementation of numpy.unpackbits. I can do it in Pytorch, but it's slow and it wastes a ton of memory. So, my question is: can I write it in Julia as efficiently as I can do it in CUDA? If yes, will it be easier to do so?
      - snicker7 6y ago
        KernelAbstractions.jl has some simple examples in its repo [^0]. The API looks simple and elegant. Code looks like regular Julia code. Documentation well behind CUDA, of course. [^0]: https://github.com/JuliaGPU/KernelAbstractions.jl/blob/master/examples/naive_transpose.jl https://github.com/JuliaGPU/KernelAbstractions.jl/blob/maste...
      - ChrisRackauckas 6y ago
        The code is essentially just Julia code: https://juliagpu.gitlab.io/KernelAbstractions.jl/examples/naive_transpose/ https://juliagpu.gitlab.io/KernelAbstractions.jl/examples/na... and you get ~60% of the best known CUDA implementation on that test: https://juliagpu.gitlab.io/KernelAbstractions.jl/examples/performance/ https://juliagpu.gitlab.io/KernelAbstractions.jl/examples/pe... Not perfect but not awful.
        p1esk 6y ago
        I’m sorry, what does it have to do with my question? I gave a specific example where I couldn’t get what I needed from Pytorch. Would Julia be any different in this situation?

p1esk 6y ago

Writing a fast CUDA kernel requires quite a bit of low level GPU knowledge. You typically do that when you really need it and it's not already available - like when you invented a new sparse GEMM algorithm, or want to speed up convolutions with binary weights/inputs. If you do need that, congrats, you're the 1%, and you're likely to not be afraid of mixing languages and getting your hands dirty with C++.

pjmlp 6y ago
One of the reasons CUDA wipedout OpenCL is due to its polyglot support, no need to get hands dirty with C++. Fortran, Java, .NET. Julia, D, Rust and anything else able to either generate PTX or integrate with CUDA LLVM backend can be used.
amkkma 6y ago
>Writing a fast CUDA kernel requires quite a bit of low level GPU knowledge False. Not in Julia: https://juliagpu.gitlab.io/KernelAbstractions.jl/ https://juliagpu.gitlab.io/KernelAbstractions.jl/ https://github.com/mcabbott/Tullio.jl https://github.com/mcabbott/Tullio.jl
- ChrisRackauckas 6y ago
  To share the details, here's a post demonstrating Tullio.jl, a Julia-based higher order tensor library, is faster than OpenBLAS for gemm kernels: https://discourse.julialang.org/t/realistically-how-close-is-gaius-jl-to-becoming-a-full-replacement-for-blas-in-julia/42780/5 https://discourse.julialang.org/t/realistically-how-close-is... And PaddedMatrices.jl is another example: https://discourse.julialang.org/t/ann-paddedmatrices-jl-julia-blas-and-partially-sized-arrays/38215 https://discourse.julialang.org/t/ann-paddedmatrices-jl-juli... So there isn't just one example but multiples at this point. RecursiveFactorization's `lu` tends to perform better than MKL for small matrices and this is what we use in the differential equation solvers when there are <100 ODEs. That's all on CPUs though. KernelAbstractions.jl is the GPU version and that's still being constructed but moving strong.
- amkkma 6y ago
  For the person who downvoted me, care to refute my examples?
  - p1esk 6y ago
    I haven't downvoted you, but from KernelAbstractions.jl docs it's not clear you will be able to write a fast kernel (emphasis on fast) without having as much GPU/CUDA knowledge as what's required to write the same fast kernel in CUDA. I might be wrong, and if so that would be great. Here's an example: not long ago I needed a fast GPU implementation of numpy.unpackbits. I can do it in Pytorch, but it's slow and it wastes a ton of memory. So, my question is: can I write it in Julia as efficiently as I can do it in CUDA? If yes, will it be easier to do so?
    - snicker7 6y ago
      KernelAbstractions.jl has some simple examples in its repo [^0]. The API looks simple and elegant. Code looks like regular Julia code. Documentation well behind CUDA, of course. [^0]: https://github.com/JuliaGPU/KernelAbstractions.jl/blob/master/examples/naive_transpose.jl https://github.com/JuliaGPU/KernelAbstractions.jl/blob/maste...
    - ChrisRackauckas 6y ago
      The code is essentially just Julia code: https://juliagpu.gitlab.io/KernelAbstractions.jl/examples/naive_transpose/ https://juliagpu.gitlab.io/KernelAbstractions.jl/examples/na... and you get ~60% of the best known CUDA implementation on that test: https://juliagpu.gitlab.io/KernelAbstractions.jl/examples/performance/ https://juliagpu.gitlab.io/KernelAbstractions.jl/examples/pe... Not perfect but not awful.
      - p1esk 6y ago
        I’m sorry, what does it have to do with my question? I gave a specific example where I couldn’t get what I needed from Pytorch. Would Julia be any different in this situation?