5 ms·

I can't wait until compilers can start auto generating GPU kernels. That will be when GPGPU really takes off for most people who's applications aren't critical

I can't wait until compilers can start auto generating GPU kernels. That will be when GPGPU really takes off for most people who's applications aren't critical enough to spend hours writing these by hand but would benefit from the significant speed up.

pcwalton 9y ago
I'm not sure that will ever happen, at least without changing languages. Autovectorization alone is difficult in C and C++ because the languages provide almost no useful information about aliasing. Precise aliasing info is just the tip of the iceberg regarding what you would need for GPU-based autovectorization.
- moosingin3space 9y ago
  Is there any discussion about doing this in Rust? (maybe surrounding the SIMD RFCs)
- willvarfar 9y ago
  (On the Mill CPU the retire stations snoop/snarf writes and spot aliasing as it happens at runtime. This is a big part of our auto-vectorisation)
  - nicwilson 9y ago
    Cool, I'm looking forward to seeing more news about the mill.
    - willvarfar 9y ago
      https://news.ycombinator.com/item?id=14797350 https://news.ycombinator.com/item?id=14797350 probably won't get on the front page of HN :|
  - CyberDildonics 9y ago
    Interesting, are there benchmarks I can look at?
- WalterBright 9y ago
  D has array expressions, which look like: a[] = c * b[]; The idea of them is they are parallelizable, and do not require an auto-vectoring loop optimizer. Auto-vectorizing is fraught with problems, like the user not being aware if the auto-vectorizing succeeded or not. Another aspect of D that enables parallization is the use of ranges + algorithms. Although currently unexploited, it has the potential to be able to express parallel algorithms, and then take advantage of that.
  - VHRanger 9y ago
    > Another aspect of D that enables parallization is the use of ranges + algorithms. Is that similar to the C++ std::algorithm execution policy features introduced in C++17?
    - nicwilson 9y ago
      No. What Walter is talking about in D is if you have Foo[] foo = ...; foreach(f; foo) { f.bar.baz.quux; } and the computation is parallelisable then you can write import std.parallelism; foreach(f; parallel(foo)) { f.bar.baz.quux; } and it will parallelised across threads. I guess std.parallelism's parallel might be more analogous to an "instance" of an executor.
      - VHRanger 9y ago
        So that's similar to OpenMP parallel for looping?
    - WalterBright 9y ago
      I'm not familiar with that aspect of C++17.
      - nicwilson 9y ago
        Its the tag based dispatching for the `par` in std::sort(par,a.begin(),a.end()); to call a parallel implementation of sort on `a`.
jamesaross 9y ago
There are several efforts including VexCL [0] expression templates, OpenACC [1] preprocessor, Kokkos [2] template metaprogramming, SYCL [3], and many other similar projects. Parallel programming for non-trivial parallelism is still...non-trivial. [0] https://github.com/ddemidov/vexcl https://github.com/ddemidov/vexcl [1] https://www.openacc.org/ https://www.openacc.org/ [2] https://github.com/kokkos/kokkos https://github.com/kokkos/kokkos [3] https://www.khronos.org/sycl https://www.khronos.org/sycl
- VHRanger 9y ago
  If OpenACC worked as well as OpenMP (or OpenMP offloads worked as simply) it'd be huge, I would think
moosingin3space 9y ago
Apparently Numba can (kind of) do this. It JITs Python/Numpy/Scipy code through the LLVM-based toolchain for CUDA. [1] http://numba.pydata.org/numba-doc/latest/cuda/kernels.html http://numba.pydata.org/numba-doc/latest/cuda/kernels.html [2] http://numba.pydata.org/numba-doc/latest/cuda/reduction.html#reduce http://numba.pydata.org/numba-doc/latest/cuda/reduction.html...
- VHRanger 9y ago
  You still need to write what's basically CUDA code for it to run. It's not at all like you write python and it translates it to CUDA or anything.
  - helgie 9y ago
    Using the 'cuda.jit' method as linked does require you to do things like manually setting threads and blocks, though one could argue it makes it easier than doing it in CUDA C. However numba's 'vectorize' and 'guvectorize' decorators can also run code on the GPU. The current documentation doesn't show good GPU examples, but here's examples from the documentation for the deprecated numbapro (the CUDA things from numbapro were later added into numba): https://docs.continuum.io/numbapro/CUDAufunc https://docs.continuum.io/numbapro/CUDAufunc @vectorize(['float32(float32, float32, float32)', 'float64(float64, float64, float64)'], target='gpu') def cu_discriminant(a, b, c): return math.sqrt(b ** 2 - 4 * a * c) The 'float32/64' type signatures are not strictly necessary, unless you want to define the output type (so if the inputs are 32-bit floats and you don't want it to return 64-bit floats); if given no signature numba will automatically compile a new kernel each time the function is called with a new type signature. So that function would become (but in current numba 'gpu' should be replaced with 'cuda'): @vectorize(target='gpu') def cu_discriminant(a, b, c): return math.sqrt(b ** 2 - 4 * a * c) Vectorize is a little limited in that it only operates on scalars and broadcasts those scalar operations over arrays. guvectorize is more powerful and can operate on arrays directly so something like convolution or a moving average are possible, but is slightly more complicated to use than vectorize. Update: fixed code formatting
Athas 9y ago
It is really difficult to extract parallelism from sequentially written programs. It is even worse when it comes to restricted fine-grained parallelism, as on a GPU. I'm not aware of any really robust auto-parallelising compiler, and even compilers for specialised parallel languages struggle to reach performance close to hand-written code for more complex problems.
e9 9y ago
not sure if this is what you are looking for but I had success with this Java library http://aparapi.com/ http://aparapi.com/, it looks at function's JVM byte code and runs it on GPU if it can, otherwise falls back to CPU. I suspect same thing can be done in other languages. https://github.com/Syncleus/aparapi/ https://github.com/Syncleus/aparapi/ "Aparapi allows developers to write native Java code capable of being executed directly on a graphics card GPU by converting Java byte code to an OpenCL kernel dynamically at runtime. Because it is backed by OpenCL Aparapi is compatible with all OpenCL compatible Graphics Cards."
nicwilson 9y ago
One of the main goals for DCompute is to lower the barrier of entry, so that people how have embarrassingly parallel problems can take advantage of their hardware without being an expert.

pcwalton 9y ago

I'm not sure that will ever happen, at least without changing languages. Autovectorization alone is difficult in C and C++ because the languages provide almost no useful information about aliasing. Precise aliasing info is just the tip of the iceberg regarding what you would need for GPU-based autovectorization.

moosingin3space 9y ago
Is there any discussion about doing this in Rust? (maybe surrounding the SIMD RFCs)
willvarfar 9y ago
(On the Mill CPU the retire stations snoop/snarf writes and spot aliasing as it happens at runtime. This is a big part of our auto-vectorisation)
- nicwilson 9y ago
  Cool, I'm looking forward to seeing more news about the mill.
  - willvarfar 9y ago
    https://news.ycombinator.com/item?id=14797350 https://news.ycombinator.com/item?id=14797350 probably won't get on the front page of HN :|
- CyberDildonics 9y ago
  Interesting, are there benchmarks I can look at?
WalterBright 9y ago
D has array expressions, which look like: a[] = c * b[]; The idea of them is they are parallelizable, and do not require an auto-vectoring loop optimizer. Auto-vectorizing is fraught with problems, like the user not being aware if the auto-vectorizing succeeded or not. Another aspect of D that enables parallization is the use of ranges + algorithms. Although currently unexploited, it has the potential to be able to express parallel algorithms, and then take advantage of that.
- VHRanger 9y ago
  > Another aspect of D that enables parallization is the use of ranges + algorithms. Is that similar to the C++ std::algorithm execution policy features introduced in C++17?
  - nicwilson 9y ago
    No. What Walter is talking about in D is if you have Foo[] foo = ...; foreach(f; foo) { f.bar.baz.quux; } and the computation is parallelisable then you can write import std.parallelism; foreach(f; parallel(foo)) { f.bar.baz.quux; } and it will parallelised across threads. I guess std.parallelism's parallel might be more analogous to an "instance" of an executor.
    - VHRanger 9y ago
      So that's similar to OpenMP parallel for looping?
  - WalterBright 9y ago
    I'm not familiar with that aspect of C++17.
    - nicwilson 9y ago
      Its the tag based dispatching for the `par` in std::sort(par,a.begin(),a.end()); to call a parallel implementation of sort on `a`.

jamesaross 9y ago

There are several efforts including VexCL [0] expression templates, OpenACC [1] preprocessor, Kokkos [2] template metaprogramming, SYCL [3], and many other similar projects. Parallel programming for non-trivial parallelism is still...non-trivial. [0] https://github.com/ddemidov/vexcl https://github.com/ddemidov/vexcl [1] https://www.openacc.org/ https://www.openacc.org/ [2] https://github.com/kokkos/kokkos https://github.com/kokkos/kokkos [3] https://www.khronos.org/sycl https://www.khronos.org/sycl

VHRanger 9y ago
If OpenACC worked as well as OpenMP (or OpenMP offloads worked as simply) it'd be huge, I would think

moosingin3space 9y ago

Apparently Numba can (kind of) do this. It JITs Python/Numpy/Scipy code through the LLVM-based toolchain for CUDA. [1] http://numba.pydata.org/numba-doc/latest/cuda/kernels.html http://numba.pydata.org/numba-doc/latest/cuda/kernels.html [2] http://numba.pydata.org/numba-doc/latest/cuda/reduction.html#reduce http://numba.pydata.org/numba-doc/latest/cuda/reduction.html...

VHRanger 9y ago
You still need to write what's basically CUDA code for it to run. It's not at all like you write python and it translates it to CUDA or anything.
- helgie 9y ago
  Using the 'cuda.jit' method as linked does require you to do things like manually setting threads and blocks, though one could argue it makes it easier than doing it in CUDA C. However numba's 'vectorize' and 'guvectorize' decorators can also run code on the GPU. The current documentation doesn't show good GPU examples, but here's examples from the documentation for the deprecated numbapro (the CUDA things from numbapro were later added into numba): https://docs.continuum.io/numbapro/CUDAufunc https://docs.continuum.io/numbapro/CUDAufunc @vectorize(['float32(float32, float32, float32)', 'float64(float64, float64, float64)'], target='gpu') def cu_discriminant(a, b, c): return math.sqrt(b ** 2 - 4 * a * c) The 'float32/64' type signatures are not strictly necessary, unless you want to define the output type (so if the inputs are 32-bit floats and you don't want it to return 64-bit floats); if given no signature numba will automatically compile a new kernel each time the function is called with a new type signature. So that function would become (but in current numba 'gpu' should be replaced with 'cuda'): @vectorize(target='gpu') def cu_discriminant(a, b, c): return math.sqrt(b ** 2 - 4 * a * c) Vectorize is a little limited in that it only operates on scalars and broadcasts those scalar operations over arrays. guvectorize is more powerful and can operate on arrays directly so something like convolution or a moving average are possible, but is slightly more complicated to use than vectorize. Update: fixed code formatting

Athas 9y ago

It is really difficult to extract parallelism from sequentially written programs. It is even worse when it comes to restricted fine-grained parallelism, as on a GPU. I'm not aware of any really robust auto-parallelising compiler, and even compilers for specialised parallel languages struggle to reach performance close to hand-written code for more complex problems.

e9 9y ago

not sure if this is what you are looking for but I had success with this Java library http://aparapi.com/ http://aparapi.com/, it looks at function's JVM byte code and runs it on GPU if it can, otherwise falls back to CPU. I suspect same thing can be done in other languages. https://github.com/Syncleus/aparapi/ https://github.com/Syncleus/aparapi/ "Aparapi allows developers to write native Java code capable of being executed directly on a graphics card GPU by converting Java byte code to an OpenCL kernel dynamically at runtime. Because it is backed by OpenCL Aparapi is compatible with all OpenCL compatible Graphics Cards."

nicwilson 9y ago

One of the main goals for DCompute is to lower the barrier of entry, so that people how have embarrassingly parallel problems can take advantage of their hardware without being an expert.