4 ms·

Awesome, thanks. I've seen the name mentioned several times, but I didn't know what was behind. It seems (unsurprisingly for an hardware abstraction) that perf

Awesome, thanks. I've seen the name mentioned several times, but I didn't know what was behind. It seems (unsurprisingly for an hardware abstraction) that performance is a problem, but at least it's a problem that is being worked on. Maybe at some point we will stop to think "we can't implement that, that's way too massive to perform properly" and start thinking "this is a job for the GPU" :) At the very least, databases come to mind, with their sorting/filtering of massive data.

autopoiesis 9y ago
Also there's work towards a pure C++ abstraction layer for programming these accelerators, called SYCL [0]. It lets you define your compute kernels using normal C++ lambdas or functions, and then automatically infers the data dependencies to do the kernel execution. In particular, it provides an implementation of the C++ "Parallel STL" [1] (some intro slides at [2]), which in turn provides "execution policies" for various standard library functions, such as sorting. The aim is precisely the kind of thing you're talking about! [0] I found a nice intro at https://blog.tartanllama.xyz/sycl/ https://blog.tartanllama.xyz/sycl/ [1] http://en.cppreference.com/w/cpp/experimental/parallelism http://en.cppreference.com/w/cpp/experimental/parallelism [2] https://www.khronos.org/assets/uploads/developers/library/2016-supercomputing/SYCL_ParallelSTL-Ruyman-Reyes_Nov16.pdf https://www.khronos.org/assets/uploads/developers/library/20...
roel_v 9y ago
The problem with OpenCL isn't performance per se, but performance portability (well it's only a problem for those that need such a thing, of course - many people don't). When you write OpenCL code and you tweak it for one CPU or GPU, it might run at 1/10th the speed on another. This is of course something you don't have with an API that works only on GPU's from one vendor, although even there different generations of hardware might prefer different parameters or tradeoffs. Now you can write OpenCL kernels that automatically tweak themselves to run as fast as possible on different hardware, but that requires significant extra work over just getting it to work at all. And finally, CUDA has a bunch of hand-tweaked libraries for doing common numerical operations (matrix multiply, FFT, ...) that are (partly) written in 'NVIDIA GPU assembly) (ptx), so those operations will be faster on CUDA than on OpenCL. CUDA is also (a bit) easier to write/use than OpenCL code and the tooling is better, so that's another reason people often default to CUDA.
- pjmlp 9y ago
  One big reason for developers favouring CUDA is that since the early days it supported C++, Fortran and any other language with a PTX backend, where Khronos wanted everyone to just shut up and use C99. Finally they understood that they world moved on and better support to other languages had to be provided, so lets see how much OpenCL 2.2 and SPIR can improve the situation.
  - dragontamer 9y ago
    The main problem is that NVidia is screwing things up. NVidia is only supporting OpenCL1.1, which means that if you want to use C++ / SPIR, you pretty much are locked to AMD / Intel (Intel CPUs have an OpenCL -> AVX layer, so you can always "worst-case" turn OpenCL code into native CPU code) NVidia of course owns CUDA, which means they want those "premium features" locked to CUDA-only. -------- AMD's laptop offerings offer some intriguing features on OpenCL as well. Since their APUs have a CPU AND a GPU on the same die, the data-transfer between CPU / GPU on the AMD APUs (ie: an A10 laptop chip) is absurdly fast. Like, they share L2 cache IIRC, so the data doesn't even hit main-memory, or even leave the chip. But there's basically no point optimizing for that architecture, as far as I can tell anyway.
    - pjmlp 9y ago
      Another example would be with Vulkan I guess. NVidia has lots of models with DX 12 level support that don't have Vulkan drivers.
  - ryanpepper 9y ago
    In academia it's also because NVidia does a lot of stuff to make your life easy. For example, NVidia came to our University in the UK and provided training for £20 an academic/PhD student for a 2 day course on how to use CUDA and with performance tips, hands on porting of code, etc. They also give away CUDA cards to academics under a hardware grant scheme, so it's possible to get a free Titan Xp this year for a research group. There's not really an equivalent for AMD or Intel; a Xeon Phi Knights Landing chip is significantly more expensive than a consumer level GPU, and the same cost as a workstation GPU, and it's a lot harder to get good performance from it. It also doesn't seem like AMD are targeting this market, at least not currently.

autopoiesis 9y ago

Also there's work towards a pure C++ abstraction layer for programming these accelerators, called SYCL [0]. It lets you define your compute kernels using normal C++ lambdas or functions, and then automatically infers the data dependencies to do the kernel execution. In particular, it provides an implementation of the C++ "Parallel STL" [1] (some intro slides at [2]), which in turn provides "execution policies" for various standard library functions, such as sorting. The aim is precisely the kind of thing you're talking about! [0] I found a nice intro at https://blog.tartanllama.xyz/sycl/ https://blog.tartanllama.xyz/sycl/ [1] http://en.cppreference.com/w/cpp/experimental/parallelism http://en.cppreference.com/w/cpp/experimental/parallelism [2] https://www.khronos.org/assets/uploads/developers/library/2016-supercomputing/SYCL_ParallelSTL-Ruyman-Reyes_Nov16.pdf https://www.khronos.org/assets/uploads/developers/library/20...

roel_v 9y ago

The problem with OpenCL isn't performance per se, but performance portability (well it's only a problem for those that need such a thing, of course - many people don't). When you write OpenCL code and you tweak it for one CPU or GPU, it might run at 1/10th the speed on another. This is of course something you don't have with an API that works only on GPU's from one vendor, although even there different generations of hardware might prefer different parameters or tradeoffs. Now you can write OpenCL kernels that automatically tweak themselves to run as fast as possible on different hardware, but that requires significant extra work over just getting it to work at all. And finally, CUDA has a bunch of hand-tweaked libraries for doing common numerical operations (matrix multiply, FFT, ...) that are (partly) written in 'NVIDIA GPU assembly) (ptx), so those operations will be faster on CUDA than on OpenCL. CUDA is also (a bit) easier to write/use than OpenCL code and the tooling is better, so that's another reason people often default to CUDA.

pjmlp 9y ago
One big reason for developers favouring CUDA is that since the early days it supported C++, Fortran and any other language with a PTX backend, where Khronos wanted everyone to just shut up and use C99. Finally they understood that they world moved on and better support to other languages had to be provided, so lets see how much OpenCL 2.2 and SPIR can improve the situation.
- dragontamer 9y ago
  The main problem is that NVidia is screwing things up. NVidia is only supporting OpenCL1.1, which means that if you want to use C++ / SPIR, you pretty much are locked to AMD / Intel (Intel CPUs have an OpenCL -> AVX layer, so you can always "worst-case" turn OpenCL code into native CPU code) NVidia of course owns CUDA, which means they want those "premium features" locked to CUDA-only. -------- AMD's laptop offerings offer some intriguing features on OpenCL as well. Since their APUs have a CPU AND a GPU on the same die, the data-transfer between CPU / GPU on the AMD APUs (ie: an A10 laptop chip) is absurdly fast. Like, they share L2 cache IIRC, so the data doesn't even hit main-memory, or even leave the chip. But there's basically no point optimizing for that architecture, as far as I can tell anyway.
  - pjmlp 9y ago
    Another example would be with Vulkan I guess. NVidia has lots of models with DX 12 level support that don't have Vulkan drivers.
- ryanpepper 9y ago
  In academia it's also because NVidia does a lot of stuff to make your life easy. For example, NVidia came to our University in the UK and provided training for £20 an academic/PhD student for a 2 day course on how to use CUDA and with performance tips, hands on porting of code, etc. They also give away CUDA cards to academics under a hardware grant scheme, so it's possible to get a free Titan Xp this year for a research group. There's not really an equivalent for AMD or Intel; a Xeon Phi Knights Landing chip is significantly more expensive than a consumer level GPU, and the same cost as a workstation GPU, and it's a lot harder to get good performance from it. It also doesn't seem like AMD are targeting this market, at least not currently.