5 ms·

Cool to see research in that field, I realized this year that for gaming GPU and VRAM were actually more important than CPU and RAM, it probably tells something

Cool to see research in that field, I realized this year that for gaming GPU and VRAM were actually more important than CPU and RAM, it probably tells something about underusage of GPU in general computing. There's one big blocker as far as I can tell, though: portability. When it comes to gaming, neural networks, cryptomining, etc, I always see "only nvidia cards supported", or "work best on amd". If we were to use GPU in just any kind of application, we would need to have hardware abstraction library which support any kind of GPU, including intel chips. Is such effort already being worked on, at any stage of completion?

idle_zealot 9y ago
Something like https://en.wikipedia.org/wiki/OpenCL https://en.wikipedia.org/wiki/OpenCL ?
- oelmekki 9y ago
  Awesome, thanks. I've seen the name mentioned several times, but I didn't know what was behind. It seems (unsurprisingly for an hardware abstraction) that performance is a problem, but at least it's a problem that is being worked on. Maybe at some point we will stop to think "we can't implement that, that's way too massive to perform properly" and start thinking "this is a job for the GPU" :) At the very least, databases come to mind, with their sorting/filtering of massive data.
  - autopoiesis 9y ago
    Also there's work towards a pure C++ abstraction layer for programming these accelerators, called SYCL [0]. It lets you define your compute kernels using normal C++ lambdas or functions, and then automatically infers the data dependencies to do the kernel execution. In particular, it provides an implementation of the C++ "Parallel STL" [1] (some intro slides at [2]), which in turn provides "execution policies" for various standard library functions, such as sorting. The aim is precisely the kind of thing you're talking about! [0] I found a nice intro at https://blog.tartanllama.xyz/sycl/ https://blog.tartanllama.xyz/sycl/ [1] http://en.cppreference.com/w/cpp/experimental/parallelism http://en.cppreference.com/w/cpp/experimental/parallelism [2] https://www.khronos.org/assets/uploads/developers/library/2016-supercomputing/SYCL_ParallelSTL-Ruyman-Reyes_Nov16.pdf https://www.khronos.org/assets/uploads/developers/library/20...
  - roel_v 9y ago
    The problem with OpenCL isn't performance per se, but performance portability (well it's only a problem for those that need such a thing, of course - many people don't). When you write OpenCL code and you tweak it for one CPU or GPU, it might run at 1/10th the speed on another. This is of course something you don't have with an API that works only on GPU's from one vendor, although even there different generations of hardware might prefer different parameters or tradeoffs. Now you can write OpenCL kernels that automatically tweak themselves to run as fast as possible on different hardware, but that requires significant extra work over just getting it to work at all. And finally, CUDA has a bunch of hand-tweaked libraries for doing common numerical operations (matrix multiply, FFT, ...) that are (partly) written in 'NVIDIA GPU assembly) (ptx), so those operations will be faster on CUDA than on OpenCL. CUDA is also (a bit) easier to write/use than OpenCL code and the tooling is better, so that's another reason people often default to CUDA.
    - pjmlp 9y ago
      One big reason for developers favouring CUDA is that since the early days it supported C++, Fortran and any other language with a PTX backend, where Khronos wanted everyone to just shut up and use C99. Finally they understood that they world moved on and better support to other languages had to be provided, so lets see how much OpenCL 2.2 and SPIR can improve the situation.
      - dragontamer 9y ago
        The main problem is that NVidia is screwing things up. NVidia is only supporting OpenCL1.1, which means that if you want to use C++ / SPIR, you pretty much are locked to AMD / Intel (Intel CPUs have an OpenCL -> AVX layer, so you can always "worst-case" turn OpenCL code into native CPU code) NVidia of course owns CUDA, which means they want those "premium features" locked to CUDA-only. -------- AMD's laptop offerings offer some intriguing features on OpenCL as well. Since their APUs have a CPU AND a GPU on the same die, the data-transfer between CPU / GPU on the AMD APUs (ie: an A10 laptop chip) is absurdly fast. Like, they share L2 cache IIRC, so the data doesn't even hit main-memory, or even leave the chip. But there's basically no point optimizing for that architecture, as far as I can tell anyway.
- richdougherty 9y ago
  Interesting article about OpenCL and Vulkan... https://www.pcper.com/reviews/Graphics-Cards/Follow-Neil-Trevett-and-Tom-Olson-Khronos-Group-Discuss-OpenCL-and-Vulkan-Roa https://www.pcper.com/reviews/Graphics-Cards/Follow-Neil-Tre...
VHRanger 9y ago
OpenACC
falcolas 9y ago
> for gaming GPU and VRAM were actually more important than CPU and RAM It's probably just the games I play, but these days I'm rarely GPU bound, just CPU bound. It would be great to see some frameworks make GPU accelerated physics (with automatic CPU fallback) easier for smaller game development companies to take advantage of. For example: Factorio and Space Engineers are primarily CPU bound due to the tracking of millions of objects (Factorio) & physics (Space Engineers).
- meschi 9y ago
  Nvidia PhysX?
- overcast 9y ago
  It must be the games you're playing, large scale strategy? Everything in my gaming rig is from 2008(LGA1366). The only thing I've upgraded since then is video cards. I ran an I7 920 Nehalem chip up until last year, when I "upgraded" to a Xeon X5675 from ebay as a drop in cpu replacement. Runs everything I throw at it completely maxed out at 2560x1600 resolution vsynced to the monitor refresh rate of 60hz.
  - deleted 9y ago
    [deleted]
Eridrus 9y ago
AMD is taking a stab at this with ROCm/HIP/HCC to try and get a slice of the machine learning market: https://rocm.github.io/languages.html https://rocm.github.io/languages.html I wish them all the best, but I haven't seen much adoption yet. It's not particularly clear that this is the right path yet because folks are building special-purpose neural net accelerators, and those may not have the same programming model and may make this whole HIP thing irrelevant for ML. I'm also not totally convinced software developers are ready to take advantage of something like this; developers are barely taking advantage of multiple CPU cores, let alone the more limited GPGPU environment.
14113 9y ago
The problem isn't that a particular library is AMD or NVIDIA only in terms of computation, but NVIDIA or AMD only in terms of performance. GPU implementations are usually optimised for a specific architecture, which is then the "supported" architecture. On occasion, such implementations do use vendor specific tools (such as CUDA), but there are a plethora of tools such as OpenCL, SyCL etc that provide portability - but not always performance portability, meaning that they will still be tuned to a specific architecture. For performance portability, the LIFT project (http://www.lift-project.org/ http://www.lift-project.org/) provides a partial solution. Our approach relies on a high level model of computation (think or something like a functional, or pattern based programming language) coupled with a rewrite-based compiler that explores the space of OpenCL programs with which to implement a computation. That lets us "optimise" a given implementation to a specific architecture, entirely automatically, in a way that many other low level approaches simply aren't able to, as they contain too many implementation (rather than computation) details.
- marmaduke 9y ago
  Looks interesting conceptually but it would be good to refute the "sufficiently clever compiler" in anticipation. My beef with this sort of thing or Furthark or OpenACC is that they don't seem to understand complex data layouts required for real problems.
  - 14113 9y ago
    Define "complex data layouts"? One reason that tools like Futhark often don't is that complex data structures don't provide good performance on GPUs. If, however, you mean complicated compositions of arrays, then that is something we support, as well as efficient ways for describing (e.g.) coalesced accesses or stencil operations.
    - marmaduke 9y ago
      Thanks for the reply. I should not have included Furthark with OpenACC. The layouts I have in mind are ring buffer of arrays and ND arrays. I think Furthark would work well actually but I simply had not the time to get accustomed to it.

idle_zealot 9y ago

Something like https://en.wikipedia.org/wiki/OpenCL https://en.wikipedia.org/wiki/OpenCL ?

oelmekki 9y ago
Awesome, thanks. I've seen the name mentioned several times, but I didn't know what was behind. It seems (unsurprisingly for an hardware abstraction) that performance is a problem, but at least it's a problem that is being worked on. Maybe at some point we will stop to think "we can't implement that, that's way too massive to perform properly" and start thinking "this is a job for the GPU" :) At the very least, databases come to mind, with their sorting/filtering of massive data.
- autopoiesis 9y ago
  Also there's work towards a pure C++ abstraction layer for programming these accelerators, called SYCL [0]. It lets you define your compute kernels using normal C++ lambdas or functions, and then automatically infers the data dependencies to do the kernel execution. In particular, it provides an implementation of the C++ "Parallel STL" [1] (some intro slides at [2]), which in turn provides "execution policies" for various standard library functions, such as sorting. The aim is precisely the kind of thing you're talking about! [0] I found a nice intro at https://blog.tartanllama.xyz/sycl/ https://blog.tartanllama.xyz/sycl/ [1] http://en.cppreference.com/w/cpp/experimental/parallelism http://en.cppreference.com/w/cpp/experimental/parallelism [2] https://www.khronos.org/assets/uploads/developers/library/2016-supercomputing/SYCL_ParallelSTL-Ruyman-Reyes_Nov16.pdf https://www.khronos.org/assets/uploads/developers/library/20...
- roel_v 9y ago
  The problem with OpenCL isn't performance per se, but performance portability (well it's only a problem for those that need such a thing, of course - many people don't). When you write OpenCL code and you tweak it for one CPU or GPU, it might run at 1/10th the speed on another. This is of course something you don't have with an API that works only on GPU's from one vendor, although even there different generations of hardware might prefer different parameters or tradeoffs. Now you can write OpenCL kernels that automatically tweak themselves to run as fast as possible on different hardware, but that requires significant extra work over just getting it to work at all. And finally, CUDA has a bunch of hand-tweaked libraries for doing common numerical operations (matrix multiply, FFT, ...) that are (partly) written in 'NVIDIA GPU assembly) (ptx), so those operations will be faster on CUDA than on OpenCL. CUDA is also (a bit) easier to write/use than OpenCL code and the tooling is better, so that's another reason people often default to CUDA.
  - pjmlp 9y ago
    One big reason for developers favouring CUDA is that since the early days it supported C++, Fortran and any other language with a PTX backend, where Khronos wanted everyone to just shut up and use C99. Finally they understood that they world moved on and better support to other languages had to be provided, so lets see how much OpenCL 2.2 and SPIR can improve the situation.
    - dragontamer 9y ago
      The main problem is that NVidia is screwing things up. NVidia is only supporting OpenCL1.1, which means that if you want to use C++ / SPIR, you pretty much are locked to AMD / Intel (Intel CPUs have an OpenCL -> AVX layer, so you can always "worst-case" turn OpenCL code into native CPU code) NVidia of course owns CUDA, which means they want those "premium features" locked to CUDA-only. -------- AMD's laptop offerings offer some intriguing features on OpenCL as well. Since their APUs have a CPU AND a GPU on the same die, the data-transfer between CPU / GPU on the AMD APUs (ie: an A10 laptop chip) is absurdly fast. Like, they share L2 cache IIRC, so the data doesn't even hit main-memory, or even leave the chip. But there's basically no point optimizing for that architecture, as far as I can tell anyway.
richdougherty 9y ago
Interesting article about OpenCL and Vulkan... https://www.pcper.com/reviews/Graphics-Cards/Follow-Neil-Trevett-and-Tom-Olson-Khronos-Group-Discuss-OpenCL-and-Vulkan-Roa https://www.pcper.com/reviews/Graphics-Cards/Follow-Neil-Tre...

VHRanger 9y ago

OpenACC

falcolas 9y ago

> for gaming GPU and VRAM were actually more important than CPU and RAM It's probably just the games I play, but these days I'm rarely GPU bound, just CPU bound. It would be great to see some frameworks make GPU accelerated physics (with automatic CPU fallback) easier for smaller game development companies to take advantage of. For example: Factorio and Space Engineers are primarily CPU bound due to the tracking of millions of objects (Factorio) & physics (Space Engineers).

meschi 9y ago
Nvidia PhysX?
overcast 9y ago
It must be the games you're playing, large scale strategy? Everything in my gaming rig is from 2008(LGA1366). The only thing I've upgraded since then is video cards. I ran an I7 920 Nehalem chip up until last year, when I "upgraded" to a Xeon X5675 from ebay as a drop in cpu replacement. Runs everything I throw at it completely maxed out at 2560x1600 resolution vsynced to the monitor refresh rate of 60hz.
- deleted 9y ago
  [deleted]

Eridrus 9y ago

AMD is taking a stab at this with ROCm/HIP/HCC to try and get a slice of the machine learning market: https://rocm.github.io/languages.html https://rocm.github.io/languages.html I wish them all the best, but I haven't seen much adoption yet. It's not particularly clear that this is the right path yet because folks are building special-purpose neural net accelerators, and those may not have the same programming model and may make this whole HIP thing irrelevant for ML. I'm also not totally convinced software developers are ready to take advantage of something like this; developers are barely taking advantage of multiple CPU cores, let alone the more limited GPGPU environment.

14113 9y ago

The problem isn't that a particular library is AMD or NVIDIA only in terms of computation, but NVIDIA or AMD only in terms of performance. GPU implementations are usually optimised for a specific architecture, which is then the "supported" architecture. On occasion, such implementations do use vendor specific tools (such as CUDA), but there are a plethora of tools such as OpenCL, SyCL etc that provide portability - but not always performance portability, meaning that they will still be tuned to a specific architecture. For performance portability, the LIFT project (http://www.lift-project.org/ http://www.lift-project.org/) provides a partial solution. Our approach relies on a high level model of computation (think or something like a functional, or pattern based programming language) coupled with a rewrite-based compiler that explores the space of OpenCL programs with which to implement a computation. That lets us "optimise" a given implementation to a specific architecture, entirely automatically, in a way that many other low level approaches simply aren't able to, as they contain too many implementation (rather than computation) details.

marmaduke 9y ago
Looks interesting conceptually but it would be good to refute the "sufficiently clever compiler" in anticipation. My beef with this sort of thing or Furthark or OpenACC is that they don't seem to understand complex data layouts required for real problems.
- 14113 9y ago
  Define "complex data layouts"? One reason that tools like Futhark often don't is that complex data structures don't provide good performance on GPUs. If, however, you mean complicated compositions of arrays, then that is something we support, as well as efficient ways for describing (e.g.) coalesced accesses or stencil operations.
  - marmaduke 9y ago
    Thanks for the reply. I should not have included Furthark with OpenACC. The layouts I have in mind are ring buffer of arrays and ND arrays. I think Furthark would work well actually but I simply had not the time to get accustomed to it.