4 ms·

These are indeed interesting questions, thanks for asking them. If you have a workload that runs really well on GPU, then it can be adapted to run on CPU. Ther

These are indeed interesting questions, thanks for asking them. If you have a workload that runs really well on GPU, then it can be adapted to run on CPU. There are automated tools for this, but it's not mainstream. To answer one of your questions, spirv-cross has a C++ generator. There are a bunch of things that are in the space between GPU and traditional scalar CPU. One of the most interesting is Intel's ispc. The Larabee project was also an attempt to build some GPU features into a CPU, and that is evolving into AVX-512. The new mask features are particularly useful for emulating the bitmap of active threads, as this is something that's hard to emulate with traditional SIMD. I think it would be a very interesting project to build a compiler from SPIR-V compute workloads to optimized multithreaded SIMD, and there are projects exploring that: https://software.intel.com/en-us/articles/spir-v-to-ispc-convert-gpu-compute-to-the-cpu https://software.intel.com/en-us/articles/spir-v-to-ispc-con... The main reason this hasn't happened is that when doing optimization it's always better to target the actual hardware than go through layers of translation. If you want to run a neural network on CPU, you're definitely going to get better performance out of matrix multiply code tuned for the specific SIMD implementation than something adapted from GPU. But I think there may still be a niche, especially if it's possible to get "pretty good" results. For machine learning and imaging workloads in particular, there's probably a lot more juice in having a high level (Halide) or medium level (MLIR) representation, and targeting both GPU and CPU as backends that do optimization specific to their targets. I'm really interested to see how this space evolves, it feels like there are major opportunities, while the scalar CPU side feels largely mined out to me.

SomeoneFromCA 7y ago
Here is the question, though: would you consider Intel HD a native part of CPU or a "some GPU". I am still quite perplexed, why everyone targets CUDA and no one tries to use ubiquitous Intel HD for AI computations. I mean, Intel CPUs are everywhere (I know, know, Ryzens are chaging the situation but still..), and almost always they have GPUs onboard.
- raphlinus 7y ago
  They definitely qualify as GPU. I think the main reason people aren't using them is that tools are primitive. Also, historically they've been pretty anemic in horsepower, though are getting better. Still not competitive with high end discrete graphics cards, but absolutely getting there on the low end. Intel GPU has three hidden strengths: * CPU <-> GPU communication is cheaper, because they can actually share memory (a separate copy to staging buffers is not needed). * Latency is potentially lower, although it's not clear to me yet that driver software can take advantage of the potential offered by the hardware. (More empirical measurement is needed) * Subgroup operations on Intel appear to be wicked-fast, shuffle in particular. Long story short, I think there is opportunity here that most people aren't exploiting.
  - auggierose 7y ago
    It seems to me that if you use Apple Metal, then you are definitely (and automatically) exploiting Intel GPUs. So lots of people are actually exploiting it!
    - hellofunk 7y ago
      Metal targets whatever GPU is in the system, not necessarily Intel. In most cases there’s a discrete in there by AMD or Nvidia.
      - auggierose 7y ago
        Well, that's my point. As a Metal programmer, you don't have to distinguish (except of being aware of shared memory in integrated gpu). The talk made it sound like it is a totally different world between mobile / desktop, not to mention Intel GPUs. From a certain point of view, and for many algorithms, it isn't. Especially when the algorithm can be viewed as a functional program, as championed in the talk. In particular, most of the time there is an integrated one in there, because there are many more mobiles out there than desktops.

SomeoneFromCA 7y ago

Here is the question, though: would you consider Intel HD a native part of CPU or a "some GPU". I am still quite perplexed, why everyone targets CUDA and no one tries to use ubiquitous Intel HD for AI computations. I mean, Intel CPUs are everywhere (I know, know, Ryzens are chaging the situation but still..), and almost always they have GPUs onboard.

raphlinus 7y ago
They definitely qualify as GPU. I think the main reason people aren't using them is that tools are primitive. Also, historically they've been pretty anemic in horsepower, though are getting better. Still not competitive with high end discrete graphics cards, but absolutely getting there on the low end. Intel GPU has three hidden strengths: * CPU <-> GPU communication is cheaper, because they can actually share memory (a separate copy to staging buffers is not needed). * Latency is potentially lower, although it's not clear to me yet that driver software can take advantage of the potential offered by the hardware. (More empirical measurement is needed) * Subgroup operations on Intel appear to be wicked-fast, shuffle in particular. Long story short, I think there is opportunity here that most people aren't exploiting.
- auggierose 7y ago
  It seems to me that if you use Apple Metal, then you are definitely (and automatically) exploiting Intel GPUs. So lots of people are actually exploiting it!
  - hellofunk 7y ago
    Metal targets whatever GPU is in the system, not necessarily Intel. In most cases there’s a discrete in there by AMD or Nvidia.
    - auggierose 7y ago
      Well, that's my point. As a Metal programmer, you don't have to distinguish (except of being aware of shared memory in integrated gpu). The talk made it sound like it is a totally different world between mobile / desktop, not to mention Intel GPUs. From a certain point of view, and for many algorithms, it isn't. Especially when the algorithm can be viewed as a functional program, as championed in the talk. In particular, most of the time there is an integrated one in there, because there are many more mobiles out there than desktops.