5 ms·

Long video, so I haven't watched it all yet, but I have a bit of a naiive question: Does Vulkan and WebGL/WebGPU "fall back" to running on the CPU if "appropria

by geokon 7y ago

Long video, so I haven't watched it all yet, but I have a bit of a naiive question: Does Vulkan and WebGL/WebGPU "fall back" to running on the CPU if "appropriate" hardware isn't available? Or are developers required to maintain separate code paths based on hardware? I remember considering rewriting some tight loops in OpenCL but then the maintenance headache of having multiple code paths made the refactor seem not worth it. I'd guess this is/was generally a major speedbump for adoption. I know there is POCL which will run your kernels on the CPU, but it's not something you can't expect available on every platform. Maybe if POCL was part of the kernel or bundled with every distro then the situation would be different. I've seen some project do compute in OpenGL ES 2.0 b/c that's ubiquitous and will always run (I think it's even a requirement for Android systems)

raphlinus 7y ago
This is a good question. No, Vulkan and WebGPU will not fall back, they require hardware and driver support. What I think you're looking for is a higher level layer that will target GPU compute if it's available, otherwise CPU (or possibly other resources). Projects in this space include Halide and MLIR.
- geokon 7y ago
  I have a more theoretical followup then :) So you spend a lot of time in the talk massaging your sequential CPU task into an appropriately GPU task. It's clearly tricky and involves reworking your algorithms to leverage the parallelism and the memory "features" of the GPU. But through rewriting, have you actually substantively hurt the performance of the sequential program? The big picture question is : is the reverse problem of going from a GPU program to a CPU problem ever problematic? I mean in a algorithmic sense - without looking at micro optimizations for leveraging CPU SIMD instructions or whatever. Or are you always going to be pretty okay with running your shaders sequentially one after another on your big luxurious CPU memory pool? and ultimately is there anything stopping you from compiling SPIR-V for a CPU? Could you not autgenerate in the kernel dispatcher just a switch that'll branch and run a precompiled on-CPU kernel if no driver is found? Then you'd finally really get compile-once-run-everywhere GPU code I guess since it's not being done then I'm missing something haha :) Maybe you are going to often hit scenarios where you'd say "No, if I'm going to run this on the CPU I need to fundamentally rewrite the algorithm"
  - raphlinus 7y ago
    These are indeed interesting questions, thanks for asking them. If you have a workload that runs really well on GPU, then it can be adapted to run on CPU. There are automated tools for this, but it's not mainstream. To answer one of your questions, spirv-cross has a C++ generator. There are a bunch of things that are in the space between GPU and traditional scalar CPU. One of the most interesting is Intel's ispc. The Larabee project was also an attempt to build some GPU features into a CPU, and that is evolving into AVX-512. The new mask features are particularly useful for emulating the bitmap of active threads, as this is something that's hard to emulate with traditional SIMD. I think it would be a very interesting project to build a compiler from SPIR-V compute workloads to optimized multithreaded SIMD, and there are projects exploring that: https://software.intel.com/en-us/articles/spir-v-to-ispc-convert-gpu-compute-to-the-cpu https://software.intel.com/en-us/articles/spir-v-to-ispc-con... The main reason this hasn't happened is that when doing optimization it's always better to target the actual hardware than go through layers of translation. If you want to run a neural network on CPU, you're definitely going to get better performance out of matrix multiply code tuned for the specific SIMD implementation than something adapted from GPU. But I think there may still be a niche, especially if it's possible to get "pretty good" results. For machine learning and imaging workloads in particular, there's probably a lot more juice in having a high level (Halide) or medium level (MLIR) representation, and targeting both GPU and CPU as backends that do optimization specific to their targets. I'm really interested to see how this space evolves, it feels like there are major opportunities, while the scalar CPU side feels largely mined out to me.
    - SomeoneFromCA 7y ago
      Here is the question, though: would you consider Intel HD a native part of CPU or a "some GPU". I am still quite perplexed, why everyone targets CUDA and no one tries to use ubiquitous Intel HD for AI computations. I mean, Intel CPUs are everywhere (I know, know, Ryzens are chaging the situation but still..), and almost always they have GPUs onboard.
      - raphlinus 7y ago
- pcwalton 7y ago
  Note that both DX11 and DX12 do have the WARP CPU rasterizer to fall back on. WARP is quite capable; it can't compete with native hardware but is actually reasonably fast.
  - modeless 7y ago
    Vulkan now has a capable CPU implementation as well, with SwiftShader: https://github.com/google/swiftshader https://github.com/google/swiftshader
    - geokon 7y ago
      Right, and OpenCL has POCL. The original question wasn't so much if the technology exists but will it be ubiquitous and guaranteed to be present on whatever platform you run on. Looking at SwiftShader - it looks like you can bundle it with your application? Is that correct? I don't see a way to bundle POCL though
      - modeless 7y ago
        Yes, you bundle SwiftShader with your application.
- modeless 7y ago
  > Vulkan and WebGPU will not fall back WebGL does automatically fall back to the CPU in Chrome, via SwiftShader[1]. WebGPU hasn't shipped yet but I think it's likely that a CPU backend with SwiftShader will be available eventually. And SwiftShader implements Vulkan now too, so while it's not automatic for Vulkan you can certainly have a fallback if you want it. [1] https://github.com/google/swiftshader https://github.com/google/swiftshader
flumpcakes 7y ago
Hi, If you want to target GPU in some form to increase performance (open source standards that currently fill this niche are OpenCL for compute, OpenGL/Vulkan for graphics) but you want to use the same code for CPU and GPU there is a solution! There's a new standard called SYCL. Currently it is based on C++11 and there are implementations that target OpenGL and CUDA.
- VHRanger 7y ago
  OpenCL can target the CPU
  - raphlinus 7y ago
    Indeed. I think it's fair to say that OpenCL is excellent for targeting a very wide range of devices, with the exception of most high performance GPUs.
- pjmlp 7y ago
  Except that one needs to rely on tooling from CodePlay or Intel. NVidia isn't that keen in supporting OpenCL beyond 1.2 and is pushing for OpenACC in what concerns GPGPU outside CUDA on their hardware.

by geokon 7y ago

raphlinus 7y ago

This is a good question. No, Vulkan and WebGPU will not fall back, they require hardware and driver support. What I think you're looking for is a higher level layer that will target GPU compute if it's available, otherwise CPU (or possibly other resources). Projects in this space include Halide and MLIR.

geokon 7y ago
I have a more theoretical followup then :) So you spend a lot of time in the talk massaging your sequential CPU task into an appropriately GPU task. It's clearly tricky and involves reworking your algorithms to leverage the parallelism and the memory "features" of the GPU. But through rewriting, have you actually substantively hurt the performance of the sequential program? The big picture question is : is the reverse problem of going from a GPU program to a CPU problem ever problematic? I mean in a algorithmic sense - without looking at micro optimizations for leveraging CPU SIMD instructions or whatever. Or are you always going to be pretty okay with running your shaders sequentially one after another on your big luxurious CPU memory pool? and ultimately is there anything stopping you from compiling SPIR-V for a CPU? Could you not autgenerate in the kernel dispatcher just a switch that'll branch and run a precompiled on-CPU kernel if no driver is found? Then you'd finally really get compile-once-run-everywhere GPU code I guess since it's not being done then I'm missing something haha :) Maybe you are going to often hit scenarios where you'd say "No, if I'm going to run this on the CPU I need to fundamentally rewrite the algorithm"
- raphlinus 7y ago
  These are indeed interesting questions, thanks for asking them. If you have a workload that runs really well on GPU, then it can be adapted to run on CPU. There are automated tools for this, but it's not mainstream. To answer one of your questions, spirv-cross has a C++ generator. There are a bunch of things that are in the space between GPU and traditional scalar CPU. One of the most interesting is Intel's ispc. The Larabee project was also an attempt to build some GPU features into a CPU, and that is evolving into AVX-512. The new mask features are particularly useful for emulating the bitmap of active threads, as this is something that's hard to emulate with traditional SIMD. I think it would be a very interesting project to build a compiler from SPIR-V compute workloads to optimized multithreaded SIMD, and there are projects exploring that: https://software.intel.com/en-us/articles/spir-v-to-ispc-convert-gpu-compute-to-the-cpu https://software.intel.com/en-us/articles/spir-v-to-ispc-con... The main reason this hasn't happened is that when doing optimization it's always better to target the actual hardware than go through layers of translation. If you want to run a neural network on CPU, you're definitely going to get better performance out of matrix multiply code tuned for the specific SIMD implementation than something adapted from GPU. But I think there may still be a niche, especially if it's possible to get "pretty good" results. For machine learning and imaging workloads in particular, there's probably a lot more juice in having a high level (Halide) or medium level (MLIR) representation, and targeting both GPU and CPU as backends that do optimization specific to their targets. I'm really interested to see how this space evolves, it feels like there are major opportunities, while the scalar CPU side feels largely mined out to me.
  - SomeoneFromCA 7y ago
    Here is the question, though: would you consider Intel HD a native part of CPU or a "some GPU". I am still quite perplexed, why everyone targets CUDA and no one tries to use ubiquitous Intel HD for AI computations. I mean, Intel CPUs are everywhere (I know, know, Ryzens are chaging the situation but still..), and almost always they have GPUs onboard.
    - raphlinus 7y ago
pcwalton 7y ago
Note that both DX11 and DX12 do have the WARP CPU rasterizer to fall back on. WARP is quite capable; it can't compete with native hardware but is actually reasonably fast.
- modeless 7y ago
  Vulkan now has a capable CPU implementation as well, with SwiftShader: https://github.com/google/swiftshader https://github.com/google/swiftshader
  - geokon 7y ago
    Right, and OpenCL has POCL. The original question wasn't so much if the technology exists but will it be ubiquitous and guaranteed to be present on whatever platform you run on. Looking at SwiftShader - it looks like you can bundle it with your application? Is that correct? I don't see a way to bundle POCL though
    - modeless 7y ago
      Yes, you bundle SwiftShader with your application.
modeless 7y ago
> Vulkan and WebGPU will not fall back WebGL does automatically fall back to the CPU in Chrome, via SwiftShader[1]. WebGPU hasn't shipped yet but I think it's likely that a CPU backend with SwiftShader will be available eventually. And SwiftShader implements Vulkan now too, so while it's not automatic for Vulkan you can certainly have a fallback if you want it. [1] https://github.com/google/swiftshader https://github.com/google/swiftshader

flumpcakes 7y ago

Hi, If you want to target GPU in some form to increase performance (open source standards that currently fill this niche are OpenCL for compute, OpenGL/Vulkan for graphics) but you want to use the same code for CPU and GPU there is a solution! There's a new standard called SYCL. Currently it is based on C++11 and there are implementations that target OpenGL and CUDA.

VHRanger 7y ago
OpenCL can target the CPU
- raphlinus 7y ago
  Indeed. I think it's fair to say that OpenCL is excellent for targeting a very wide range of devices, with the exception of most high performance GPUs.
pjmlp 7y ago
Except that one needs to rely on tooling from CodePlay or Intel. NVidia isn't that keen in supporting OpenCL beyond 1.2 and is pushing for OpenACC in what concerns GPGPU outside CUDA on their hardware.