3 ms·

> My hope is eventually we start to invest in Vulkan Compute. Vulkan is for graphics. Khronos' compute standard that's most similar to Cuda is SYCL. Both compi

> My hope is eventually we start to invest in Vulkan Compute. Vulkan is for graphics. Khronos' compute standard that's most similar to Cuda is SYCL. Both compile shaders to SPIR-V though.

rektide 5y ago
> Vulkan is for graphics. Incorrect. Quoting the spec: > Vulkan is an API (Application Programming Interface) for graphics and compute hardware. Vulkan has compute shaders[1], which are generally usable. Libraries like VkFFT[2] demonstrate basic signal processing in Vulkan. There are plenty of other non-graphical Compute Shader examples, and this is part of the design of Vulkan (and also WebGPU). Further, there is a Vulkan ML TSG (Technical Subgroup)[3], which is supposed to be working on ML. Even Nvidia is participating, with extensions like VK_NV_cooperative_matrix, which specifically target the ml tensor cores. A more complex & current example, we see works like Google's IREE, which allow inference/Tensorflow Lite execution on a variety of drivers, including Vulkan[4], which has broad portability across hardware & fairly decent performance, even on mobile chips. There's people could probably say this better/more specifically, but I'll give it a try: Vulkan is, above all, an general standard for modelling, dispatching & orchestrating work usually on a GPU. Right now that usage is predominately graphics, but that is far from a limit. The ideas of representing GPU resources, dispatching/queueing work are generic, apply fairly reasonably to all GPU systems, and can model any workload done on a GPU. A good general introduction to Vulkan Compute is this great write up here[5]: https://www.duskborn.com/posts/a-simple-vulkan-compute-example/ https://www.duskborn.com/posts/a-simple-vulkan-compute-examp... > Khronos' compute standard that's most similar to Cuda is SYCL. SYCL is, imo, the opposite of where we need to go. It's the old historical legacy that CUDA has, of writing really dumb ignorant code & hoping the tools can make it run well on a GPU. Vulkan, on the other hand, asks us to consider deeply what near-to-the-metal resources we are going to need, and demands that we define, dispatch, & complete the actual processing engines on the GPU that will do the work. It's a much much much harder task, but it invites in fantastic levels of close optimization & tuning, allows for far more advanced pipelining & possibilities. If the future is good, it should abandon lo-fi easy options like SYCL and CUDA, and bother to get good at Vulkan, which will allow us to work intimately with the GPU. This is a relationship worth forging, and no substitutes will cut it. [1] https://vkguide.dev/docs/gpudriven/compute_shaders/ https://vkguide.dev/docs/gpudriven/compute_shaders/ [2] https://github.com/DTolm/VkFFT https://github.com/DTolm/VkFFT [3] https://www.khronos.org/assets/uploads/developers/presentations/Khronos-Vulkan-ML-Japan-Virtual-Open-House-Feb21.pdf https://www.khronos.org/assets/uploads/developers/presentati... [4] https://google.github.io/iree/deployment-configurations/gpu-vulkan/ https://google.github.io/iree/deployment-configurations/gpu-... [5] https://www.duskborn.com/posts/a-simple-vulkan-compute-example/ https://www.duskborn.com/posts/a-simple-vulkan-compute-examp...
- sorenjan 5y ago
  History has shown that it's not necessarily the most performant option that wins, it's usually the most convenient. Otherwise we wouldn't use Python for data science and ML, or Javascript for IDEs. Remeber AMD's Close To Metal? It wasn't a big hit. Take a look at the Blender announcement that's on the front page right now, Blender is removing OpenCL code and replacing it with HIP/Cuda to get a single codebase [0]. There's room for both solutions, but I think it's important to have a relatively easy way to use accelerators like GPUs in a cross platform way, without being an expert or having to rewrite code for new architectures. It is my understanding that because both Vulkan and SYCL use SPIR-V, the work done on drivers and compilers for one of them benefits the other as well. [0] https://news.ycombinator.com/item?id=29234671 https://news.ycombinator.com/item?id=29234671
  - rektide 5y ago
    AMD's close-to-the-metal was the blueprint for Vulkan, or at leat the majorest contributor (ed: oops, im thinking of AMD Mantle, a little bit latter & also extremely super low level). It won. It's ideas power almost all GPU technology today. Vulkans not relatively easy to do from scratch, but it is cross platform, and again, most folks dont need to write Vulkan. They're using ML frameworks that abstract that away. Vulkan will enable use of countless great extensions & capabilities that tools like SYCL wont be smart enough to use. Maybe the SPIR-V can be optimized by the drivers well, but the code SYCL spits out, i'd wager, will be world's less than what we do if we try. This is what CUDA does: it allows bad/cheap SYCL like code, but most folks use NV's vast ultra-complete libraries that are far far far better written in much lower level code & tweaked for every last oz of performance. CUDA is basically a scripting language + close to the metal library. SYCL might eventually be able to become similar, but only if we do the hard work of making really good Vulkan libraries. Otherwise it'll be pretty much trash. There's a lot of argumemts in favor of being bad, of not going for thr good stuff. But end of the day, we just should do the right thing. Everyone other than NV should make a real start, should make a bid for their & the survival of everyone else. Vulkan is the only real bid i see.
    - pjmlp 5y ago

> My hope is eventually we start to invest in Vulkan Compute. Vulkan is for graphics. Khronos' compute standard that's most similar to Cuda is SYCL. Both compile shaders to SPIR-V though.

rektide 5y ago

> Vulkan is for graphics. Incorrect. Quoting the spec: > Vulkan is an API (Application Programming Interface) for graphics and compute hardware. Vulkan has compute shaders[1], which are generally usable. Libraries like VkFFT[2] demonstrate basic signal processing in Vulkan. There are plenty of other non-graphical Compute Shader examples, and this is part of the design of Vulkan (and also WebGPU). Further, there is a Vulkan ML TSG (Technical Subgroup)[3], which is supposed to be working on ML. Even Nvidia is participating, with extensions like VK_NV_cooperative_matrix, which specifically target the ml tensor cores. A more complex & current example, we see works like Google's IREE, which allow inference/Tensorflow Lite execution on a variety of drivers, including Vulkan[4], which has broad portability across hardware & fairly decent performance, even on mobile chips. There's people could probably say this better/more specifically, but I'll give it a try: Vulkan is, above all, an general standard for modelling, dispatching & orchestrating work usually on a GPU. Right now that usage is predominately graphics, but that is far from a limit. The ideas of representing GPU resources, dispatching/queueing work are generic, apply fairly reasonably to all GPU systems, and can model any workload done on a GPU. A good general introduction to Vulkan Compute is this great write up here[5]: https://www.duskborn.com/posts/a-simple-vulkan-compute-example/ https://www.duskborn.com/posts/a-simple-vulkan-compute-examp... > Khronos' compute standard that's most similar to Cuda is SYCL. SYCL is, imo, the opposite of where we need to go. It's the old historical legacy that CUDA has, of writing really dumb ignorant code & hoping the tools can make it run well on a GPU. Vulkan, on the other hand, asks us to consider deeply what near-to-the-metal resources we are going to need, and demands that we define, dispatch, & complete the actual processing engines on the GPU that will do the work. It's a much much much harder task, but it invites in fantastic levels of close optimization & tuning, allows for far more advanced pipelining & possibilities. If the future is good, it should abandon lo-fi easy options like SYCL and CUDA, and bother to get good at Vulkan, which will allow us to work intimately with the GPU. This is a relationship worth forging, and no substitutes will cut it. [1] https://vkguide.dev/docs/gpudriven/compute_shaders/ https://vkguide.dev/docs/gpudriven/compute_shaders/ [2] https://github.com/DTolm/VkFFT https://github.com/DTolm/VkFFT [3] https://www.khronos.org/assets/uploads/developers/presentations/Khronos-Vulkan-ML-Japan-Virtual-Open-House-Feb21.pdf https://www.khronos.org/assets/uploads/developers/presentati... [4] https://google.github.io/iree/deployment-configurations/gpu-vulkan/ https://google.github.io/iree/deployment-configurations/gpu-... [5] https://www.duskborn.com/posts/a-simple-vulkan-compute-example/ https://www.duskborn.com/posts/a-simple-vulkan-compute-examp...

sorenjan 5y ago
History has shown that it's not necessarily the most performant option that wins, it's usually the most convenient. Otherwise we wouldn't use Python for data science and ML, or Javascript for IDEs. Remeber AMD's Close To Metal? It wasn't a big hit. Take a look at the Blender announcement that's on the front page right now, Blender is removing OpenCL code and replacing it with HIP/Cuda to get a single codebase [0]. There's room for both solutions, but I think it's important to have a relatively easy way to use accelerators like GPUs in a cross platform way, without being an expert or having to rewrite code for new architectures. It is my understanding that because both Vulkan and SYCL use SPIR-V, the work done on drivers and compilers for one of them benefits the other as well. [0] https://news.ycombinator.com/item?id=29234671 https://news.ycombinator.com/item?id=29234671
- rektide 5y ago
  AMD's close-to-the-metal was the blueprint for Vulkan, or at leat the majorest contributor (ed: oops, im thinking of AMD Mantle, a little bit latter & also extremely super low level). It won. It's ideas power almost all GPU technology today. Vulkans not relatively easy to do from scratch, but it is cross platform, and again, most folks dont need to write Vulkan. They're using ML frameworks that abstract that away. Vulkan will enable use of countless great extensions & capabilities that tools like SYCL wont be smart enough to use. Maybe the SPIR-V can be optimized by the drivers well, but the code SYCL spits out, i'd wager, will be world's less than what we do if we try. This is what CUDA does: it allows bad/cheap SYCL like code, but most folks use NV's vast ultra-complete libraries that are far far far better written in much lower level code & tweaked for every last oz of performance. CUDA is basically a scripting language + close to the metal library. SYCL might eventually be able to become similar, but only if we do the hard work of making really good Vulkan libraries. Otherwise it'll be pretty much trash. There's a lot of argumemts in favor of being bad, of not going for thr good stuff. But end of the day, we just should do the right thing. Everyone other than NV should make a real start, should make a bid for their & the survival of everyone else. Vulkan is the only real bid i see.
  - pjmlp 5y ago