10 ms·

I suspect this article may even be underestimating the impact of WebGPU. I'll make two observations. First, for AI and machine learning type workloads, the inf

I suspect this article may even be underestimating the impact of WebGPU. I'll make two observations. First, for AI and machine learning type workloads, the infrastructure situation is a big mess right now unless you buy into the Nvidia / CUDA ecosystem. If you're a research, you pretty much have to, but increasingly people will just want to run models that have already been trained. Fairly soon, WebGPU will be an alternative that more or less Just Works, although I do expect things to be rough in the early days. There's also a performance gap, but I can see it closing. Second, for compute shaders in general (potentially accelerating a large variety of tasks), the barrier to entry falls dramatically. That's especially true on web deployments, where running your own compute shader costs somewhere around 100 lines of code. But it becomes practical on native too, especially Rust where you can just pull in a wgpu dependency. As for text being one of the missing pieces, I'm hoping Vello and supporting infrastructure will become one of the things people routinely reach for. That'll get you not just text but nice 2D vector graphics with fills, strokes, gradients, blend modes, and so on. It's not production-ready yet, but I'm excited about the roadmap. [Note: very lightly adapted from a comment at cohost; one interesting response was by Tom Forsyth, suggesting I look into SYCL]

chrisjc 3y ago
This is the discussion I hoped to find when clicking on the comments. > Fairly soon, WebGPU will be an alternative... So while the blog focused on the graphical utility of WebGPU, the underlying implementation of WebGPU is currently about the way that websites/apps can now interface with the GPU in a more direct/advantageous way to render graphics. But what you're suggesting is that in the future new functionality will likely be added to take advantage of your GPU in other ways, such as training ML models and then using them via an inference engine all powered by your local GPU? Is the reason you can't accomplish that today bc APIs haven't been created or opened up to allow such workloads? Are there not lower level APIs available/exposed today in WebGPU that would allow developers to begin the design of browser based ML frameworks/libraries? Was it possible to interact with the GPU before WebGPU via Web Assembly? Other than ML and graphics/games (and someone is probably going to mention crypto), are there any other potentially novel uses for WebGPU?
- flohofwoe 3y ago
  > Was it possible to interact with the GPU before WebGPU via Web Assembly? Only with tons of restrictions by going through WebGL, or by writing your own or extending an existing WASM runtime outside the browser which connects to system-native GPU APIs like D3D12, Vulkan or Metal. > are there any other potentially novel uses for WebGPU In general, the only thing that WebGPU has over calling into D3D12, Vulkan or Metal directly is that it provides a common subset of those 3 APIs (so you don't need to write 2 or 3 implementations to cover all popular operating systems) and it's easier to use (at least compared to D3D12 and Vulkan). WebGPU is an important step into the right direction, but not a 'game changer' per se (at least outside the browser).
- raphlinus 3y ago
  > ...in the future new functionality will likely be added to take advantage of your GPU in other ways, such as training ML models and then using them via an inference engine all powered by your local GPU? Yes. > Is the reason you can't accomplish that today bc APIs haven't been created or opened up to allow such workloads? Are there not lower level APIs available/exposed today in WebGPU that would allow developers to begin the design of browser based ML frameworks/libraries? That is correct, there is no way before WebGPU to access compute capability of GPU hardware through the Web. There have been some hacks based on WebGL, but those are seriously limited. The fragmentation of the existing API space is a major reason we haven't seen as much progress on this. > Was it possible to interact with the GPU before WebGPU via Web Assembly? Only in limited ways through WebGL - no access to workgroup shared memory, ability to do random access writes to storage buffers, etc. > Other than ML and graphics/games (and someone is probably going to mention crypto), are there any other potentially novel uses for WebGPU? Yes! There is research on doing parallel compilers on GPU (Aaron Hsu's co-dfns as well as Voetter's work[1]). There's quite a bit of work on implementing Fourier transforms at extremely high throughput. Obviously, physics simulations and other scientific workloads are a good fit. To me, it feels like things are wide open. [1]: https://dl.acm.org/doi/pdf/10.1145/3528416.3530249 https://dl.acm.org/doi/pdf/10.1145/3528416.3530249
- ianpurton 3y ago
  > GPU in other ways, such as training ML models and then using them via an inference engine all powered by your local GPU? Have a look at wonnix https://github.com/webonnx/wonnx https://github.com/webonnx/wonnx A WebGPU-accelerated ONNX inference run-time written 100% in Rust, ready for native and the web
tikkun 3y ago
Could someone explain what kinds of useful things will become possible with it? I don't get it yet, but HN seems excited about it, so I'd like to understand it. What I get so far - running models that can be run on consumer sized GPUs will become easier, because users won't need to download a desktop app to do so. This is limited for now by the lack of useful models that can be run on consumer GPUs, but we'll get smaller models in the future. And it'll be easier to make visualizations, games, and VR apps in the browser. Is that right and what other popular use cases where people currently have to resort to WebGL or building a desktop app will get easier that I'm missing?
- sebzim4500 3y ago
  It's not so much that people want to run the models in the browser. They want to be able to write and publish one desktop app that e.g. runs a LLaMA quality model and runs decently across the hundreds of different GPUs that exist on their users' machines.
  - tikkun 3y ago
    So it's more of a CUDA alternative for building desktop apps that are GPU powered?
    - deleted 3y ago
      [deleted]
    - dm33tri 3y ago
      Nowhere near CUDA. Maybe OpenCL and Metal replacement because nobody bothers to support them, so just a fallback option for AMD and ARM chips.
      - tikkun 3y ago
        So then: If your app needs CUDA, you'd need to write it in CUDA. If you don't need CUDA, you'd write it for WebGPU instead? If you benefit from CUDA but don't strictly require it, then you can write it in CUDA with a WebGPU fallback. Is that right?
why_only_15 3y ago
WebGPU has no equivalent to tensor cores to my understanding; are there plans to add something like this? Or would this be "implementation sees matmul-like code; replaces with tensor core instruction". For optimal performance, my understanding is that you need tight control of e.g. shared memory as well -- is that possible with WebGPU? On NVIDIA GPUs, flops without tensor cores are ~1/10th flops with tensor cores, so this is a pretty big deal for inference and definitely for training.
- raphlinus 3y ago
  Shared memory, yes, with the goodies: atomics and barriers. We rely on that heavily in Vello, so we've pushed very hard on it. For example, WebGPU introduces the "workgroupUniformLoad" built-in, which lets you broadcast a value to all threads in the workgroup while not introducing potential unsafety. Tensor cores: I can't say there are plans to add it, but it's certainly something I would like to see. You need subgroups in place first, and there's been quite a bit of discussion[1] on that as a likely extension post-1.0. [1]: https://github.com/gpuweb/gpuweb/issues/3950 https://github.com/gpuweb/gpuweb/issues/3950
- jdashg 3y ago
  Yes, we expect to have a natural path towards explicit cooperative matrix multiply ops. If you have a wishlist, we have an issue tracker! ;) https://github.com/gpuweb/gpuweb/issues https://github.com/gpuweb/gpuweb/issues
Gordonjcp 3y ago
> unless you buy into the Nvidia / CUDA ecosystem Coming at it from a graphics processing perspective, working on a lot of video editing, it's annoying that just as GPUs start to become affordable as people turn their back on cryptobro idiocy and stop chasing the Dunning-Krugerrand, they've started to get expensive again because people want hardware-accelerated Eliza chatbots. Anyway your choices for GPU computing are OpenCL and CUDA. If you write your project in CUDA, you'll wish you'd used OpenCL. If you write your project in OpenCL, you'll wish you'd used CUDA.
- singhrac 3y ago
  I've never once regretted my decision to write GPU code in CUDA... I mean, I wish there were alternatives because being locked into Nvidia isn't fun, but CUDA is a great developer experience.
- garbagecoder 3y ago
  >the Dunning-Krugerrand Stolen. Genius. You made my day.
  - Gordonjcp 3y ago
    I stole it from someone else first, and I'm sorry to say I can't remember who. Credit where it's due, though, it's an awesome term.
    - fanf2 3y ago
      I think I first heard it from @cstross
- pjmlp 3y ago
  > If you write your project in CUDA, you'll wish you'd used OpenCL. Only if one is stuck in a C for GPGPUs mindset.
- MobiusHorizons 3y ago
  spir-v seems to be a real option anywhere with vulkan available
dwheeler 3y ago
I haven't tried it myself, but it looks like several are already looking at implementing machine learning with WebGPU, and that this is one of the goals of WebGPU. Some info I found: * "WebGPU powered machine learning in the browser with Apache TVM" - https://octoml.ai/blog/webgpu-powered-machine-learning-in-the-browser-with-apache-tvm/ https://octoml.ai/blog/webgpu-powered-machine-learning-in-th... * "Fastest DNN Execution Framework on Web Browser" https://mil-tokyo.github.io/webdnn/ https://mil-tokyo.github.io/webdnn/ * "Google builds WebGPU into Chrome to speed up rendering and AI tasks" https://siliconangle.com/2023/04/07/google-builds-webgpu-chrome-speed-rendering-ai-tasks/ https://siliconangle.com/2023/04/07/google-builds-webgpu-chr...
- wingman-jr 3y ago
  Also, Tensorflow.js WebGPU backend has been in the works for quite some time: https://github.com/tensorflow/tfjs/tree/master/tfjs-backend-webgpu https://github.com/tensorflow/tfjs/tree/master/tfjs-backend-...
flockonus 3y ago
The thought also come to mind, but after listening the work of Neural Magic At Practical AI [1], and how the work with model quantization over CPU is advancing by leaps and bounds, I don't foresee the strong dependence we have on CUDA persisting, even in the near future. https://twitter.com/neuralmagic/status/1653774178099625986 https://twitter.com/neuralmagic/status/1653774178099625986
kragen 3y ago
Is Vello called that because 2-D graphics are very difficult, i.e., "hairy"?

chrisjc 3y ago

This is the discussion I hoped to find when clicking on the comments. > Fairly soon, WebGPU will be an alternative... So while the blog focused on the graphical utility of WebGPU, the underlying implementation of WebGPU is currently about the way that websites/apps can now interface with the GPU in a more direct/advantageous way to render graphics. But what you're suggesting is that in the future new functionality will likely be added to take advantage of your GPU in other ways, such as training ML models and then using them via an inference engine all powered by your local GPU? Is the reason you can't accomplish that today bc APIs haven't been created or opened up to allow such workloads? Are there not lower level APIs available/exposed today in WebGPU that would allow developers to begin the design of browser based ML frameworks/libraries? Was it possible to interact with the GPU before WebGPU via Web Assembly? Other than ML and graphics/games (and someone is probably going to mention crypto), are there any other potentially novel uses for WebGPU?

flohofwoe 3y ago
> Was it possible to interact with the GPU before WebGPU via Web Assembly? Only with tons of restrictions by going through WebGL, or by writing your own or extending an existing WASM runtime outside the browser which connects to system-native GPU APIs like D3D12, Vulkan or Metal. > are there any other potentially novel uses for WebGPU In general, the only thing that WebGPU has over calling into D3D12, Vulkan or Metal directly is that it provides a common subset of those 3 APIs (so you don't need to write 2 or 3 implementations to cover all popular operating systems) and it's easier to use (at least compared to D3D12 and Vulkan). WebGPU is an important step into the right direction, but not a 'game changer' per se (at least outside the browser).
raphlinus 3y ago
> ...in the future new functionality will likely be added to take advantage of your GPU in other ways, such as training ML models and then using them via an inference engine all powered by your local GPU? Yes. > Is the reason you can't accomplish that today bc APIs haven't been created or opened up to allow such workloads? Are there not lower level APIs available/exposed today in WebGPU that would allow developers to begin the design of browser based ML frameworks/libraries? That is correct, there is no way before WebGPU to access compute capability of GPU hardware through the Web. There have been some hacks based on WebGL, but those are seriously limited. The fragmentation of the existing API space is a major reason we haven't seen as much progress on this. > Was it possible to interact with the GPU before WebGPU via Web Assembly? Only in limited ways through WebGL - no access to workgroup shared memory, ability to do random access writes to storage buffers, etc. > Other than ML and graphics/games (and someone is probably going to mention crypto), are there any other potentially novel uses for WebGPU? Yes! There is research on doing parallel compilers on GPU (Aaron Hsu's co-dfns as well as Voetter's work[1]). There's quite a bit of work on implementing Fourier transforms at extremely high throughput. Obviously, physics simulations and other scientific workloads are a good fit. To me, it feels like things are wide open. [1]: https://dl.acm.org/doi/pdf/10.1145/3528416.3530249 https://dl.acm.org/doi/pdf/10.1145/3528416.3530249
ianpurton 3y ago
> GPU in other ways, such as training ML models and then using them via an inference engine all powered by your local GPU? Have a look at wonnix https://github.com/webonnx/wonnx https://github.com/webonnx/wonnx A WebGPU-accelerated ONNX inference run-time written 100% in Rust, ready for native and the web

tikkun 3y ago

Could someone explain what kinds of useful things will become possible with it? I don't get it yet, but HN seems excited about it, so I'd like to understand it. What I get so far - running models that can be run on consumer sized GPUs will become easier, because users won't need to download a desktop app to do so. This is limited for now by the lack of useful models that can be run on consumer GPUs, but we'll get smaller models in the future. And it'll be easier to make visualizations, games, and VR apps in the browser. Is that right and what other popular use cases where people currently have to resort to WebGL or building a desktop app will get easier that I'm missing?

sebzim4500 3y ago
It's not so much that people want to run the models in the browser. They want to be able to write and publish one desktop app that e.g. runs a LLaMA quality model and runs decently across the hundreds of different GPUs that exist on their users' machines.
- tikkun 3y ago
  So it's more of a CUDA alternative for building desktop apps that are GPU powered?
  - deleted 3y ago
    [deleted]
  - dm33tri 3y ago
    Nowhere near CUDA. Maybe OpenCL and Metal replacement because nobody bothers to support them, so just a fallback option for AMD and ARM chips.
    - tikkun 3y ago
      So then: If your app needs CUDA, you'd need to write it in CUDA. If you don't need CUDA, you'd write it for WebGPU instead? If you benefit from CUDA but don't strictly require it, then you can write it in CUDA with a WebGPU fallback. Is that right?

why_only_15 3y ago

WebGPU has no equivalent to tensor cores to my understanding; are there plans to add something like this? Or would this be "implementation sees matmul-like code; replaces with tensor core instruction". For optimal performance, my understanding is that you need tight control of e.g. shared memory as well -- is that possible with WebGPU? On NVIDIA GPUs, flops without tensor cores are ~1/10th flops with tensor cores, so this is a pretty big deal for inference and definitely for training.

raphlinus 3y ago
Shared memory, yes, with the goodies: atomics and barriers. We rely on that heavily in Vello, so we've pushed very hard on it. For example, WebGPU introduces the "workgroupUniformLoad" built-in, which lets you broadcast a value to all threads in the workgroup while not introducing potential unsafety. Tensor cores: I can't say there are plans to add it, but it's certainly something I would like to see. You need subgroups in place first, and there's been quite a bit of discussion[1] on that as a likely extension post-1.0. [1]: https://github.com/gpuweb/gpuweb/issues/3950 https://github.com/gpuweb/gpuweb/issues/3950
jdashg 3y ago
Yes, we expect to have a natural path towards explicit cooperative matrix multiply ops. If you have a wishlist, we have an issue tracker! ;) https://github.com/gpuweb/gpuweb/issues https://github.com/gpuweb/gpuweb/issues

Gordonjcp 3y ago

> unless you buy into the Nvidia / CUDA ecosystem Coming at it from a graphics processing perspective, working on a lot of video editing, it's annoying that just as GPUs start to become affordable as people turn their back on cryptobro idiocy and stop chasing the Dunning-Krugerrand, they've started to get expensive again because people want hardware-accelerated Eliza chatbots. Anyway your choices for GPU computing are OpenCL and CUDA. If you write your project in CUDA, you'll wish you'd used OpenCL. If you write your project in OpenCL, you'll wish you'd used CUDA.

singhrac 3y ago
I've never once regretted my decision to write GPU code in CUDA... I mean, I wish there were alternatives because being locked into Nvidia isn't fun, but CUDA is a great developer experience.
garbagecoder 3y ago
>the Dunning-Krugerrand Stolen. Genius. You made my day.
- Gordonjcp 3y ago
  I stole it from someone else first, and I'm sorry to say I can't remember who. Credit where it's due, though, it's an awesome term.
  - fanf2 3y ago
    I think I first heard it from @cstross
pjmlp 3y ago
> If you write your project in CUDA, you'll wish you'd used OpenCL. Only if one is stuck in a C for GPGPUs mindset.
MobiusHorizons 3y ago
spir-v seems to be a real option anywhere with vulkan available

dwheeler 3y ago

I haven't tried it myself, but it looks like several are already looking at implementing machine learning with WebGPU, and that this is one of the goals of WebGPU. Some info I found: * "WebGPU powered machine learning in the browser with Apache TVM" - https://octoml.ai/blog/webgpu-powered-machine-learning-in-the-browser-with-apache-tvm/ https://octoml.ai/blog/webgpu-powered-machine-learning-in-th... * "Fastest DNN Execution Framework on Web Browser" https://mil-tokyo.github.io/webdnn/ https://mil-tokyo.github.io/webdnn/ * "Google builds WebGPU into Chrome to speed up rendering and AI tasks" https://siliconangle.com/2023/04/07/google-builds-webgpu-chrome-speed-rendering-ai-tasks/ https://siliconangle.com/2023/04/07/google-builds-webgpu-chr...

wingman-jr 3y ago
Also, Tensorflow.js WebGPU backend has been in the works for quite some time: https://github.com/tensorflow/tfjs/tree/master/tfjs-backend-webgpu https://github.com/tensorflow/tfjs/tree/master/tfjs-backend-...

flockonus 3y ago

The thought also come to mind, but after listening the work of Neural Magic At Practical AI [1], and how the work with model quantization over CPU is advancing by leaps and bounds, I don't foresee the strong dependence we have on CUDA persisting, even in the near future. https://twitter.com/neuralmagic/status/1653774178099625986 https://twitter.com/neuralmagic/status/1653774178099625986

kragen 3y ago

Is Vello called that because 2-D graphics are very difficult, i.e., "hairy"?