5 ms·

Slightly OT: I never heard of webGPU before. So, in theory it will be feasible in a few years to run models like Stable Diffusion on my GPU through the browser

by planb 4y ago

Slightly OT: I never heard of webGPU before. So, in theory it will be feasible in a few years to run models like Stable Diffusion on my GPU through the browser without fighting with conda, pip and drivers? Or did I get the scope of this wrong?

Tade0 4y ago
It's a replacement for WebGL, but the key difference is that it supports compute shaders.
- boppo1 4y ago
  Is it a replacement? It looks more like a framework that lives on top of webgl.
  - incrudible 4y ago
    WebGPU is to WebGL as Vulkan is to OpenGL, a lower level, low overhead graphics API.
  - msk-lywenn 4y ago
    It's the other way around. WebGPU is to Vulkan what WebGL is to OpenGL. It's less low level than vulkan though.
  - danielvaughn 4y ago
    You shouldn't be downvoted, your question is reasonable even if it demonstrates a misunderstanding. So it's not a wrapper over WebGL, it's a more modern, lower-level GPU programming model. There are equivalent efforts in native environments to replace OpenGL - notably Vulkan and (Apple's) Metal. As far as I understand, the approach allows far more flexibility, at the expense of higher complexity. It's less "stateful" than WebGL, which basically gives you a big class that manages everything OOP-style.
- irate2942 4y ago
  WebGL2 also supports them! And now that Apple finally added support (like half a year ago(?)) to it, it can be fairly safely used
  - raphlinus 4y ago
    I believe WebGL compute shader development has been halted[1], in favor of making WebGPU happen. Though it's possible to run it in dev builds in some cases, I'm pretty sure there's no browser that has it on by default, and likely this won't happen. Apple does support WebGL2, but compute shader support is not part of that core spec. The demos[2] certainly don't work in Safari in my quick test. [1]: https://registry.khronos.org/webgl/specs/latest/2.0-compute/ https://registry.khronos.org/webgl/specs/latest/2.0-compute/ [2]: https://9ballsyndrome.github.io/WebGL_Compute_shader/webgl-compute-boids/dist/ https://9ballsyndrome.github.io/WebGL_Compute_shader/webgl-c...
    - pjmlp 4y ago
      Thanks Google that dropped them out of Chrome and we all know how the Web is basically whatever Chrome does, so WebGPU it is.
- unconed 4y ago
  It also supports many more features in rendering which previously were unavailable or so badly supported you couldn't use them. It's basically like going from 2005 to 2015 in terms of capabilities. There are still notable gaps, like no bindless resources or raytracing though. So don't expect e.g. unreal engine to run on WebGPU without some significant compromises.
  - markdog12 4y ago
    Someone patched Chrome to enable raytracing with WebGPU: https://github.com/maierfelix/chromium-ray-tracing https://github.com/maierfelix/chromium-ray-tracing
carpenecopinum 4y ago
In principle, yes. With webGPU (but also WebGL2) we get access to compute shaders, which makes it feasible/easier to evaluate a model like Stable Diffusion. The biggest issue I see is that those models (or rather their trained parameters) are usually pretty large (several GiB). So it'll take a while to set up in the browser before the evaluation can actually start. It'll also require a lot of bandwidth on both ends. A lot of those things should already be doable with the fragment shaders we get from WebGL and a lot of hackery, like clever (ab)use of textures. So the actual issue that we're not seeing a lot of this is probably not due to it being impossible right now...
- capableweb 4y ago
  > The biggest issue I see is that those models (or rather their trained parameters) are usually pretty large (several GiB). So it'll take a while to set up in the browser before the evaluation can actually start. It'll also require a lot of bandwidth on both ends. Might not be feasible due to memory constraints (I'm not sure), but browsers can load data from disk into memory without having to touch the network. So you could in theory ask the users to download the model separately, then ask them to select it from a file picker, and the browser can have access to it that way.
  - flyteuk 4y ago
    It'd be nice if the models could be loaded via WebTorrent[1], which would significantly reduce the bandwidth requirement for whomever releases it. [1] https://github.com/webtorrent/webtorrent https://github.com/webtorrent/webtorrent
    - tough 4y ago
      The models are certainly available on torrent though
  - lmeyerov 4y ago
    Next step: browser JS VMs need to allow you access gigs of RAM in-memory, it's surprisingly locked down
modeless 4y ago
You can already run ML models with GPU acceleration in WebGL using tensorflow.js and other libraries. WebGPU will make things better for sure, but I think the major obstacles to running large models like Stable Diffusion will not be solved by WebGPU. WebGPU will not expose Nvidia-specific tensor cores, at least initially. But the main issues are with loading and processing gigabytes of data in browsers, which aren't addressed by WebGPU at all. You'll have difficulty downloading and storing all that data and quickly run into out of memory crashes.

by planb 4y ago

Tade0 4y ago

It's a replacement for WebGL, but the key difference is that it supports compute shaders.

boppo1 4y ago
Is it a replacement? It looks more like a framework that lives on top of webgl.
- incrudible 4y ago
  WebGPU is to WebGL as Vulkan is to OpenGL, a lower level, low overhead graphics API.
- msk-lywenn 4y ago
  It's the other way around. WebGPU is to Vulkan what WebGL is to OpenGL. It's less low level than vulkan though.
- danielvaughn 4y ago
  You shouldn't be downvoted, your question is reasonable even if it demonstrates a misunderstanding. So it's not a wrapper over WebGL, it's a more modern, lower-level GPU programming model. There are equivalent efforts in native environments to replace OpenGL - notably Vulkan and (Apple's) Metal. As far as I understand, the approach allows far more flexibility, at the expense of higher complexity. It's less "stateful" than WebGL, which basically gives you a big class that manages everything OOP-style.
irate2942 4y ago
WebGL2 also supports them! And now that Apple finally added support (like half a year ago(?)) to it, it can be fairly safely used
- raphlinus 4y ago
  I believe WebGL compute shader development has been halted[1], in favor of making WebGPU happen. Though it's possible to run it in dev builds in some cases, I'm pretty sure there's no browser that has it on by default, and likely this won't happen. Apple does support WebGL2, but compute shader support is not part of that core spec. The demos[2] certainly don't work in Safari in my quick test. [1]: https://registry.khronos.org/webgl/specs/latest/2.0-compute/ https://registry.khronos.org/webgl/specs/latest/2.0-compute/ [2]: https://9ballsyndrome.github.io/WebGL_Compute_shader/webgl-compute-boids/dist/ https://9ballsyndrome.github.io/WebGL_Compute_shader/webgl-c...
  - pjmlp 4y ago
    Thanks Google that dropped them out of Chrome and we all know how the Web is basically whatever Chrome does, so WebGPU it is.
unconed 4y ago
It also supports many more features in rendering which previously were unavailable or so badly supported you couldn't use them. It's basically like going from 2005 to 2015 in terms of capabilities. There are still notable gaps, like no bindless resources or raytracing though. So don't expect e.g. unreal engine to run on WebGPU without some significant compromises.
- markdog12 4y ago
  Someone patched Chrome to enable raytracing with WebGPU: https://github.com/maierfelix/chromium-ray-tracing https://github.com/maierfelix/chromium-ray-tracing

carpenecopinum 4y ago

In principle, yes. With webGPU (but also WebGL2) we get access to compute shaders, which makes it feasible/easier to evaluate a model like Stable Diffusion. The biggest issue I see is that those models (or rather their trained parameters) are usually pretty large (several GiB). So it'll take a while to set up in the browser before the evaluation can actually start. It'll also require a lot of bandwidth on both ends. A lot of those things should already be doable with the fragment shaders we get from WebGL and a lot of hackery, like clever (ab)use of textures. So the actual issue that we're not seeing a lot of this is probably not due to it being impossible right now...

capableweb 4y ago
> The biggest issue I see is that those models (or rather their trained parameters) are usually pretty large (several GiB). So it'll take a while to set up in the browser before the evaluation can actually start. It'll also require a lot of bandwidth on both ends. Might not be feasible due to memory constraints (I'm not sure), but browsers can load data from disk into memory without having to touch the network. So you could in theory ask the users to download the model separately, then ask them to select it from a file picker, and the browser can have access to it that way.
- flyteuk 4y ago
  It'd be nice if the models could be loaded via WebTorrent[1], which would significantly reduce the bandwidth requirement for whomever releases it. [1] https://github.com/webtorrent/webtorrent https://github.com/webtorrent/webtorrent
  - tough 4y ago
    The models are certainly available on torrent though
- lmeyerov 4y ago
  Next step: browser JS VMs need to allow you access gigs of RAM in-memory, it's surprisingly locked down

modeless 4y ago

You can already run ML models with GPU acceleration in WebGL using tensorflow.js and other libraries. WebGPU will make things better for sure, but I think the major obstacles to running large models like Stable Diffusion will not be solved by WebGPU. WebGPU will not expose Nvidia-specific tensor cores, at least initially. But the main issues are with loading and processing gigabytes of data in browsers, which aren't addressed by WebGPU at all. You'll have difficulty downloading and storing all that data and quickly run into out of memory crashes.